\documentclass[main.tex]{subfiles}
\begin{document}

\marginpar{Saturday\\ 2020-3-14, \\ compiled \\ \today}

\section{Radiative energy density}

We define the energy density per unit solid angle, \(u_{\nu }(\Omega )\) by: \(\dd{E} = u_{\nu } (\Omega ) \dd{V} \dd{\Omega } \dd{\nu }\).
This is the differential amount of energy in the volume \(\dd{V}\), carried by radiation coming from the solid angle \(\dd{\Omega }\) which has energies between  \(\nu \) and \(\nu + \dd{\nu }\).

We consider a cylinder for our volume, its axis being aligned with the direction the radiation is coming from.
Its volume can be expressed as \(\dd{V} =\dd{A} c \dd{t}\), where \(\dd{t}\) is the time taken by light to cross the height of the cylinder.

We can also express the differential energy using the definition of the specific intensity: then, we can compare the following two equations:
%
\begin{subequations}
\begin{align}
\dd{E} &= u_{\nu } (\Omega ) c \dd{A} \dd{t} \dd{\Omega } \dd{\nu }  \\
&= I_{\nu } \dd{A} \dd{t} \dd{\Omega } \dd{\nu }
\,,
\end{align}
\end{subequations}
%
which allows us to conclude that \(u_{\nu } = I_{ \nu } / c\).
Also, if we want to get the total energy density we just need to integrate over the volume of the whole sphere:
%
\begin{align}
u_{\nu } = \frac{1}{c} \int I_{\nu }  (\Omega ) \dd{\Omega }
\overset{\text{def}}{=} \frac{4 \pi }{c} J_{\nu }
\,,
\end{align}
%
where \(J_{\nu  }\) is the \emph{mean intensity}: \(J_{\nu } = \expval{I_{\nu }}_{\Omega }\).

We can also integrate over frequencies to get the total energy density:
%
\begin{align}
u = \int u_{\nu } \dd{\nu } = \frac{4 \pi }{c} \int J_{\nu } \dd{\nu }
\,.
\end{align}

\subsection{Isotropic radiation field}

An isotropic radiation field is one for which the specific intensity does not depend on angles.
Let us start from the definitions of \(u_{\nu }\) and \(P_{\nu }\):
%
\begin{subequations}
\begin{align}
u_{\nu } &= \int \frac{I_{\nu }}{c} \dd{\Omega } = \frac{4 \pi J_{\nu }}{c}  \\
P_{\nu } &= \int  \frac{I_{\nu }}{c}  \cos^2\theta \dd{\Omega } =
\int \frac{I_{\nu }}{c} \cos^2\theta \sin \theta \dd{\theta } \dd{\varphi }
\,,
\end{align}
\end{subequations}
%
and let us use the assumption that \(I_\nu \) does not depend on \(\Omega \): so we can bring it out of the integrals, to find
%
\begin{subequations}
\begin{align}
u_{\nu } &= 4 \pi  \frac{I_{\nu }}{c}  \\
P_{\nu } &= - 2 \pi  \frac{I_{\nu }}{c} \int \cos^2\theta \dd{\cos \theta } = 2 \pi \frac{I_{\nu }}{c} \int_{-1}^{1} x^2 \dd{x}  \marginnote{The differential is negative, but we swap the integration bounds.}\\
&= \frac{4 \pi }{3} \frac{I_{\nu }}{c}
\,,
\end{align}
\end{subequations}
%
which gives us the result we sought:
%
\begin{align}
P_{\nu } = \frac{u_{\nu }}{3}
\,.
\end{align}

\subsection{Specific intensity along a ray}

We wish to see how the specific intensity \(I_{\nu }\) changes along a beam of light rays. Let us consider two positions \(1, 2\) along the beam, separated by a distance \(R\). Then, by definition we will have, for \(i =1, 2\):
%
\begin{align}
\dd{E_{i}} = I_{\nu, i} \dd{A_{i}} \dd{t_{i}} \dd{\Omega_{i}} \dd{\nu_{i}}
\,.
\end{align}

First, we make the assumption of the gravitational field being weak: therefore time dilation is negligible, so \(\dd{t_{1}} = \dd{t_2 }\) and \(\dd{\nu_{1}} = \dd{\nu_2 }\).
Now, we ask these two expressions to describe the same beam: the same photons will pass through \(\dd{A_1 }\) and  \(\dd{A_2 }\).
Therefore, by conservation of energy, \(\dd{E_1 } = \dd{E_2 }\).

This means that
%
\begin{align}
I_{\nu, 1} \dd{A_1 } \dd{\Omega_1 } =
I_{\nu, 2} \dd{A_2 } \dd{\Omega_2 }
\,.
\end{align}
%

We can treat the photons' motion as time-reversal symmetric: so, whether they pass through \(\dd{A_1 }\) or \(\dd{A_2 }\) first is irrelevant.
The linear scale of the differential area element is infinitesimal, while the separation between the two points, \(R\), is macroscopic: so, we can consider all the photons which come through \(\dd{A_1}\) to be coming from a point source from position \(2\) and vice versa.
This will define the angular size as seen from the ``pointlike'' position 2, \(\dd{\Omega_2 }\).

Therefore, the differential solid angle will look like
%
\begin{align}
\dd{\Omega_2 } = \frac{ \dd{A_1 } }{R^2}
\,,
\end{align}
%
and we can apply the same reasoning reversing the photons' motion to find the same, alternate relation with \((1 \leftrightarrow 2)\).
We can use this to write
%
\begin{subequations}
\begin{align}
I_{\nu , 1} \frac{ \dd{A_1 }}{ \dd{\Omega_2 }} &=
I_{\nu , 2} \frac{ \dd{A_2 }}{ \dd{\Omega_1 }}  \\
I_{\nu ,1 } R^2 &= I_{\nu , 2} R^2  \\
I_{\nu , 1} &= I_{\nu , 2}
\,.
\end{align}
\end{subequations}

This means that, under our assumptions, the specific intensity is conserved:
%
\begin{align}
\dv{I_{\nu }}{s} = 0
\,,
\end{align}
%
where \(s\) is a parameter describing the light ray's trajectory.
This is useful since, if the variation of the specific intensity is zero in a vacuum, then its variation in the presence of matter will only be due to transfer phenomena, and the sign of the variation will describe whether energy is being added or removed.

\section{Radiative transfer}

In general, as radiation passes through matter, its specific intensity changes.
This is due to emission and absorption, but also to scattering, which preserves the total number of photons: even in the low-energy limit it can change the angular distribution of the radiation, and in general it also changes the energy of the photon.

\subsection{Emission}

Emission is a process through which photons are created.
We can define the grey emission coefficient \(j\) and the monochromatic emission coefficient \(j_{\nu }\) as:
%
\begin{subequations}
\begin{align}
\dd{E} &= j \dd{V} \dd{\Omega } \dd{t}  \\
\dd{E} &= j_{\nu } \dd{V} \dd{\Omega } \dd{t} \dd{\nu }
\,,
\end{align}
\end{subequations}
%
they quantify the energy added to the radiation field per unit volume, solid angle (in order to account for the direction of emission) and unit time. For the monochromatic coefficient, we restrict ourselves to radiation emitted in the range from \(\nu \) to \(\nu + \dd{\nu }\).

In the case of an isotropic emission we can integrate over the solid angle to find
%
\begin{align}
P_{\nu } = 4 \pi j_{\nu }
\,,
\end{align}
%
the radiated power per unit volume and frequency.

Another useful concept is the emissivity \(\epsilon_{\nu }\): it is the energy added to the radiation field per unit time, frequency and mass in the directions described by the solid angle \(\dd{\Omega }\).
We express the infinitesimal mass as \(\dd{m} = \rho \dd{V}\), so that in the case of isotropic emission we have
%
\begin{align}
\dd{E} = \epsilon_{\nu } \rho \dd{V} \dd{t} \dd{\nu } \frac{ \dd{\Omega }}{4\pi }
\,,
\end{align}
%
so the emissivity \(\epsilon_{\nu }\) and the emission coefficient \(j_{\nu }\) are connected by
%
\begin{align}
j_{\nu } = \frac{\epsilon_{\nu } \rho }{4 \pi }
\,.
\end{align}

We wish to describe the variation in specific intensity due to this emission.
Let us consider a beam of cross section \(\dd{A}\) going through a length \(\dd{s}\), so that the volume it occupies is \(\dd{V} = \dd{A} \dd{s}\).

Now, if we compare the definitions of \(j_{\nu }\) and \(I_{\nu }\) we find that they differ by a factor \(\dd{V} / \dd{A} = \dd{s}\), the length of the beam cylinder we defined.

The difference between the specific intensities at the start and end of the cylinder would be zero without emission, now instead their difference can be calculated from the energy added; as we said most of the differentials simplify and we get that the variation of specific intensity is
%
\begin{align}
\dd{I_{\nu }} = j_{\nu } \dd{s}
\,.
\end{align}

\subsection{Absorption}

Absorption is described by a coefficient \(\alpha_{\nu }>0\), which is dimensionally an inverse length.
The absorption law which defines the coefficient gives the decrease in radiative intensity for radiation of intensity \(I_{\nu }\) crossing an absorbing medium of length \(\dd{s}\):
%
\begin{align}
\dd{I_{\nu }} =- \alpha_{\nu } I_{\nu } \dd{s}
\,.
\end{align}

Why should the variation in intensity be proportional to the intensity itself? We give a simple argument:
let us assume that absorption is due to randomly absorbers with number density \(n\) and (frequency dependent) cross section \(\sigma_{\nu }\).

Let us consider our usual cylinder with cross sectional area \(\dd{A}\) and length \(\dd{s}\): the number of absorbers in it will be \(\dd{N} = n \dd{A} \dd{s}\).
The total effective cross section area presented for absorption will be \(\sigma_{\nu } \dd{N}\).
The energy contained in photons in this cross sectional area will be lost: the energy lost \(- \dd{I_{\nu }}\) can be calculated as
%
\begin{subequations}
\begin{align}
- \dd{I_{\nu }} \dd{A} \dd{t} \dd{\Omega} \dd{\nu }
&=  I_{\nu } \qty(\sigma_{\nu } n \dd{A} \dd{s}) \dd{t} \dd{\Omega } \dd{\nu }  \\
\dd{I_{\nu }} &= - n \sigma_{\nu } I_{\nu } \dd{s}
\,,
\end{align}
\end{subequations}
%
which is the relation written above, with \(n \sigma_{\nu } = \alpha_{\nu }\).
The number density is proportional to the mass density: \(n \overline{m} = \rho \) where \(\overline{m} \) is the average mass of a particle. Therefore, we can express \(\alpha_{\nu }\) as
%
\begin{align}
\alpha_{\nu } = \rho \kappa_{\nu } = n \overline{m} \frac{ \sigma_{\nu }}{\overline{m}}
\,,
\end{align}
%
so we can see that \(\kappa_{\nu }\) is a cross sectional area per unit mass. It is called the \emph{mass absorption coefficient} or the \emph{opacity}.

\paragraph{Conditions for validity}

This line of reasoning holds as long as the inter-absorber distances \(d \sim n^{-1/3}\) are large compared to the linear scale of the cross section \(\sigma_{\nu }^{1/2}\): we ask
%
\begin{align}
\sigma_{\nu }^{1/2} \ll n^{-1/3}
\,,
\end{align}
%
and also we must assume that the absorbers are independent and randomly distributed (at least locally).
These assumptions are usually met in astrophysical systems.

\subsection{The radiative transfer equation}

We can account for both absorption and emission in a combined equation for the derivative with respect to the beam length travelled \(s\) of the specific intensity \(I_{\nu }\):
%
\begin{align}
\dv{I_{\nu }}{s} = - \alpha_{\nu } I_{\nu } + j_{\nu }
\,,
\end{align}
%
and we can see that in the absence of emission and absorption \(I_{\nu }\) is unchanged, as we have shown before.
If \(j_{\nu }\) and \(\alpha_{\nu }\) are known we can integrate this differential equation to find the specific intensity.

This will \emph{not} be the case when we will include scattering: the scattering term will not just depend on \(I_{\nu }\) but in its integral on the sphere, making this an integro-differential equation.

\subsubsection{Solutions to the transfer equation in simple cases}

If there is only emission, that is, only \(j_{\nu }\) is nonzero, the intensity increases (linearly in \(s\) for constant \(j_{\nu }\)):
%
\begin{align}
\dv{I_{\nu }}{s} = j_{\nu } \implies
I_{\nu } = I_{\nu }(0) + \int_{s_0 }^{s} j_{\nu } (\widetilde{s}) \dd{\widetilde{s}}
\,.
\end{align}

If there is only absorption, that is, only \(\alpha_{\nu }\) is nonzero, then the intensity decreases (exponentially in \(s\) for constant \(j_{\nu }\)):
%
\begin{align}
\dv{I_{\nu }}{s} = - \alpha_{\nu } I_{\nu } \implies
I_{\nu } = I_{\nu }(s_0 ) \exp(- \int_{s_0 }^{s} \alpha_{\nu }(\widetilde{s}) \dd{\widetilde{s}})
\,.
\end{align}

\subsection{Optical depth and source function}

The optical depth \(\tau_{\nu }\) is defined so that it increases by 1 when the intensity of light at frequency \(\nu \) decreases \(e\)-fold: its differential is
%
\begin{align}
\dd{\tau_{\nu }} = \alpha_{\nu } \dd{s}
\,,
\end{align}
%
so that the solution in the absorption-only case reads \(I_{\nu } \propto e^{-\int \dd{\tau }} = e^{-\tau }\).

So, a useful distinction to make is based on the magnitude of \(\tau \), since it quantifies how much light can shine through a medium:
\begin{enumerate}
  \item if \(\tau \gg 1\) the medium is said to be \emph{opaque} or \emph{optically thick};
  \item if \(\tau \ll 1\) the medium is said to be \emph{transparent} or \emph{optically thin};
  \item if \(\tau \approx 1\) the medium is said to be \emph{translucent}.
\end{enumerate}

If we define the source function
%
\begin{align}
S_{\nu } = \frac{j_{\nu }}{\alpha_{\nu }}
\,,
\end{align}
%
we can write the radiative transfer equation as
%
\begin{align}
\dv{I_{\nu }}{\tau_{\nu }} = - I_{\nu } + S_{\nu } \marginnote{Divided through by \(\alpha_{\nu }\), used definition of \(\tau_{\nu }\).}
\,.
\end{align}

\subsection{A formal solution of the radiative transfer equation}

We can solve this equation by defining \(Y_{\nu } = I_{\nu }e^{\tau_{\nu }}\), which obeys
%
\begin{align}
\dv{Y_{\nu }}{\tau_{\nu }} = \dv{I_{\nu }}{\tau_{\nu }} e^{\tau_{\nu  }} + I_{\nu } e^{\tau_{\nu }}
\,,
\end{align}
%
so we can multiply the radiative transport equation by \(e^{\tau_{\nu }}\) to get
%
\begin{subequations}
\begin{align}
\dv{I_{\nu }}{\tau_{\nu }} e^{\tau_{\nu }} &= - I_{\nu }e^{\tau_{\nu }} + S_{\nu } e^{\tau_{\nu }}  \\
\dv{Y_{\nu }}{\tau_{\nu }} &= S_{\nu } e^{\tau_{\nu }}  \\
Y_{\nu }(\tau_{\nu }) &= Y_{\nu } (0 ) +  \int_{0}^{\tau_{\nu }} S_{\nu } (\widetilde{\tau}_{\nu }) e^{\widetilde{\tau}_{\nu }} \dd{\widetilde{\tau}_{\nu }}  \\
I_{\nu }(\tau_{\nu }) &= I_{\nu } (0) e^{-\tau_{\nu }}
+ \int_{0}^{\tau_{\nu }} S_{\nu }(\widetilde{\tau}_{\nu }) e^{\widetilde{\tau}_{\nu } - \tau_{\nu }} \dd{\widetilde{\tau}_{\nu }} \marginnote{Divided through by \(e^{\tau_{\nu }}\).}
\,,
\end{align}
\end{subequations}
%
which has a direct intuitive meaning: the intensity at a certain point must be computed accounting for the initial one and emission all through the beam before the point we are considering, and each of these contributions to the emission is weighted by an exponential factor: the relevance of a term decreases if the optical distance increases.

If \(S_{\nu } \) is a constant, we have the simplified expression
%
\begin{align}
I_{\nu } (\tau_{\nu }) = I_{\nu }(0) e^{-\tau_{\nu }}
+ S_{\nu } \qty(1 - e^{-\tau_{\nu }})
\,,
\end{align}
%
and we can see that for large optical depths the intensity is dictated purely by the source at that point, since \(1 - e^{-\tau_{\nu }}\) approaches 0 for large \(\tau_{\nu }\).

\end{document}


\documentclass[t]{beamer}
\title{Building Recommender Systems}
\subtitle{Lecture 7}
\author{Boris Shminke}
\institute{Université Côte d'Azur, CNRS, LJAD, France}
\date{03/01/2021}
\AtBeginSection[]{
  \begin{frame}
    \frametitle{Outline}
    \tableofcontents[currentsection]
  \end{frame}
}
\begin{document}
\begin{frame}
  \titlepage
\end{frame}
\begin{frame}
  \frametitle{Outline}
  \tableofcontents
\end{frame}
\section{Introduction}
\begin{frame}
  \frametitle{What is cold-start?}
  \begin{itemize}
  \item collaborative filtering works better than content based
  \item but it demands a user having long enough history
  \item what to recommend to new users (with a few or no interactions)?
  \item how to recommend new or less known items?
  \end{itemize}
\end{frame}
\section{Cold Start in Neighbourhood-based Recommendations}
\begin{frame}
  \frametitle{Neighbourhood-based Recommender}
  $$\hat{r}_{ui}=\sum\limits_jS_{ij}r_{uj}$$

  where

  $r_{uj}$ --- (observed) rating of item $j$ given by user $u$

  $S_{ij}$ --- a similarity measure between items $i$ and $j$

  $\hat{r}_{ui}$ --- (predicted) rating of item $i$ given by user $u$
\end{frame}
\begin{frame}
  \frametitle{Collaborative Neighbourhood-based Recommender}
  \begin{itemize}
  \item $\hat{r}_{ui}=\sum\limits_jS_{ij}r_{uj}$
  \item $S_{ij}$ is based on user interactions
  \item if $i$ is a new item, $S_{ij}=0$
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Content-based Neighbourhood-based Recommender}
  \begin{itemize}
  \item $\hat{r}_{ui}=\sum\limits_jS_{ij}r_{uj}$
  \item $S_{ij}$ is based on item features only
  \item if $i$ is a new item, still sometimes $S_{ij}\ne0$
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Hybrid Neighbourhood-based Recommender}
  \begin{itemize}
  \item $\hat{r}_{ui}=\alpha\sum\limits_jA_{ij}r_{uj}+\left(1-\alpha\right)\sum\limits_jB_{ij}r_{uj}$
  \item $A_{ij}$ is based on user interactions
  \item $B_{ij}$ is based on item features
  \item no cold-start problem!
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Hybrid Neighbourhood-based Recommender}
  \begin{itemize}
  \item $\hat{r}_{ui}=\alpha\sum\limits_jA_{ij}r_{uj}+\left(1-\alpha\right)\sum\limits_jB_{ij}r_{uj}$
  \item $\alpha$ can depend on the number of users who rated item $i$
  \item for example, $\alpha$ can be zero if item collected more than 10 ratings from different users
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Hybrid Neighbourhood-based Recommender}
  \begin{itemize}
  \item $\hat{r}_{ui}=\alpha\sum\limits_jA_{ij}r_{uj}+\left(1-\alpha\right)\sum\limits_jB_{ij}r_{uj}$
  \item no cold-start problem for items!
  \item still can't recommend anything to new users
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Universal Recommender} $$\hat{r}_{ui}=\alpha_1\sum\limits_jA_{1,ij}r_{1,uj}+\alpha_2\sum\limits_jA_{2,ij}r_{2,uj}+\dots+\alpha_n\sum\limits_jA_{n,ij}r_{n,uj}$$

  where

$\hat{r}_{ui}$ --- (predicted) probability of user $u$ making a target action with item $i$

$A_{1,ij},A_{2,ij},\dots$ --- a matrices of co-occurrences of actions of type $1,2,\dots$ on item $j$ with the target action on item $i$.

$r_{1,uj},r_{2,uj},\dots$ --- histories of actions of type $1,2,\dots$ for user $u$

$\alpha_1,\alpha_2,\dots$ --- model hyper-parameters, balancing different types of actions
\end{frame}
\begin{frame}
  \frametitle{Universal Recommender} $$\hat{r}_{ui}=\alpha_1\sum\limits_jA_{1,ij}r_{1,uj}+\alpha_2\sum\limits_jA_{2,ij}r_{2,uj}+\dots+\alpha_n\sum\limits_jA_{n,ij}r_{n,uj}$$
\begin{itemize}
\item uses a variety of interactions: buy, add to basket, add to wishlist, recommend to a friend, comment, like, report, abandon\dots
\item can recommend items with no target interactions
\item can recommend to users who have no target interactions
\end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Universal Recommender} $$\hat{r}_{ui}=\alpha_1\sum\limits_jA_{1,ij}r_{1,uj}+\alpha_2\sum\limits_jA_{2,ij}r_{2,uj}+\dots+\alpha_n\sum\limits_jA_{n,ij}r_{n,uj}$$
\begin{itemize}
\item popularized by \href{https://actionml.com/}{ActionML} in 2015
\item open-sourced as \href{https://attic.apache.org/projects/predictionio.html}{PredictionIO} (in Apache Attic since Sep 2020)
\item uses very basic, but very powerful idea
\end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Cold Start in Neighbourhood-based Recommendations} \begin{itemize}
\item easy to implement and scalable
\item can use item features (for cold items)
\item can use non-target actions (for cold users and items)
\end{itemize}
\end{frame}
\section{Cold Start in Latent Factors Recommendations}
\begin{frame}
  \frametitle{Latent Factors Recommendations}
  $$\hat{r}_{ui}=\left<x_u,y_i\right>$$

  where

  $\hat{r}_{ui}$ --- (predicted) rating of item $i$ given by user $u$

  $x_u$ --- latent representation (embedding) of a user $u$

  $y_i$ --- latent representation (embedding) of a user $i$

  $\left<.,.\right>$ --- inner product
\end{frame}
\begin{frame}
  \frametitle{Latent Factors Recommendations}
  $$\hat{r}_{ui}=\left<x_u,y_i\right>$$
  \begin{itemize}
  \item $x_u$ and $y_i$ can be results of matrix decomposition
  \item $x_u$ and $y_i$ are learn as embedding layers in a deep neural network
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Latent Factors Recommendations}
  $$\hat{r}_{ui}=\left<x_u,y_i\right>$$
  \begin{itemize}
  \item if user is new, $x_u$ is unknown, can't recommend
  \item if item is new, $y_i$ is unknown, can't recommend
  \item true for users and items not known at training time
  \item training a very resource intense (MF and especially DL)
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Similarity-based Solution}
  $$\hat{y_i}=\sum\limits_jS_{ij}y_j$$

  where

  \begin{itemize}
  \item $\hat{y_i}$ --- predicted embedding for a new item $i$
  \item $y_j$ --- embeddings of known items, similar to $i$
  \item $S_{ij}$ --- (pre-processed) similarity measure between items $i$ (new) and $j$ (known ones)
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Deep Learning Solution: CB2CF}
  \begin{itemize}
  \item $\hat{y_i}$ is defined as a NN from item features
  \item the model is trained on known items and their embeddings
  \item \href{https://dl.acm.org/doi/10.1145/3298689.3347038}{published} by Microsoft in 2019
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{CB2CF: Content-to-Collaborative filtering}
  \begin{enumerate}
  \item train a NN on know items and users interaction
  \item train two more NNs for features-to-embedding maps for users and items
  \item for new items and users get embeddings from features
  \item get recommendations from embeddings
  \end{enumerate}
\end{frame}
\begin{frame}
  \frametitle{CB2CF: Content-to-Collaborative filtering}
  \begin{figure}
    \includegraphics[scale=0.25]{CB2CF}
  \end{figure}
  RecSys '19: Proceedings of the 13th ACM Conference on Recommender SystemsSeptember 2019 Pages 228–236

  https://doi.org/10.1145/3298689.3347038
\end{frame}
\begin{frame}
  \frametitle{CB2CF: Content-to-Collaborative filtering}
  \begin{itemize}
  \item relatively simple solution
  \item works well with both new items and new users
  \item requires training several models instead of one
  \end{itemize}
\end{frame}
\section{Folding In}
\begin{frame}
  \frametitle{Embeddings for New Items in Real-Time}
  \begin{itemize}
  \item getting embeddings usually requires (re)training the model
  \item model training is always done separatly from serving (inference)
  \item model training is resource intense, expensive, and slow process
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Embeddings for New Items in Real-Time}
  \begin{itemize}
  \item if a user had no history during model training we can't easily get their embedding even if after model training the user accumulated history
  \item no user embedding means no recommendations
  \item but why not to recommend to a user who already has history?
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Folding In}
  $$\sum\limits_{u,i}\left(\left<x_u,y_i\right>-r_{ui}\right)^2\longrightarrow\min_{x_u,y_i}$$
  \begin{itemize}
  \item a well known loss function
  \item can be minimized using ALS (as matrix factorization)
  \item can be minimized using SGD (as a neural network)
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Folding In}
  $$\sum\limits_i\left(\left<x',y_i\right>-r_{u'i}\right)^2+\sum\limits_{u,i}\left(\left<x_u,y_i\right>-r_{ui}\right)^2\longrightarrow\min_{x',x_u,y_i}$$
  \begin{itemize}
  \item suppose we already found the minimum
  \item now we add new $r_{u'i}$ for a previously unknown user $u'$
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Folding In}
  $$\sum\limits_i\left(\left<x',y_i\right>-r_{u'i}\right)^2\longrightarrow\min_{x'}$$
  \begin{itemize}
  \item we can minimize for new ratings of a new user separately
  \item we also assume that several interactions of one user won't affect previously found embeddings of items $y_i$
  \item we arrive at linear regression of dimension $N$ (embedding dimension)
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Folding In}
  $$\sum\limits_i\left(\left<x',y_i\right>-r_{u'i}\right)^2\longrightarrow\min_{x'}$$
  \begin{itemize}
  \item very simple technique to embed new users in real time
  \item works both for matrix factorization (e.g. ALS or SVD)
  \item also can work for neural networks (e.g. with doing optimization steps for couple SGD batches with new data)
  \end{itemize}
\end{frame}
\section{Rating Elicitation}
\begin{frame}
  \frametitle{Possible New User Scenario}
  \begin{itemize}
  \item a new website or app, many new users
  \item no information about new users coming to our app
  \item we can buy some information but that's impractical
  \item users don't want to share information about them
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Rating Elicitation}
  \begin{itemize}
  \item we ask users which items they interacted before
  \item or which items they liked (not in our app)
  \item or which items they would like to see in our app
  \item we use those ratings instead of real interactions
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Rating Elicitation: Problem}
  \begin{itemize}
  \item we ask users which items they interacted before
  \item we need to show several to choose
  \item those items should be popular
  \item and also they should be diverse
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Popularity and Diversity of Embeddings}
  \begin{itemize}
  \item the more popular item is, the larger the embedding norm
  \item the more unsimilar items are, the larger the angle between the embeddings
  \item (that works for inner-product based models!)
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Popularity and Diversity of Embeddings}
\begin{figure}
\includegraphics[scale=0.2]{parallelepiped}
\end{figure}
\end{frame}
\begin{frame}
  \frametitle{Popularity and Diversity of Embeddings}
  \begin{itemize}
  \item embeddings can form a parallelepiped of some volume
  \item the larger the embedding norm, the greater the volume
  \item the larger the angle between the embeddings, the greater the volume
  \item ergo: we need to maximize volume!
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Generalized Volume}
  \begin{itemize}
  \item volume in $N$-dimensional embedding space is defined only for $N$ vectors (edges of the parallelepiped)
  \item if we want to choose $M\ne N$ vectors, what to maximize?
  \item $\det A$ is a volume of a parallelepiped formed by vectors $A_i$ (columns of the matrix $A$)
  \item generalized volume is defined as $\det A^TA$
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Generalized Volume}
  \begin{itemize}
  \item generalized volume is defined as $\det A^TA$
  \item $\det A^TA=\prod\limits_{i=1}^N\sigma_i$ where $\sigma_i$ are singular values of matrix $A$
  \item generalized volume maximization problem is related to SVD
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Maximal Generalized Volume}
  \begin{itemize}
  \item generalized volume is defined as $\det A^TA$
  \item generalized volume maximization problem is related to SVD
  \item there is a Python package \href{https://pypi.org/project/maxvolpy/}{\texttt{maxvolpy}} for computing it
  \end{itemize}
\end{frame}
\begin{frame}
  \frametitle{Rating Elicitation with MaxVol}
  \begin{itemize}
  \item build a recommender (get embeddings of items)
  \item apply rectangular MaxVol on embeddings
  \item show user the suggested subset of items
  \item recommend for the new user as if they had history (e.g. using folding in)
  \end{itemize}
\end{frame}
\end{document}


%intro
\section{Introduction}
\subsection{Overview}

\begin{frame}{•}
\frametitle{Overview}
\bi
\item<1-> Question from algebra:\\
\indent Unify classes of problems and reapply modern day algebra?\\
\item<2-> Answer:
\begin{block}{affirmative for}
\bi
\item<3-> differential algebras of $char = 0$,
\item<4-> iterative diff algebras of $char > 0$,
\item<5-> group algebras of arbitrary $char$,
\item<6-> difference algebras of arbitrary $char$,
\ei
\end{block}
\ei
\end{frame}

\subsection{Outline}

\begin{frame}{•}
\frametitle{Outline}
\bi
\item We focussed on differential algebras of $char = 0$ in this talk
\item Outline:
\
\bn
\item Classical differential Galois theory, the Picard-Vessiot theory (wrt. $\qzcl$),
\item Umemura functor for the same problem in a broader setting (i.g. $\qz$),
\en
\ei
\end{frame}


\input{permve-ntnu-latex-assignment.tex}

\usepackage{mathtools}

\title{
\normalfont \normalsize
\textsc{Norwegian University of Science and Technology\\TDT4136 -- Introduction to Artificial Intelligence} \\ [25pt]
\horrule{0.5pt} \\[0.4cm]
\huge Assignment 2:\\ Propositional and Predicate Logics\\
\horrule{2pt} \\[0.5cm]
}

\author{Per Magnus Veierland\\permve@stud.ntnu.no}

\date{\normalsize\today}

\newacro{AIMA}{Artificial Intelligence: A Modern Approach}
\newacro{CNF}{Conjunctive Normal Form}

\begin{document}

\maketitle

\section{Models and Entailment in Propositional Logic}

\newcommand{\F}{{\color{WildStrawberry}\small false}}
\newcommand{\T}{{\color{YellowGreen}\small true}}

\begin{enumerate}
\item
\begin{enumerate}
\item
\begin{tabular}{ccc|c|ccc}
$A$ & $\land$ & $\neg B$ & $\Rightarrow$ & $A$ & $\lor$ & $B$ \\
\hline
\F  & \F      & \T       & \T            & \F  & \F     & \F  \\
\F  & \F      & \F       & \T            & \F  & \T     & \T  \\
\T  & \T      & \T       & \T            & \T  & \T     & \F  \\
\T  & \F      & \F       & \T            & \T  & \T     & \T  \\
\end{tabular}

By the deduction theorem, the statement $A \land \neg B \vDash A \lor B$ is true as the statement $A \land \neg B \Rightarrow A \lor B$ is valid, i.e. true in all models.
\item
\begin{tabular}{ccc|c|ccc}
$A$ & $\lor$ & $B$ & $\Rightarrow$ & $A$ & $\land$ & $\neg B$ \\
\hline
\F  & \F     & \F  & \T            & \F  & \F      & \T       \\
\F  & \T     & \T  & \F            & \F  & \F      & \F       \\
\T  & \T     & \F  & \T            & \T  & \T      & \T       \\
\T  & \T     & \T  & \F            & \T  & \F      & \F       \\
\end{tabular}

By the deduction theorem, the statement $A \lor B \vDash A \land \neg B$ is false as the statement $A \lor B \Rightarrow A \land \neg B$ is not valid, since there are models in which it is not true.
\item
\begin{tabular}{ccc|c|ccc}
$A$ & $\Leftrightarrow$ & $B$ & $\Rightarrow$ & $A$ & $\Rightarrow$ & $B$ \\
\hline
\F  & \T                & \F  & \T            & \F  & \T            & \F  \\
\F  & \F                & \T  & \T            & \F  & \T            & \T  \\
\T  & \F                & \F  & \T            & \T  & \F            & \F  \\
\T  & \T                & \T  & \T            & \T  & \T            & \T  \\
\end{tabular}

By the deduction theorem, the statement $A \Leftrightarrow B \vDash A \Rightarrow B$ is true as the statement $A \Leftrightarrow B \Rightarrow A \Rightarrow B$ is valid, i.e. true in all models.
\item
\begin{tabular}{ccccc|c|ccccc}
$(A$ & $\Leftrightarrow$ & $B)$ & $\Leftrightarrow$ & $C$ & $\Rightarrow$ & $(A$ & $\lor$ & $\neg B)$ & $\lor$ & $\neg C$ \\
\hline
\F   & \T                & \F   & \F                & \F  & \T            & \F   & \T     & \T        & \T     & \T       \\
\F   & \T                & \F   & \T                & \T  & \T            & \F   & \T     & \T        & \T     & \F       \\
\F   & \F                & \T   & \T                & \F  & \T            & \F   & \F     & \F        & \T     & \T       \\
\F   & \F                & \T   & \F                & \T  & \T            & \F   & \F     & \F        & \F     & \F       \\
\T   & \F                & \F   & \T                & \F  & \T            & \T   & \T     & \T        & \T     & \T       \\
\T   & \F                & \F   & \F                & \T  & \T            & \T   & \T     & \T        & \T     & \F       \\
\T   & \T                & \T   & \F                & \F  & \T            & \T   & \T     & \F        & \T     & \T       \\
\T   & \T                & \T   & \T                & \T  & \T            & \T   & \T     & \F        & \T     & \F       \\
\end{tabular}

By the deduction theorem, the statement $(A \Leftrightarrow B) \Leftrightarrow C \vDash A \lor \neg B \lor \neg C$ is true as the statement $(A \Leftrightarrow B) \Leftrightarrow C \Rightarrow A \lor \neg B \lor \neg C$ is valid, i.e. true in all models.
\item
\begin{tabular}{ccc|c|ccc}
$(\neg A$ & $\land$ & $B)$ & $\land$ & $(A$ & $\Rightarrow$ & $B)$\\
\hline
\T        & \F      & \F   & \F      & \F   & \T            & \F  \\
\T        & \T      & \T   & \T      & \F   & \T            & \T  \\
\F        & \F      & \F   & \F      & \T   & \F            & \F  \\
\F        & \F      & \T   & \F      & \T   & \T            & \T  \\
\end{tabular}

The statement $(\neg A \land B) \land (A \Rightarrow B)$ is satisfiable, since it is true in at least one model.
\item
\begin{tabular}{ccc|c|ccc}
$(\neg A$ & $\land$ & $B)$ & $\land$ & $(A$ & $\Leftrightarrow$ & $B)$\\
\hline
\T        & \F      & \F   & \F      & \F   & \T            & \F  \\
\T        & \T      & \T   & \F      & \F   & \F            & \T  \\
\F        & \F      & \F   & \F      & \T   & \F            & \F  \\
\F        & \F      & \T   & \F      & \T   & \T            & \T  \\
\end{tabular}

The statement $(\neg A \land B) \land (A \Leftrightarrow B)$ is not satisfiable, since it is not true in any model.
\end{enumerate}
\item
\begin{enumerate}
\item $\big(\frac{1}{2}\big)^2 Q = \frac{1}{2^2} 2^{100} = 2^{98}$

The logical sentence $A_{31} \land \neg A_{76}$ is satisfied by $2^{98}$ models.
\item $\big(\frac{1}{2}\big)^3 Q = \frac{1}{2^3} 2^{100} = 2^{97}$

The logical sentence $A_{44} \land A_{49} \land A_{78}$ is satisfied by $2^{97}$ models.
\item $\Big(1 - \big(\frac{1}{2}\big)^3\Big) Q = 2^{100} - \frac{1}{2^3} 2^{100} = \frac{8}{8} 2^{100} - \frac{1}{8} 2^{100} = \frac{7}{8} 2^{100} = 7 \frac{2^{100}}{2^3} = 7 \cdot 2^{97}$

The logical sentence $A_{44} \lor A_{49} \lor A_{78}$ is satisfied by $7 \cdot 2^{97}$ models.
\item $\frac{1}{2} Q = \frac{1}{2} 2^{100} = 2^{99}$

The logical sentence $A_{70} \Rightarrow \neg A_{92}$ is satisfied by $2^{99}$ models.
\item $\frac{1}{2} \cdot \frac{1}{2} \cdot Q = \frac{1}{2^2} 2^{100} = 2^{98}$

The logical sentence $(A_7 \Leftrightarrow A_{72}) \land (A_{83} \Leftrightarrow A_{84})$ is satisfied by $2^{98}$ models.
\item $\big(\frac{1}{2}\big)^8 Q = \frac{1}{2^8} 2^{100} = 2^{92}$

The logical sentence $\neg A_9 \land \neg A_{19} \land A_{37} \land A_{50} \land A_{68} \land A_{73} \land A_{79} \land A_{81}$ is satisfied by $2^{92}$ models.
\end{enumerate}
\item
\begin{tabular}{cccc|ccc|c}
$P_{3,1}$ & $P_{3,2}$ & $P_{3,3}$ & $P_{4,4}$ & $\alpha_1$ & $\alpha_2$ & $\alpha_3$ & \textit{KB} \\
\hline
\F        & \F        & \F        & \F        & \F         & \F         & \F         & \F          \\
\F        & \F        & \F        & \T        & \F         & \F         & \T         & \F          \\
\F        & \F        & \T        & \F        & \F         & \T         & \T         & \F          \\
\F        & \F        & \T        & \T        & \F         & \T         & \T         & \F          \\
\F        & \T        & \F        & \F        & \F         & \F         & \F         & \F          \\
\F        & \T        & \F        & \T        & \F         & \F         & \T         & \F          \\
\F        & \T        & \T        & \F        & \F         & \T         & \T         & \F          \\
\F        & \T        & \T        & \T        & \F         & \T         & \T         & \F          \\
\T        & \F        & \F        & \F        & \T         & \F         & \F         & \F          \\
\T        & \F        & \F        & \T        & \T         & \F         & \T         & \T          \\
\T        & \F        & \T        & \F        & \T         & \T         & \T         & \T          \\
\T        & \F        & \T        & \T        & \T         & \T         & \T         & \T          \\
\T        & \T        & \F        & \F        & \T         & \F         & \F         & \F          \\
\T        & \T        & \F        & \T        & \T         & \F         & \T         & \F          \\
\T        & \T        & \T        & \F        & \T         & \T         & \T         & \F          \\
\T        & \T        & \T        & \T        & \T         & \T         & \T         & \F          \\
\end{tabular}

By the deduction theorem, the statement $\textit{KB} \vDash \alpha_1$ is true as the statement $\textit{KB} \Rightarrow \alpha_1$ is valid, i.e. true in all models.

The statement $\textit{KB} \vDash \alpha_2$ is false as the statement $\textit{KB} \Rightarrow \alpha_2$ is not valid, since there are models in which it is not true.

The statement $\textit{KB} \vDash \alpha_3$ is true as the statement $\textit{KB} \Rightarrow \alpha_3$ is valid, i.e. true in all models.
\end{enumerate}

\section{Resolution in Propositional Logic}

\begin{enumerate}
\item
\begin{enumerate}
\item
$\neg A \lor (B \land C) \equiv
\underline{(\neg A \lor B) \land (\neg A \lor C)}$

\item
$\neg (A \Rightarrow B) \land \neg (C \Rightarrow D)\\
\equiv \neg (\neg A \lor B) \land \neg (\neg C \lor D)\\
\equiv (\neg \neg A \land \neg B) \land (\neg \neg C \land \neg D)\\
\equiv (A \land \neg B) \land (C \land \neg D)\\
\equiv \underline{A \land \neg B \land C \land \neg D}$

\item
$\neg (A \Rightarrow B) \lor \neg (C \Rightarrow D)\\
\equiv \neg (\neg A \lor B) \lor \neg (\neg C \lor D)\\
\equiv (\neg \neg A \land \neg B) \lor (\neg \neg C \land \neg D)\\
\equiv (A \land \neg B) \lor (C \land \neg D)\\
\equiv (A \lor (C \land \neg D)) \land (\neg B \lor (C \land \neg D))\\
\equiv ((A \lor C) \land (A \lor \neg D)) \land ((\neg B \lor C) \land (\neg B \lor \neg D))\\
\equiv \underline{(A \lor C) \land (A \lor \neg D) \land (\neg B \lor C) \land (\neg B \lor \neg D)}$

\item
$(A \Rightarrow B) \Leftrightarrow C\\
\equiv ((A \Rightarrow B) \Rightarrow C) \land (C \Rightarrow (A \Rightarrow B))\\
\equiv (\neg (\neg A \lor B) \lor C) \land (\neg C \lor (\neg A \lor B))\\
\equiv ((\neg \neg A \land \neg B) \lor C) \land (\neg C \lor \neg A \lor B)\\
\equiv ((A \land \neg B) \lor C) \land (\neg C \lor \neg A \lor B)\\
\equiv ((A \lor C) \land (\neg B \lor C)) \land (\neg C \lor \neg A \lor B)\\
\equiv \underline{(A \lor C) \land (\neg B \lor C) \land (\neg C \lor \neg A \lor B)}$
\end{enumerate}

\item
Determining whether $(\textit{KB} \vDash \neg A)$ using resolution is done using the principle of proof by contradiction. Showing $(\textit{KB} \vDash \neg A)$ is done by showing that $(\textit{KB} \land A)$ is unsatisfiable.

The first step is to convert $(\textit{KB} \land A)$ into \ac{CNF}:

$((X \land \neg Y) \Rightarrow \neg B) \land (\neg X \Rightarrow C) \land (B \land \neg Y) \land (A \Rightarrow \neg C) \land (A)\\
\equiv (\neg (X \land \neg Y) \lor \neg B) \land (\neg \neg X \lor C) \land (B) \land (\neg Y) \land (\neg A \lor \neg C) \land (A)\\
\equiv ((\neg X \lor \neg \neg Y) \lor \neg B) \land (X \lor C) \land (B) \land (\neg Y) \land (\neg A \lor \neg C) \land (A)\\
\equiv \underline{(\neg X \lor Y \lor \neg B) \land (X \lor C) \land (B) \land (\neg Y) \land (\neg A \lor \neg C) \land (A)}$

\medskip

Applying resolution to these six clauses yields:

\begin{equation}
\frac{\neg X \lor Y \lor \neg B, \quad X \lor C}{Y \lor \neg B \lor C}
\end{equation}

\begin{equation}
\frac{Y \lor \neg B \lor C, \quad B}{Y \lor C}
\end{equation}

\begin{equation}
\frac{Y \lor C, \quad \neg Y}{C}
\end{equation}

\begin{equation}
\frac{\neg A \lor \neg C, \quad C}{\neg A}
\end{equation}

\begin{equation}
\frac{\neg A, \quad A}{\textit{False}}
\end{equation}

This result shows by contradiction that $(\textit{KB} \land A)$ is unsatisfiable, which means that $(\textit{KB} \Rightarrow \neg A)$ must be valid, which means that the statement $(\textit{KB} \vDash \neg A)$ must be true.

\item
\begin{enumerate}
\item
\begin{tabular}{ccccc|c|ccc}
$\big(\big(\textit{Food}$ & $\lor$ & $\textit{Drinks}\,\big)$ & $\Rightarrow$ & $\textit{Party}\,\big)$ & $\Rightarrow$ & $\big(\neg \textit{Party}$ & $\Rightarrow$ & $\neg \textit{Food}\,\big)$ \\
\hline
\F                        & \F     & \F                       & \T            & \F                      & \T            & \T                         & \T            & \T                          \\
\F                        & \F     & \F                       & \T            & \T                      & \T            & \F                         & \T            & \T                          \\
\F                        & \T     & \T                       & \F            & \F                      & \T            & \T                         & \T            & \T                          \\
\F                        & \T     & \T                       & \T            & \T                      & \T            & \F                         & \T            & \T                          \\
\T                        & \T     & \F                       & \F            & \F                      & \T            & \T                         & \F            & \F                          \\
\T                        & \T     & \F                       & \T            & \T                      & \T            & \F                         & \T            & \F                          \\
\T                        & \T     & \T                       & \F            & \F                      & \T            & \T                         & \F            & \F                          \\
\T                        & \T     & \T                       & \T            & \T                      & \T            & \F                         & \T            & \F                          \\
\end{tabular}

The sentence $\big(\big(\textit{Food} \lor \textit{Drinks}\,\big) \Rightarrow \textit{Party}\,\big) \Rightarrow \big(\neg \textit{Party} \Rightarrow \neg \textit{Food}\,\big)$ is shown by enumeration to be valid, as it is true in all models.

\item
Conversion of left-hand side of main implication to \ac{CNF}:

$(F \lor D) \Rightarrow P\\
\equiv \neg (F \lor D) \lor P\\
\equiv (\neg F \land \neg D) \lor P\\
\equiv \underline{(\neg F \lor P) \land (\neg D \lor P)}$

Conversion of right-hand side of main implication to \ac{CNF}:

$\neg P \Rightarrow \neg F\\
\equiv \underline{(P \lor \neg F)}$

\item Proving the main implication is done using the principle of proof by contradiction. Showing $((F \lor D) \Rightarrow P) \Rightarrow (\neg P \Rightarrow \neg F)$ is done by showing that\\$((F \lor D) \Rightarrow P) \land \neg (\neg P \Rightarrow \neg F)$ is unsatisfiable.

The right-hand side of the main implication from part b is negated:

$\neg (P \lor \neg F) \equiv \neg P \land \neg \neg F \equiv \neg P \land F \equiv \underline{(\neg P) \land (F)}$

This results in the following four clauses for resolution:

$(\neg F \lor P) \land (\neg D \lor P) \land (\neg P) \land (F)$

Applying resolution to these four clauses yields:

\setcounter{equation}{0}

\begin{equation}
\frac{\neg F \lor P, \quad \neg P}{\neg F}
\end{equation}

\begin{equation}
\frac{\neg F, \quad F}{\textit{False}}
\end{equation}

This result shows by contradiction that $((F \lor D) \Rightarrow P) \land \neg (\neg P \Rightarrow \neg F)$ is unsatisfiable, which means that the sentence $((F \lor D) \Rightarrow P) \Rightarrow (\neg P \Rightarrow \neg F)$ must be valid. This agrees with the result from part a where the sentence was found to be valid by enumeration.
\end{enumerate}
\end{enumerate}

\section{Representations in First-Order Logic}

\begin{enumerate}
\item
\begin{enumerate}
\item $\textit{PlayedCharacter}(\textit{Bale}, \textit{Batman}) \land\\
\textit{PlayedCharacter}(\textit{Clooney}, \textit{Batman}) \land\\
\textit{PlayedCharacter}(\textit{Kilmer}, \textit{Batman})$
\item $\neg \exists c ~ \textit{PlayedCharacter}(\textit{Ledger}, c) \land \textit{PlayedCharacter}(\textit{Bale}, c)$
\item $\forall m ~ \textit{CharacterInMovie}(\textit{Batman}, m) \land \textit{Directed}(\textit{Nolan}, m) \Rightarrow \textit{PlayedInMovie}(\textit{Bale}, m)$
\item $\exists m ~ \textit{CharacterInMovie}(\textit{Joker}, m) \land \textit{CharacterInMovie}(\textit{Batman}, m)$
\item $\exists m ~ \textit{Directed}(\textit{Costner}, m) \land \textit{PlayedInMovie}(\textit{Costner}, m)$
\item $\Big[\neg \exists m ~ \textit{PlayedInMovie}(\textit{Clooney}, m) \land \textit{PlayedInMovie}(\textit{Tarantino}, m)\Big] \land\\
\Big[ \neg \exists m ~ \textit{Directed}(\textit{Tarantino}, m) \land \textit{PlayedInMovie}(\textit{Clooney}, m) \Big]$
\item $\exists c, m ~ \textit{PlayedCharacter}(\textit{Thurman}, c) \land \textit{CharacterInMovie}(c, m) \land \textit{Directed}(\textit{Tarantino}, m)$
\end{enumerate}

\item
\begin{enumerate}
\item $\forall x, y ~ \textit{Divisible}(x, y) \Leftrightarrow \big[ \exists z ~ (z < x) \land (x = z \times y) \big]$
\item $\forall x ~ \textit{Even}(x) \Leftrightarrow (x = 0) \lor \textit{Divisible}(x, 2)$
\item $\forall x ~ \textit{Odd}(x) \Leftrightarrow \neg \textit{Divisible}(x, 2)$
\item $\forall x ~ \textit{Odd}(x) \Leftrightarrow \exists y \big[ \textit{Even}(y) \land (x = y + 1) \big]$
\item $\forall x ~ \textit{Prime}(x) \Leftrightarrow \textit{Divisible}(x, x) \land \neg \exists y \big[ (x \neq y) \land (y \neq 1) \land \textit{Divisible}(x, y) \big]$
\end{enumerate}

\item
\begin{itemize}
\item \textit{Person} is a unary predicate which is true if the referred object is a person.
\item \textit{MotherOf} is a unary function mapping from a person object to the object representing the person's mother.
\item \textit{FatherOf} is a unary function mapping from a person object to the object representing the person's father.
\item \textit{D{\footnotesize NA}Of\thinspace} is a unary function mapping from a person object to the object representing the DNA of the person.
\item \textit{D{\footnotesize NA}Combine\thinspace} is a binary function mapping from two DNA objects to the DNA object representing the combination of the two DNA objects.
\end{itemize}

``Everyone's DNA is unique and is derived from their parent's DNA.''

$\big[\forall \; p_1, p_2 \; : \; \textit{Person}(p_1) \land \textit{Person}(p_2) \land (p_1 \neq p_2) \Rightarrow \textit{D{\footnotesize NA}Of\thinspace}(p_1) \neq \textit{D{\footnotesize NA}Of\thinspace}(p_2) \big] \land \\
\Bigg[ \forall \; p \; : \textit{Person}(p) \Rightarrow \bigg(\textit{D{\footnotesize NA}Of\thinspace}(p) = \textit{D{\footnotesize NA}Combine\thinspace}\Big(\textit{D{\footnotesize NA}Of\thinspace}\big(\textit{MotherOf\thinspace}(p)\big), \textit{D{\footnotesize NA}Of\thinspace}\big(\textit{FatherOf\thinspace}(p)\big)\Big)\bigg) \Bigg]$

\end{enumerate}

\section{Resolution in First-Order Logic}

\begin{enumerate}
\item
\begin{enumerate}
\item $\textsc{Unify}\big(\textit{Philosopher}(x), \textit{Philosopher}(\textit{Plato})\big) = \{ x / \textit{Plato} \,\}$
\item $\textsc{Unify}\big(\textit{Write}(\textit{Plato}, \textit{TheRepublic}), \textit{Write}(\textit{Plato}, y)\big) = \{ y / \textit{TheRepublic} \,\}$
\item $\textsc{Unify}\big(\textit{Read}(x, \textit{Metaphysics}), \textit{Read}(\textit{Peter}, y)\big) = \{ x / \textit{Peter}, y / \textit{Metaphysics} \,\}$
\item The atomic sentences are standardized apart to allow unification.\\Without renaming the variables the sentences cannot be unified.

$\textsc{Unify}\big(\textit{Write}(x_1, \textit{Fear}), \textit{Write}(\textit{Kierkegaard}, x_2)\big) =
\{ x_1 / \textit{Kierkegaard}, x_2 / \textit{Fear} \,\}$

\item Following the \textsc{Unify}-algorithm as described in \textit{\ac{AIMA}}:
\begin{enumerate}
\item \textsc{Unify}$\Big(\textit{Write}\big(\textit{Kant}, \textit{Critique}\big), \textit{Write}\big(\textit{Author}(y), y\big), []\Big)$

$\big[ \theta \neq \textit{failure} \big] \land \big[x \neq y\big] \land \big[\textsc{Variable?}(x) = \textit{False}\big] \land \big[\textsc{Variable?}(y) = \textit{False}\big] \land\\
\big[ \textsc{Compound?}(x) = \textit{True} \big] \land \big[ \textsc{Compound}(y) = \textit{True} \big]$

\item \textsc{Unify}(\textit{Write}, \textit{Write}, [])

$\big[\theta \neq \textit{failure}\big] \land \big[x = y\big]$

Returns: $[]$

\item \textsc{Unify}$\Big(\big[\textit{Kant}, \textit{Critique}\big], \big[\textit{Author}(y), y\big], []\Big)$

$\big[\theta \neq \textit{failure}\big] \land \big[x \neq y\big] \land \big[\textsc{Variable?}(x) = \textit{False}\big] \land \big[\textsc{Variable?}(y) = \textit{False}\big] \land\\
\big[ \textsc{Compound?}(x) = \textit{False} \land \textsc{Compound?}(y) = \textit{False} \big] \land\\
\big[ \textsc{List?}(x) = \textit{True} \land \textsc{List?}(y) = \textit{True} \big]$

\item \textsc{Unify}$\big(\textit{Kant}, \textit{Author}(y), []\big)$

$\big[\theta \neq \textit{failure}\big] \land \big[x \neq y\big] \land \big[\textit{Variable?}(x) = \textit{False}\big] \land \big[\textsc{Variable?}(y) = \textit{False}\big] \land\\
\big[ \textsc{Compound?}(x) = \textit{False} \land \textsc{Compound?}(y) = \textit{True} \big] \land\\
\big[ \textsc{List?}(x) = \textit{False} \land \textsc{List?}(y) = \textit{False} \big]$

Returns: \textit{failure}
\end{enumerate}

The \textsc{Unify} algorithm fails to find a \textit{unifier} for the pair of atomic sentences.\\However, the unifier $\{y / \textit{Critique}\,\}$ makes the following statement true:

$\big[ \textsc{Subst}(\{y / \textit{Critique} \,\}, \textit{Write}(\textit{Kant}, \textit{Critique})) =
\textsc{Subst}(\{y / \textit{Critique} \,\}, \textit{Write}(\textit{Author}(y), y)) \big]$

$\Leftrightarrow$

$\big[ \textit{Write}(\textit{Kant}, \textit{Critique}) =
\textit{Write}(\textit{Author}(\textit{Critique}), \textit{Critique}) \big]$
\end{enumerate}

\item
\begin{enumerate}
\item
$\exists x \, \exists y ~ \textit{Philosopher}(x) \land \textit{StudentOf}(y, x) \equiv \textit{Philosopher}(X) \land \textit{StudentOf}(Y, X)$

\item
$\forall y, x ~ \textit{Philosopher}(x) \land \textit{StudentOf}(y, x) \Rightarrow \big[ \exists z ~ \textit{Book}(z) \land \textit{Write}(x, z) \land \textit{Read}(y, z) \big] \equiv$

$\forall y, x ~ \textit{Philosopher}(x) \land \textit{StudentOf}(y, x) \Rightarrow \big[ \textit{Book}(Z(y, x)) \land \textit{Write}(x, Z(y, x)) \land \textit{Read}(y, Z(y, x)) \big]$
\end{enumerate}

\item
Initial conversion to \ac{CNF}:
\begin{itemize}
\item
$\forall x ~ \textit{SuperActor}(x) \Leftrightarrow \big[ \exists m ~ \textit{PlayedInMovie}(x, m) \land \textit{Directed}(x, m) \big] \equiv$

$\forall x ~ \Big[ \textit{SuperActor}(x) \Rightarrow \big[ \exists m ~ \textit{PlayedInMovie}(x, m) \land \textit{Directed}(x, m) \big] \Big] \land$\\
$\Big[ \big[ \exists m ~ \textit{PlayedInMovie}(x, m) \land \textit{Directed}(x, m) \big] \Rightarrow \textit{SuperActor}(x) \Big] \equiv $

$\forall x ~ \Big[ \neg \textit{SuperActor}(x) \lor \big[ \exists m ~ \textit{PlayedInMovie}(x, m) \land \textit{Directed}(x, m) \big] \Big] \land$\\
$\Big[ \big[ \neg \exists m ~ \textit{PlayedInMovie}(x, m) \land \textit{Directed}(x, m) \big] \lor \textit{SuperActor}(x) \Big] \equiv $

$\forall x ~ \Big[ \neg \textit{SuperActor}(x) \lor \big[ \exists m ~ \textit{PlayedInMovie}(x, m) \land \textit{Directed}(x, m) \big] \Big] \land$\\
$\Bigg[ \Big[ \forall m ~ \neg \big[ \textit{PlayedInMovie}(x, m) \land \textit{Directed}(x, m) \big] \Big] \lor \textit{SuperActor}(x) \Bigg] \equiv $

$\forall x, m ~ \Big[ \neg \textit{SuperActor}(x) \lor \big[ \textit{PlayedInMovie}(x, M(x)) \land \textit{Directed}(x, M(x)) \big] \Big] \land$\\
$\Bigg[ \Big[ \neg \big[ \textit{PlayedInMovie}(x, m)) \land \textit{Directed}(x, m) \big] \Big] \lor \textit{SuperActor}(x) \Bigg] \equiv $

$\forall x, m ~ \Big[ \big[ \neg \textit{SuperActor}(x) \lor \textit{PlayedInMovie}(x, M(x)) \big] \land \big[ \neg \textit{SuperActor}(x) \lor \textit{Directed}(x, M(x)) \big] \Big] \land$\\
$\big[ \neg \textit{PlayedInMovie}(x, m) \lor \neg \textit{Directed}(x, m) \lor \textit{SuperActor}(x) \big]$

\underline{$\big[ \neg \textit{SuperActor}(x) \lor \textit{PlayedInMovie}(x, M(x)) \big] \land \big[ \neg \textit{SuperActor}(x) \lor \textit{Directed}(x, M(x)) \big] \land$}\\
\underline{$\big[ \neg \textit{PlayedInMovie}(x, m) \lor \neg \textit{Directed}(x, m) \lor \textit{SuperActor}(x) \big]$}

\clearpage

\item
$\forall a ~ \textit{Directed}(\textit{Tarantino}, a) \Leftrightarrow \textit{PlayedInMovie}(\textit{UmaThurman}, a) \equiv$

$\forall a ~ \big[ \textit{Directed}(\textit{Tarantino}, a) \Rightarrow \textit{PlayedInMovie}(\textit{UmaThurman}, a) \big] \land$\\
$\big[ \textit{PlayedInMovie}(\textit{UmaThurman}, a) \Rightarrow \textit{Directed}(\textit{Tarantino}, a) \big] \equiv$

$\forall a ~ \big[ \neg \textit{Directed}(\textit{Tarantino}, a) \lor \textit{PlayedInMovie}(\textit{UmaThurman}, a) \big] \land$\\
$\big[ \neg \textit{PlayedInMovie}(\textit{UmaThurman}, a) \lor \textit{Directed}(\textit{Tarantino}, a) \big] \equiv$

$\underline{\big[ \neg \textit{Directed}(\textit{Tarantino}, a) \lor \textit{PlayedInMovie}(\textit{UmaThurman}, a) \big] \land}$\\
$\underline{\big[ \neg \textit{PlayedInMovie}(\textit{UmaThurman}, a) \lor \textit{Directed}(\textit{Tarantino}, a) \big]}$

NB: $a / A$ is used as an identifier instead of $m$ for standardization.

\item
$\exists b ~ \textit{PlayedInMovie}(\textit{UmaThurman}, b) \land \textit{PlayedInMovie}(\textit{Tarantino}, b) \equiv$

$\underline{\big[ \textit{PlayedInMovie}(\textit{UmaThurman}, B) \big] \land \big[ \textit{PlayedInMovie}(\textit{Tarantino}, B) \big]}$

NB: $b / B$ is used as an identifier instead of $m$ for standardization.
\end{itemize}

\begin{enumerate}
\item
We prove that Tarantino is a super actor by showing a contradiction if we assume that the opposite is true.
\begin{enumerate}
\item
{\footnotesize
\begin{displaymath}
\frac{\neg \textit{PlayedInMovie}(x, m) \lor \neg \textit{Directed}(x, m) \lor \textit{SuperActor}(x), \quad \neg \textit{SuperActor}(\textit{Tarantino})}{\textsc{Subst}\big(\{x / \textit{Tarantino}\}, \neg \textit{PlayedInMovie}(x, m) \lor \neg \textit{Directed}(x, m)\big)}
\end{displaymath}
}
Resolvent clause: $\neg \textit{PlayedInMovie}(\textit{Tarantino}, m) \lor \neg \textit{Directed}(\textit{Tarantino}, m)$.

\item
{\footnotesize
\begin{displaymath}
\frac{\neg \textit{PlayedInMovie}(\textit{UmaThurman}, a) \lor \textit{Directed}(\textit{Tarantino}, a), \quad \textit{PlayedInMovie}(\textit{UmaThurman}, B)}{\textsc{Subst}\big(\{ a / B\}, \textit{Directed}(\textit{Tarantino}, a)  \big)}
\end{displaymath}
}
Resolvent clause: $\textit{Directed}(\textit{Tarantino}, B)$.

\item
{\footnotesize
\begin{displaymath}
\frac{\neg \textit{PlayedInMovie}(\textit{Tarantino}, m) \lor \neg \textit{Directed}(\textit{Tarantino}, m), \quad \textit{Directed}(\textit{Tarantino}, B)}{\textsc{Subst}\big(\{m / B\}, \neg \textit{PlayedInMovie}(\textit{Tarantino}, m) \big)}
\end{displaymath}
}
Resolvent clause: $\neg \textit{PlayedInMovie}(\textit{Tarantino}, B)$.

\item
{\footnotesize
\begin{displaymath}
\frac{\neg \textit{PlayedInMovie}(\textit{Tarantino}, B), \quad \textit{PlayedInMovie}(\textit{Tarantino}, B)}{\textsc{Subst}\big(\{\}, \textit{False}\big)}
\end{displaymath}
}
Resolvent clause: \textit{False}.
\end{enumerate}

\item
\begin{itemize}
\item Someone is a super actor if and only if there exists a movie which they have both played in and directed.
\item Tarantino directs a movie if and only if Uma Thurman plays in the movie.
\item There exists a movie which both Uma Thurman and Tarantino plays in.
\end{itemize}

We know that the requirement for being a super actor is that one must have both been an actor and a director for the same movie. Since we know that Tarantino directs all movies which Uma Thurman plays in, and we know that Uma Thurman and Tarantino have both played in the same movie, it follows that Tarantino both directed and played in this movie. Since Tarantino has been both an actor and director in the same movie at least once, we know that he must be a super actor.
\end{enumerate}
\end{enumerate}

\end{document}


\documentclass{memoir}
\usepackage{notestemplate}

\logo{~/LibreMath/Auxiliary Resources/resources/png/logo.png}
\institute{Rice University}
%\faculty{Faculty of Whatever Sciences}
\department{Department of Mathematics}
\title{A Study of Kakeya Sets}
%\subtitle{Based on MATH xxx}
\author{\textit{Author}\\Gabriel \textsc{Gress}}
%\supervisor{Linus \textsc{Torvalds}}
\context{Techniques and Results in the Field}
\date{\today}

\begin{document}

\maketitle

% Notes taken on 04/12/21

These notes summarize and discuss a variant of the Kakeya conjecture in finite fields. These results are profound, as the techniques developed in the restricted form of the problem can be generalized to broader tools that can help reduce the bound on dimension for the Kakeya conjecture in higher dimensions.

\chapter{Kakeya Sets in Finite Fields}
\label{cha:kakeya_sets_in_finite_fields}

This section is a summary of Dvir's paper on the size of Kakeya sets in finite fields.\\
% Inherently compact-- why?

We have a slightly different definition of Kakeya sets in finite fields:
\begin{defn}[Kakeya Set]
	Let \(\mathbb{F}\) be a finite field of \(q\) elements. A \textbf{Kakeya set} in \(\mathbb{F}^{n}\) is a set \(K\subset \mathbb{F}^{n}\) such that \(K\) contains a line in every direction. That is, for every \(x \in \mathbb{F}^{n}\), there exists a \(y \in \mathbb{F}^{n}\) such that
	\begin{align*}
		L_{y,x} := \left\{y + a\cdot x \mid a \in \mathbb{F} \right\} \subset K.
	\end{align*}
\end{defn}
% Can we work in general over n-tuple fields (not necessarily finite)? I think one can define a norm on n-tuples over fields, and hence can define unit length. How can we define so it is consistent with R and C?

% Ans-- not really possible. Finite fields have such a different structure. Many propositions/theorems are true on one and not the other (Szemeredi-Trotter theorem). Tubes are a slight resolution.

Because of the structure of the lines in \(\mathbb{F}^{n}\), a lower bound can be imposed on the size of these sets. Previously, the best lower bounds in the general case is of the form \(C_n \cdot q^{\frac{4n}{7}}\). Previous results were obtained by using an additive number theory lemma-- the theorem proved here is obtained via homogeneous polynomials and gets a near-optimal bound.

\begin{thm}
	Let \(K\subset \mathbb{F}^{n}\) be a Kakeya set. Then
	\begin{align*}
		\left| K \right| \geq C_n\cdot q^{n-1}
	\end{align*}
	where \(C_n\) depends only on \(n\).
\end{thm}
This can be improved by observing that the product of Kakeya sets is also a Kakeya set.

\begin{cor}
	For every integer \(n\) and every \(\varepsilon>0\), there exists a constant  \(C_{n,\varepsilon}\) depending only on \(n\) and \(\varepsilon\) such that any Kakeya set \(K\subset \mathbb{F}^{n}\) satisfies
	\begin{align*}
		\left| K \right| \geq C_{n,\varepsilon }\cdot q^{n-\varepsilon}.
	\end{align*}
\end{cor}
This follows from taking the Cartesian product of Kakeya sets, applying Theorem 1, then taking the \(r\)-th root to obtain the bound on \(K\).

\begin{defn}
	A set \(K\subset \mathbb{F}^{n}\) is a \textbf{\((\delta ,\gamma )\)-Kakeya set} if there exists a set \(\mathcal{L}\subset \mathbb{F}^{n}\) of size at least  \(\delta \cdot q^{n}\) such that, for every \(x \in \mathcal{L}\) there is a line in direction \(x\) that intersects \(K\) in at least \(\gamma \cdot q\) points.
\end{defn}
This broader definition will be easier to work with. We will give a lower bound on these types of Kakeya sets, and then obtain Theorem 1 by setting \(\delta =\gamma =1\).

\begin{thm}
	Let \(K\subset \mathbb{F}^{n}\) be a \((\delta ,\gamma )\)-Kakeya set. Then
	\begin{align*}
		\left| K \right| \geq {{d+n-1}\choose{n-1}} = \frac{(d+n-1)!}{(n-1)!d!},
	\end{align*}
	where
	\begin{align*}
		d = \left\lfloor q \cdot \textrm{min}\left\{ \delta ,\gamma  \right\}  \right\rfloor - 2.
	\end{align*}
\end{thm}

\section{Proof of Theorem 1.0.2}
\label{sec:proof_of_theorem_1_0_2}

To prove Theorem 2, we first need a lemma on polynomials in finite fields:
\begin{lemma}% Schwartz-Zippel
	Let \(f \in \mathbb{F}^{n}[x]\) be a non-zero polynomial with \(\textrm{deg}(f)\leq d\). Then
	\begin{align*}
		\left| \left\{x \in \mathbb{F}^{n} \mid f(x) = 0 \right\}  \right| \leq d\cdot q^{n-1}.
	\end{align*}
\end{lemma}

\begin{proof}[Proof of Theorem 1.0.2]
	Suppose for the sake of contradiction that
	\begin{align*}
		\left| K \right| < {{d+n-1}\choose{n-1}} = \frac{(d+n-1)!}{(n-1)!d!}.
	\end{align*}
	Observe that there are \(q\) monomials of degree \(d\) and hence more monomials than the size of \(K\). %Why?
Thus there must exist a homogeneous polynomial \(g\) of degree \(d\), where \(g\) is not the zero polynomial, that satisfies
\begin{align*}
	\forall x \in K, \quad g(x) = 0.
\end{align*}
In other words, because the degree of \(d\) is sufficiently high enough, we can solve a system of equations to create a polynomial that takes on zeroes on \(K\).\\

We will use this to show that \(g\) has too many zeroes and hence must be identically zero, which would contradict the above. Consider the set
\begin{align*}
	K' := \left\{c\cdot x \mid x \in K,\, c \in \mathbb{F} \right\}
\end{align*}
that contains all lines that pass through zero and intersect \(K\) at some point. By the homogeneity of \(g\), observe that
\begin{align*}
	g(c\cdot x) = c^{d}\cdot g(x)
\end{align*}
and so for all \(x \in K'\), we must have that \(g(x) = 0\).\\

Now recall the defintion of a \((\delta ,\gamma )\)-Kakeya set, and let the set \(\mathcal{L}\subset \mathbb{F}^{n}\) be given (with size \(\delta \cdot q^{n}\)).

\begin{prop}
	For every \(y \in \mathcal{L}\), \(g(y) = 0\).
\end{prop}

\begin{proof}[Proof of Proposition]

\end{proof}
Let \(y \in \mathcal{L}\) be a non-zero vector. Then by definition there exists a point \(z \in \mathbb{F}^{n}\) such that
\begin{align*}
	L_{z,y} = \left\{z + a\cdot y \mid a \in \mathbb{F} \right\}
\end{align*}
intersects \(K\) in at least \(\gamma \cdot q\) points. Therefore, since \(d+2 \leq \gamma \cdot d\), there exist \(d+2\) distinct field elements \(a_1,\ldots,a_{d+2} \in \mathbb{F}\) such that \(z+a_i \cdot y \in K\). One \(a_i\) might be zero, but because they are distinct, this still guarantees \(d+1\) distinct non-zero field elements that lie in \(K\).

% Rest of proof
\end{proof}

\end{document}


%************************************************
\chapter{Protocol}\label{ch:protocol}
%************************************************

\section{Using asymmetric keys to authenticate}

\paragraph{Paring Authenticator and Sibling.} Paring between a users Authenticator $S$ and it's Sibling $As$ is required before the Authenticator can be registered with a service and used to authenticate. First the parties generates and exchanges public/private keys to ensure secrecy and integrity of messages between the parties. All messages send between $A$ and $As$ in the remainder of this section are \textit{signed-then-encrypted} with the given key, and will for brevity not be notarised.

\paragraph{Registering with a Service Provider.}
First the Authenticator $A$ and it's sibling $As$ initiates the registration with the Service Provider $S$ by generating two new public/private key-pairs $\left(k_a,g^{k_a}\right)$ and $\left(k_{as},g^{k_{as}}\right)$ that will used only with the given $S$. The Authenticator now initiate the registration with a key exchange.

All further communication between $A$ and $S$ will be \textit{signed-then-encryp\-ted} using the key-pairs, and will for brevity not be notarised. %The message flow is also depicted in figure~\ref{msc:registration}.
\begin{align}
A   &\rightarrow S      &&  \left\{g^{k_a},g^{k_{as}}\right\}
\end{align}

S now responds with an initial seed $c$ that should be used for later authentication
\begin{align}
S   &\rightarrow A      &&  \left\{g^{k_s}, m = c_s \cdot  \left( g^{k_{as}} \cdot g^{k_a} \right)^{k_s}\right\}\\
A   &\rightarrow As     &&  \left\{g^{k_s}, m' = m \div {\left(g^{k_s}\right)^{k_a}}\right\}
\end{align}

The sibling $As$ thereafter decrypts $c_s = m' \div {\left(g^{k_s}\right)^{k_{as}}}$

\begin{figure}
%\begin{wide}
\begin{msc}{Registration}

\setlength{\instdist}{2.5cm}
\setlength{\actionwidth}{2.6cm}

\declinst{as}{}{$As$}
\declinst{a}{}{$A$}
\declinst{s}{}{$S$}
\mess{$\left\{g^{k_a},g^{k_{as}}\right\}$}{a}{s}
\nextlevel
\action{Generate $c_s$}{s}
\nextlevel[3]
\mess{$\left\{m,g^{k_s}\right\}$}{s}{a}
\nextlevel
\action{Compute $m'$}{a}
\nextlevel[3]
\mess{$\left\{m',g^{k_s}\right\}$}{a}{as}
\end{msc}
%\end{wide}
\caption{}
\label{msc:registration}
\end{figure}

\paragraph{Authentication.}
After the registration the Authenticator $A$ and it's sibling $As$ can authenticate with the service $S$ by sending a token using their secrets $c_s$ and $k_a$. The message flow is depicted in figure~\ref{msc:authentication}.
\begin{align}
A   &\rightarrow S      &&  \left\{ g^{k_a}, t = \left( g^{c_s} \cdot g^{k_s} \right)^{k_a} \right\}
\end{align}

The session with $S$ will be kept alive continuously and after every successful authentication round $As$ and $S$ computes a $c' = hash(c)$ that should be used for the next round of authentication. As the secrets to compute a valid token are distributed between $A$ and $As$, $S$ is convinced that the Authenticator and it's sibling are collaborating.

The service provider $S$ can verify the authenticity of the token received from the Authenticator $A$ by computing
\begin{align*}
check: && t \stackrel{?}{=} \left(g^{k_a}\right)^{c_s} \cdot \left(g^{k_a}\right)^{k_s}
\end{align*}

Because only the holders of either $\left(k_a,g^{c_s},g^{k_s}\right)$ or $\left(k_s,c_s,g^{k_a}\right)$ can compute a valid token, the authenticity of the token is trusted. Note that the holder of $\left(c_s,g^{k_a},g^{k_s}\right)$ cannot compute the token as
it requires the use of discrete logarithmic in the group $g$ which is computationally hard.

%\begin{align*}
%&& \left( g^{c_s} \cdot g^{k_s} \right)^{k_a} \neq \left( g^{k_a} \cdot g^{k_s} \right)^{c_s} \neq \left( g^{c_s} \cdot g^{k_a} \right)^{k_s}
%\end{align*}

\begin{figure}
%\begin{wide}
\begin{msc}{Authentication}

\setlength{\instdist}{2.5cm}
\setlength{\actionwidth}{2.6cm}

\declinst{as}{$\left\{c_s,k_{as}\right\}$}{$As$}
\declinst{a}{$\left\{g^{k_{as}},g^{k_{s}},k_a\right\}$}{$A$}
\declinst{s}{$\left\{c_s,k_{s},g^{k_a}\right\}$}{$S$}
\inlinestart[1.5cm][1.5cm]{exp1}{loop}{as}{s}
\nextlevel
\inlinestart{exp2}{opt}{a}{s}
\nextlevel[2]
\mess{$\left\{alive?, g^{k_s}\right\}$}{s}{a}
\nextlevel
\inlineend*{exp2}
\nextlevel[2]
\mess{$\left\{auth, g^{k_s}\right\}$}{a}{as}
\nextlevel
\action{compute $g^{c_s}$}{as}
\nextlevel[3]
\mess{$\left\{g^{k_s}, g^{c_s}\right\}$}{as}{a}
\nextlevel
\action{compute $h$}{a}
\nextlevel[3]
\mess{$\left\{g^{k_a},h\right\}$}{a}{s}
\nextlevel
\action{$c_{s}' = h(c_s)$}{as}
\action{$c_{s}' = h(c_s)$}{s}
\nextlevel[2]
\inlineend{exp1}
\end{msc}
%\end{wide}
\caption{}
\label{msc:authentication}
\end{figure}

\section{Attempt 2}

S now responds with a tuple of an initial seed $c$ and a shared key $k_{s,as}$ that should be used for later authentication
\begin{align}
S   &\rightarrow A      &&  \left\{g^{k_s}, m = (c_s,k_{s,as}) \cdot  \left( g^{k_{as}} \cdot g^{k_a} \right)^{k_s}\right\}\\
A   &\rightarrow As     &&  \left\{g^{k_s}, m' = m \div {\left(g^{k_s}\right)^{k_a}}\right\}
\end{align}

The sibling $As$ now decrypts $(c_s,k_{s,as}) = m' \div {\left(g^{k_s}\right)^{k_{as}}}$

After the registration the Authenticator $A$ and it's sibling $As$ can authenticate with the service $S$ by sending a token using their secrets $c_s$, $k_{s,as}$ and $k_a$. The message flow is depicted in figure~\ref{msc:authentication}.
\begin{align}
As  &\rightarrow A      &&  \left\{ c' = hash^{k_{s,as}}(c) \right\}\\
A   &\rightarrow S      &&  \left\{ g^{k_a}, t = c' \cdot \left(g^{k_s} \right)^{k_a} \right\}
\end{align}

The service provider $S$ can verify the authenticity of the token received from the Authenticator $A$ by computing
\begin{align*}
check: && t \stackrel{?}{=} hash^{k_{a,s}}(c) \cdot \left(g^{k_a} \right)^{k_s}
\end{align*}

\newpage
\section{A third attempt}

\paragraph{Standard RSA.}

A generator $GenRSA$ is ran to obtain the pair $(N,e,d)$, where

\begin{itemize}
    \item $N$ is the product of large safe primes $p$ and $q$.
    \
    \item $e$ is chosen at random from $\mathbb{Z}_{\phi(N)}^*$.
    \item $d$ is the inverse of $e$ given as $d = [e^{-1} \text{ mod } \phi(N)]$.
\end{itemize}

Let's define the functions $enc$ and $dec$ as
\begin{align*}
enc(m,e): && m^e \mod N\\
dec(c,d): && c^d \mod N
\end{align*}

Such that $dec\left(enc(m,e),d\right) = m$ if and only if $d$ is the inverse of $e$.

\paragraph{Mediated RSA.} Mediated RSA is similar to normal RSA with the difference that the private key $d$ is split into $n > 0$ segments.
\begin{align*}
&& d = \sum\limits_{i=1}^n d_i \mod \phi(N)
\end{align*}

The shares can be used for distributed decryption as the product of decryption the cipher with all shares is equal to decryption with the original decryption key.
\begin{align*}
&& \prod\limits_{i=1}^n& \quad c^{d_i} \equiv c^d \mod N
\end{align*}

Thus we have a schema where we can encrypt as with normal RSA, but can only decrypt if all key holders collaborate.

\begin{figure}
%\begin{wide}
\begin{msc}{Authentication}

\setlength{\instdist}{2.5cm}
\setlength{\actionwidth}{3cm}

\declinst{as}{$\left\{d_{As}\right\}$}{$As$}
\declinst{a}{$\left\{d_{A}\right\}$}{$A$}
\declinst{s}{$\left\{e\right\}$}{$S$}
\inlinestart[1.75cm][1.75cm]{exp1}{loop}{as}{s}
\nextlevel[2]
\mess{$m_1 = \left\{n\right\}$}{s}{a}
\nextlevel
\mess{$m_2 = \left\{n\right\}$}{a}{as}
\nextlevel
\action{$h = enc(n,d_{As})$}{as}
\nextlevel[4]
\mess{$m_3 = \left\{h\right\}$}{as}{a}
\nextlevel
\action{$h' = enc(n,d_{A})$}{a}
\nextlevel[4]
\action{$n' = h \times h' \mod n$}{a}
\nextlevel[4]
\mess{$m_4 = \left\{ n' \right\}$}{a}{s}
\nextlevel
\action{$dec(n',e) \stackrel{?}{=} n$}{s}
\nextlevel[4]
\inlineend{exp1}
\end{msc}
%\end{wide}
\caption{}
\label{msc:adwda}
\end{figure}

%*****************************************
%*****************************************
%*****************************************
%*****************************************


%-------------------------------------------------------------------------------
\section{Background}
%-------------------------------------------------------------------------------

Our approach to program analysis draws on work in three fields: Dataflow Analyis on Programs, Nonsmooth Optimization, and Automatic Differentiation. Dataflow Analysis models the flow of data through a program by tracking variable interactions and has applications in both compiler optimization and detection of security vulnerabilities, but suffers from high false positive rates that limit its utility. Nonsmooth Gradient Approximation involves a collection of methods that have been developed in the field of Nonsmooth Optimization for approximating gradients in cases where the gradient cannot be evaluated analytically. These methods make it possible to approximate gradients on discrete and nonsmooth functions in a principaled way based on the local behavior of the function. Finally, we draw on the field of Automatic Differentiation, which involves methods for computing gradients over programs compused of semi smooth numerical operations, but not general programs with discrete and nonsmooth operations.

\subsection{Definitions}

\subsection{Dataflow Analysis}

Dataflow analysis is a form of program analysis that models which variables can effect each other through a program, thereby tracking the flow of data. Typically, dataflow analysis is performed by using a set of rules for which inputs can effect which outputs for each operation. Then program variables, typically inputs, are marked with labels that are propogated through the program via the predefined rules so that internal program variables are labeled with variables that can potentially affect them.

Static dataflow analysis has been used as a tool for program analysis since it was first introduced in early 1970s ~\cite{kildall1973unified}. However, while it is useful for tasks like compile time optimization, its utility for detecting potential vulnerabilities is limited because it cannot distinguish impossible execution paths, resulting in high false positive rates. Dynamic dataflow analysis, or taint analysis, limits itself to possible executionsby operating on concrete inputs ~\cite{newsome2005dynamic}, but still suffers from high false positive rates caused by overapproximations such as the shift overapproximation shown in figure \ref{fig:taint_v_gradient}. These overapproximations in aggregate result in many inputs incorrectly being flagged as affecting taint sinks, which still limits the utility of taint analysis.

\begin{figure*}[t]
  \centering
  \begin{subfigure}[b]{0.32\textwidth}
    \includegraphics[width=\textwidth]{figs/f1}
  \end{subfigure}
  \begin{subfigure}[b]{0.32\linewidth}
    \includegraphics[width=\linewidth]{figs/f2}
  \end{subfigure}
  \begin{subfigure}[b]{0.32\linewidth}
    \includegraphics[width=\linewidth]{figs/f3}
  \end{subfigure}
  \vspace{-10pt}
   \caption{ Types of discrete and nonsmooth operations that occur in programs. \tc{f1} is a linear function that is still not differentiable because it operates on integers, while \tc{f2} performs bitwise \tc{and}, which is both discrete and nonsmooth. \tc{f3} illustrates a discontinuity due to branching and merging.}
  \label{fig:ex_funcs}
  \vspace{-10pt}
\end{figure*}

\subsection{Nonsmooth Gradient Approximation}

Programs are inherently difficult to differentiate because their behavior is generally discrete and nonsmooth, meaning that they cannot be differentiated directly. This behavior comes primarily from branches and memory operations, but also from many discrete operations such bitwise operators and integer arithmetic. Figure ~\ref{fig:ex_funcs} gives three examples of different types of operations that are difficult to differentiate. \tc{f1} is a simple linear function, but still is not analytically differentiablebecause it operates on integers and is discrete. \tc{f2} is also discrete, but introduces additional nonsmoothness with a \tc{and} operation. \tc{f3} shows how even on floating point operations branching can result in nonsmooth behavior.

Fortunately, extensive work has been done in the field of optimization on methods for approximating gradients over discrete and nonsmooth functions. The type of approximation that used depends on whether the function is convex, meaning that any point on the function can be connected to any other point by a straight line without going 'under' the function. Operations like addition and multiplication, or multiplying a variable by itself, are convex, but bitwise operations and discontinuities from branching generally are not. When the functions being analyzed are convex, a type of gradient approximation called subgradients may be used that behave similarly to gradients on smooth functions with regard to composition and global convergence properties. When functions are nonconvex, an extension of subgradients called generalized gradients may be used that allow optimization to still be performed, although they relax the composition and convergence properties of subgradients ~\cite{clarke1990optimization, rockafellar2009variational}.

\begin{figure}
  \centering
  \begin{subfigure}[b]{0.48\columnwidth}
    \includegraphics[width=\linewidth]{figs/convex_example}
    \caption{\label{fig:convex_ex} Convex function. Any line between two points on the function will pass 'over' the function.}
  \end{subfigure}
  \hspace{0.15cm}
  \begin{subfigure}[b]{0.48\columnwidth}
    \includegraphics[width=\linewidth]{figs/lipschitz_ex}
  \caption{\label{fig:lipschitz_ex} Function with local Lipschitz Constant.}
  \end{subfigure}
  \vspace{-10pt}
  \caption{\label{fig:convex_lip_ex} Subgradient and Generalized Derivative examples.}
  \vspace{-15pt}
\end{figure}

\noindent \textbf{Convex Gradient Approximation.} Subgradients have been used in these cases to successfully optimize a wide variety of nonsmooth problems when the functions involved were convex ~\cite{beck2017first}. Subgradients are defined to be any vector under a convex function that intersects the function at the point the subgradient is evaluated, or formally, a vector $v$ is a subgradient of a function $f$ at a point $\bar{x}$ if:

\vspace{-10pt}\begin{align}
  f\left(x\right) \geq f\left(\bar{x}\right) + \langle v, x-\bar{x} \rangle \textrm{ for all } x \in \textrm{dom}\left(f\right)
\end{align}

This definition means that any point on the subgradient vector $v$ ($\langle v, x-\bar{x}$) for which $f$ is defined ($x \in \textrm{dom}\left(f\right)$), the value of the subgradient starting from the point it touches the function $f$ can continue to touch $f$ but can never be greater than $f$. This means that there can actually multiple subgradients for a given point of a discrete convex function, as in the example in figure \ref{fig:subgradient}. It is important to note that subgradients can also be defined in terms of being greater than the function in the case one wants to find the highest values of $f$, but conventionally used with vectors under the surface of a function.

Subgradients are valid for discrete functions and thus can be computed on integer operations like \tc{f1} in figure \ref{fig:ex_funcs}. The chain rule also applies to them, meaning that subgradients of several sequential operations can be multiplied together to give a valid subgradient for combined operations.

\noindent \textbf{Nonconvex Gradient Approximation.} Nonconvex operations (which have multiple minima and maxima) like \tc{and} in \tc{f2} of Figure \ref{fig:ex_funcs}, have points for which there is no valid subgradient. To address these cases, generalized gradients are used to approximate gradients over nonconvex functions. Generalized gradients are composed of generalized directional derivatives, which work like subgradients, but only 'look' in single direction, starting at the point where they are evaluated and remaining under the function. This allows generalized directional derivatives to be taken at points where there is no valid subgradient, as can be seen in figure \ref{fig:dir_deriv}.

%\begin{figure}
  %\centering
  %\includegraphics[width=0.7\columnwidth]{figs/directional_deriv_example}
  %\caption{\label{fig:dir_deriv} Directional derivative applied to a nonconvex function \tc{x \& 4}.}
%\end{figure}

We denote a generalized directional derivative in a directon $v$ as $f^\circ \left(x;v\right)$, which is formally defined as follows:

\vspace{-10pt}\begin{align}
  f^\circ \left(\bar{x}; v\right) = \lim \sup_{x \rightarrow \bar{x}, \lambda \downarrow 0} \frac{f\left(x + \lambda v\right) - f\left(x\right)}{\lambda}
\end{align}

\noindent Here $\bar{x}$ is the point at which the directional derivative is evaluated, $x$ can be any other point in the domain of $f$, and $\lambda$ is a distance along the vector $v$ that the derivative is taken in. The $\lim \sup_{x \rightarrow \bar{x}}$ notation indicates that the directional deriviative takes on the largest possible value for the two points $x$ and $x+\lambda v$ closest to $\bar{x}$ in the function's domain. The generalized gradient at a given point is then the set of all generalized directional derivatives for all possible directions from that point.

\begin{figure}
  \centering
  \begin{subfigure}[b]{0.45\columnwidth}
    \includegraphics[width=\linewidth]{figs/subgradient_example}
    \caption{\label{fig:subgradient} Example of subgradient on discrete nonsmooth function.}
  \end{subfigure}
  \hspace{0.15cm}
  \begin{subfigure}[b]{0.5\columnwidth}
    \includegraphics[width=\linewidth]{figs/directional_deriv_example}
  \caption{\label{fig:dir_deriv} Directional derivative applied to a nonconvex function \tc{x \& 4}.}
  \end{subfigure}
  \vspace{-10pt}
  \caption{\label{fig:grad_exs} Subgradient and Generalized Derivative examples.}
  \vspace{-15pt}
\end{figure}

\noindent \textbf{Lipschitz Continuity.} One important requirement of subgradients and generalized gradients is that the gradients of the functions they are computed over must be bounded. This property is called 'Lipschitz Continuity', and is defined as

\vspace{-10pt}\begin{align}
  |f\left(x_1\right) - f\left(x_0\right)| \leq K|x_1 - x_0|\\
  \textrm{ for all } x_1, x_0 \in x+\eta B \nonumber
\end{align}

\noindent where $K$, the Lipschitz Constant, is the bound on the gradient and $B$ denotes the size of the region in the domain of $f$ where the property is defined. Fortunately for the purpose of modeling programs, all operations ultimately occur on fixed width data types, limiting the range of values they can take on to the minimum and maximum of these data types and making them Lipschitz Continuous even in the worse case where a function jumps between extreme values.

\noindent \textbf{Proximal Gradients.} Subgradients and generalized gradients are useful guides analysis of discrete and nonconvex functions, but cannot be evaluated on general functions without sampling every possible value, which obviously isn't practical. To make evaluation tractable, we borrow another gradient approximation method from the discrete optimization literature called proximal gradients ~\cite{parikh2014proximal}. Instead of combining every possible surface or supporting vector under a function like subgradients and generalized gradients, proximal gradients use the minimum point within a soft bounded region. This region is defined by a cost function that increases quadratically with distance from the evaluation point.

By representing the gradient with a single value and bounding the region in which it is evaluated, the proximal gradient makes the general problem of evaluating gradients tractable. The proximal operator is defined as follows when evaluated on a given point $\bar{x}$:

\vspace{-10pt}\begin{align}
  prox_f\left(\bar{x}\right) &= argmin_x \left(f\left(x\right) + \tfrac{1}{2}|| x-\bar{x}||_2^2\right)
\end{align}

The notation $argmin_x$ indicates that the operator selects the value of $x$ that minimizes both value of function $f\left(x\right)$ and the distance cost $\left(\tfrac{1}{2}\right)|| x-\bar{x}||_2^2$ that increases quadratically with the distance of $x$ from $\bar{x}$. Evaluating the proximal operator will give the minimum point near the point at which it is evaluated. This point can then be used to compute the largest directional derivative in the region near the point.

% PROXIMAL GRADIENT
\vspace{-10pt}\begin{align}
  prox_{\nabla f}\left(\bar{x}\right) &= \frac{f\left(\bar{x}\right) - f\left(prox_f\left(\bar{x}\right)\right)}{\bar{x} - prox_f\left(\bar{x}\right)}
\end{align}

\noindent \textbf{Sampling Bounds.} On functions that are Lipschitz Continuous, the region that must be sampled to evaluate the proximal operator can be hard bounded as a factor of the Lipschitz Constant $K$. This bound is derived by observing that the cost function of the proximal operator is upper bounded by the initial function value and lower bounded by linear function decreasing at rate $K$ from the point $\bar{x}$ for $prox_f(\bar{x})$.

% BOUNDS DERIVATION 1
\vspace{-10pt}\begin{align*}
  \left(f\left(\bar{x}\right) - K*||prox_f\left(\bar{x}\right) - \bar{x}||_2  + \tfrac{1}{2}|| prox_f\left(\bar{x}\right)-\bar{x}||_2^2\right)\\
    \leq \left(f\left(prox_f\left(\bar{x}\right)\right) + \tfrac{1}{2}|| prox_f\left(\bar{x}\right)-\bar{x}||_2^2\right)\\
    \leq f\left(\bar{x}\right) + \tfrac{1}{2}||\bar{x} - \bar{x}||_2^2
\end{align*}

Here the term $f\left(\bar{x}\right) - K*||prox_f\left(\bar{x}\right)$ represents the minimum value $f$, can take as $prox_f\left(\bar{x}\right)$ moves farther from $\bar{x}$ due to its Lipschitz Constant $K$, while the value $f\left(\bar{x}\right) + \tfrac{1}{2}||\bar{x} - \bar{x}||_2^2$ represents the value being minimized at the initial point $\bar{x}$. This can be simplified by dropping the middle term:

\vspace{-10pt}\begin{align*}
  \left(f(\bar{x}) - K*||prox_f\left(\bar{x}\right) - \bar{x}||_2  + \tfrac{1}{2}|| prox_f\left(\bar{x}\right)-\bar{x}||_2^2\right)\\
    \leq f\left(\bar{x}\right)
\end{align*}

as well as subtracting $f\left(\bar{x}\right)$ from both sides:

\vspace{-10pt}\begin{align*}
  \left( - K*||prox_f\left(\bar{x}\right) - \bar{x}||_2  + \tfrac{1}{2}|| prox_f\left(\bar{x}\right)-\bar{x}||_2^2\right) &\leq 0
\end{align*}

Adding $K*||\bar{x}-x||_2$ to both sides, dividing by $||\bar{x}-x||$, and multiplying by 2 gives a bound of $2K$ on the distance between the inital point $\bar{x}$ and the proximal operator result $prox_f\left(\bar{x}\right)$.

\vspace{-10pt}\begin{align}
    || prox_f(\bar{x})-\bar{x}||_2 &\leq 2K
\end{align}

This means that the distance the result of the proximal operator from its initial $\bar{x}$ is bounded by at most twice the Lipschitz Constant for the function, $2K$. In practice a scaling factor $\lambda$ is often used with the proximal operator to adjust the relative weighting of distance from the evaluation point. When the scaling factor is added the bound becomes $2 \lambda K$. We then denote the Proximal Operator with a Lipschitz sampling bound $K$ and scaling factor $\lambda$ as

\vspace{-10pt}\begin{align}
  prox_f\left(\bar{x}, K, \lambda\right) &= argmin_x \left(f\left(x\right) + \tfrac{1}{2\lambda}|| x-\bar{x}||_2^2\right) \nonumber\\
  \textrm{s.t. } &|| x-\bar{x}||_2 \leq 2 \lambda K
\end{align}

\noindent and the associated bounded Proximal Gradient is:

\vspace{-10pt}\begin{align}
prox_{\nabla f}\left(\bar{x}, K, \lambda\right) &= \frac{f\left(\bar{x}\right) - f\left(prox_f\left(\bar{x}, K, \lambda\right)\right)}{\bar{x} - prox_f\left(\bar{x}, K, \lambda\right)}
\end{align}

This provides theoretically sound and adjustable method for bounding the sampling region on Lipschitz Continuous functions when the Lipschitz Constant $K$ is known.

\gabe{Probably should also explain smoothing here}

\subsection{Automatic Gradient Computation}

Automatically computing gradients over programs composed of analytically differentiable operations is a process called Automatic Differentiation, or 'AutoDiff', that has been a longstanding tool in computational modeling and is a core component of deep learning frameworks such as Tensorflow ~\cite{Wengert:1964:SAD:355586.364791, tensorflow2015-whitepaper}. Autodiff makes use of the chain rule in calculus to compute the gradient over a program as a series of gradients of individual operations multiplied together, each of which can be computed analytically. However, AutoDiff methods and frameworks have traditionally limited themselves to working with  functions that are at minimum piecewise differentiable, and do not support nonsmooth operations found in general programs such as bitwise operations or merge operations. By leveraging proximal gradients, the Autodiff approach can be extended to work with nonsmooth operations as well, thereby making it possible to compute gradients over general programs.


\subsubsection{\stid {3.01} xSDK Sub-project: multiprecision}
\paragraph{Overview}
The multiprecision focus effort is a cross-laboratory effort to develop and deploy production-ready
mixed precision algorithms for modern hardware architectures.
The focus effort is ECP's response to two trends we see in the evolution of HPC hardware:
1) An increasing number of processor designs feature low precision special function units
to accommodate the demand of the Machine Learning community for high compute power
in low precision formats;
2) The gap between compute power on the one hand and memory bandwidth
on the other hand keeps increasing, making data access and communication prohibitively
expensive compared to arithmetic operations.
To address these trends, scientists from ECP partners teamed up to develop strategies and deploy
production-ready software that reflects the architecture trends in the algorithm design. The multiprecision focus team has bi-weekly virtual meetings in which the progress on the different efforts is presented and discussed. As another integral part of the bi-weekly phone calls, we established a series of short talks where each meeting is commenced by an invited talk presenting an idea, success story, or progress update on mixed precision functionality to the audience. The short talks have quickly become a line-up of well-known researchers from ECP, the global academic HPC community, and industry.

\paragraph{Key Challenges}
Generally, there exists a strong relationship between the precision used in arithmetic operations and the accuracy of the computed result. Since scientific applications need to provide high-quality output, replacing high precision formats with low precision formats throughout a complete application code is generally not feasible. Instead, to utilize lower precision formats, the underlying numerical algorithms have to be redesigned to employ low precision formats for the most time-consuming parts while preserving high accuracy in the solution. In this context, arithmetic operations are only one aspect. As the execution time of many scientific applications is dominated by communication and memory access, the algorithm redesign also has to include strategies for compressing data to reduce the pressure on the memory bandwidth. This aspect becomes even more relevant as the arithmetic power continues to grow faster than the memory bandwidth, therewith widening the gap between arithmetic performance and memory performance, see Figure~\ref{fig:xsdk-machinebalance}.

\begin{figure}[htb]
	\centering
	\includegraphics[width=.8\columnwidth]{projects/2.3.3-MathLibs/2.3.3.01-xSDK/xSDK-machinebalance.pdf}
	%\includegraphics[width=.8\columnwidth]{xSDK-machinebalance.pdf}
	\caption{\label{fig:xsdk-machinebalance} Evolution of the machine balance of processors over different hardware generations.}
\end{figure}

\paragraph{Solution Strategy}
In the multiprecision focus effort, we identified several action lines. These include the development of
\begin{itemize}
\item a \textbf{memory accessor} that separates the precision format used in the arithmetic operations from the precision format used for memory operations and communication. This strategy can increase the accuracy of memory-bound low precision algorithms without impacting the performance, or increase the performance for memory-bound high precision algorithms that can tolerate or compensate for some information loss in the memory operations.
\item \textbf{low precision kernels} using the reduced precision formats introduced for the machine learning community,
\item \textbf{mixed precision multigrid methods},
\item production-ready \textbf{mixed precision iterative refinement} variants for dense and sparse direct solvers,
\item \textbf{mixed precision Krylov} solvers that accelerate the solution process by using a lower precision format for parts of the computations,
\item \textbf{mixed precision preconditioners} that preserve the preconditioner quality but reduce the preconditioner application cost,
\item \textbf{mixed precision FFT} algorithms.
\end{itemize}
Furthermore, to improve mixed precision interoperability, we investigate how to allow users to combine components from different xSDK libraries in different precision formats.

\paragraph{Recent Progress}
Recent progress includes
\begin{itemize}
\item the deployment of accessor-BLAS functionality for CPUs, AMD GPUs, and NVIDIA GPUs based on the memory accessor. This functionally is equivalent to memory-bound low precision BLAS, achieves the same performance, but provides higher numerical accuracy because all arithmetic operations are handled in high precision. Currently supported are the \texttt{dot}, \texttt{gemv}, and \texttt{trsv} kernels on NVIDIA GPUs.
\item the development of a Compressed Basis (CB-) GMRES solver that can solve linear problems faster than the standard GMRES solver by storing the Krylov basis vectors in lower precision and therewith reducing the main memory access. CB-GMRES solver is a plug-in replacement for standard GMRES as it achieves the same solution approximation accuracy, and all conversion and data compression is handled by the memory accessor and thus hidden from the user.
\item the deployment of a production-ready mixed precision iterative refinement variant of the GMRES solver that can accelerate time-to-solution.
\item the prototype implementation of distributed mixed precision iterative refinement solver based on a sparse factorization,
\item the deployment of a production-ready LU-based mixed precision iterative refinement solvers for AMD GPU architectures.
\item research on FFT algorithms using lower precision for inter-node communication.
\end{itemize}

For a more comprehensive overview of the recent progress, the multiprecision focus team created an ECP report on ``Advances in Mixed Precision Algorithms: 2021 Edition'' that is available to the ECP community in Confluence. A significant portion of the material is currently under consideration for journal publication.

\paragraph{Next Steps}

Our next efforts include the publication of a compacted version of the ECP report on advances in mixed precision algorithms, the deployment of accessor-BLAS for AMD GPUs and multicore CPUs,  and the advancement of multiprecision capabilities for solvers, preconditioners, and other ECP-relevant kernels in xSDK libraries.


\documentclass[a4paper]{article}
\usepackage[a4paper,top=3cm,bottom=2cm,left=3cm,right=3cm,marginparwidth=1.75cm]{geometry}
\usepackage[english]{babel}
\usepackage[utf8x]{inputenc}
\usepackage{amsmath}
\usepackage{graphicx}

% Macro shortcuts
\newcommand{\mb}[1]{{\mathbf{#1}}}
\newcommand{\II}{\mb{I}}
\newcommand{\PP}{\mb{P}}
\newcommand{\uu}{\mb{u}}
\newcommand{\eepsilon}{\boldsymbol{\epsilon}}

\title{Stretch Bar}
\author{Xuchen Han }
\date{February 2021}

\begin{document}
\maketitle
\section{Problem Setup}

\begin{figure}[!h]
  \centering
  \includegraphics[width=0.5\columnwidth]{stretch_bar.png}
  \caption{\label{fig:problem}
    Problem description.}
\end{figure}

We consider an elastic block stretched on the side by Dirichlet boundary
conditions. The bar occupies the domain $[-\frac{Lx}{2}, \frac{Lx}{2}]
\times [-\frac{Ly}{2}, \frac{Ly}{2}] \times [-\frac{Lz}{2}, \frac{Lz}{2}]$.
We assume linear elasticity with Young's modulus $E$ and Poisson ratio $\nu$.
We apply Dirichlet boundary condition at $y = \frac{L}{2}$ and
$y = -\frac{L}{2}$ so that the bar is stretched by a factor of $s = 1+\epsilon$
in the $y$-direction. In other words, the boundary condition applied on the
displacement $\uu = (u_1(x,y,z), u_2(x,y,z), u_3(x,y,z))^T$ takes the form
\begin{align*}
  u_2(x,-\frac{L}{2},z) = -\frac{\epsilon L}{2} \\
  u_2(x,\frac{L}{2},z) = \frac{\epsilon L}{2}.
\end{align*}
We use the symmetry of the problem and restrict our attention to a quarter of
the original domain $[0, \frac{Lx}{2}] \times [-\frac{Ly}{2}, \frac{Ly}{2}]
\times [0, \frac{Lz}{2}]$ and add boundary conditions
\begin{align*}
  u_1(0,y,z) = 0, \\
  u_3(x,y,0) = 0,
\end{align*}
All other boundaries assume zero Neumann (no stress) boundary conditions.
We assume zero gravity for simplicity.

\section{Solution}
We use $\uu = (u_1(x,y,z), u_2(x,y,z), u_3(x,y,z))^T$ to denote displacement,
$\PP(x,y,z)$ to denote first Piola-Kirchhoff stress density, $\eepsilon$ to
denote the infinitesimal strain $\left(\epsilon_{ij} = \frac{1}{2}
    (\frac{\partial u_i}{\partial X_j} + \frac{\partial u_j}{\partial
      X_i})\right)$.
There is no external force as we assume zero gravity.

Under the stretching boundary condition, the bar should experience constant
uni-axial stress. Therefore, we guess the stress has the form
\begin{align*}
  \PP
  =
  \begin{bmatrix}
    0 & 0 & 0 \\
    0 & \sigma & 0 \\
    0 & 0 & 0
  \end{bmatrix}.
\end{align*}
In other words, the stress has a constant $2-2$ component and zeros everywhere
else.
With this guess, the static equilibrium equation
\begin{align*}
  \nabla \cdot \PP = 0
\end{align*}
is obviously satisfied.
We then plug this guess into the constitutive model
\begin{align*}
  \PP = 2\mu \eepsilon + \lambda \text{tr}(\eepsilon) \II,
\end{align*}
where $\mu$ and $\lambda$ are the Lame parameters related to $E$ and $\nu$ via
\begin{align}
   & \mu = \frac{E}{2(1+\nu)},\label{eq:lame1}               \\
   & \lambda = \frac{E\nu}{(1+\nu)(1-2\nu)}\label{eq:lame2},
\end{align}
and get the following equations on the diagonal:
\begin{align}
  P_{11} & = 2\mu\epsilon_{11} + \lambda(\epsilon_{11} + \epsilon_{22} +
  \epsilon_{33}) = 0 \label{eq:P11},                                     \\
  P_{22} & = 2\mu\epsilon_{22} + \lambda(\epsilon_{11} + \epsilon_{22} +
  \epsilon_{33}) = \sigma \label{eq:P22},                                \\
  P_{33} & = 2\mu\epsilon_{33} + \lambda(\epsilon_{11} + \epsilon_{22} +
  \epsilon_{33}) = 0\label{eq:P33},
\end{align}
and the following equations on the off-diagonal:
\begin{align}
  \frac{P_{12}}{2\mu} & = \epsilon_{12} = \epsilon_{21} = 0,
  \label{eq:P12}                                             \\
  \frac{P_{13}}{2\mu} & = \epsilon_{13} = \epsilon_{31} = 0,
  \label{eq:P13}                                             \\
  \frac{P_{23}}{2\mu} & = \epsilon_{23} = \epsilon_{32} = 0.
  \label{eq:P23}
\end{align}
Solving equations \eqref{eq:P11} through \eqref{eq:P33}, and utilizing the
identity in \eqref{eq:lame1} and \eqref{eq:lame2}, we get
\begin{align}
  \epsilon_{22} & = \frac{\sigma}{E}\label{eq:e22},                       \\
  \epsilon_{11} & = \epsilon_{33} = \frac{-\nu \sigma}{E}. \label{eq:e11}
\end{align}
Integrating equations \eqref{eq:e22} and \eqref{eq:e11}, we get
\begin{align}
  u_1(x,y,z) & = \frac{-\nu \sigma}{E}x + h_1(y,z), \label{eq:u1} \\
  u_2(x,y,z) & = \frac{\sigma}{E}y + h_2(x,z), \label{eq:u2}      \\
  u_3(x,y,z) & = \frac{-\nu \sigma}{E}z + h_3(x,y), \label{eq:u3}
\end{align}
where $h_1, h_2, h_3$ are arbitrary functions resulting from integration.
Using boundary conditions $u_2(x,\frac{L}{2},z) =\frac{\epsilon L}{2}$ and
$u_2(x,-\frac{L}{2},z) = -\frac{\epsilon L}{2}$, we get
\begin{align*}
  h_2(x,z) = 0, \\
  \sigma = \epsilon E.
\end{align*}
Using boundary conditions $u_1(0,y,z) = 0$ and $u_3(x,y,0) = 0$, we get
\begin{align*}
  h_1(y, z) = 0, \\
  h_3(x, y) = 0.
\end{align*}
Therefore, the final displacement is given by:
\begin{align*}
  u_1(x,y,z) & = -\nu \epsilon x, \\
  u_2(x,y,z) & = \epsilon y,      \\
  u_3(x,y,z) & = -\nu \epsilon z.
\end{align*}
We easily verify the displacement above satisfies equations \eqref{eq:P12}
through \eqref{eq:P23}.
\end{document}