\epigraph{``Since Newton, mankind has come to realise that the law of physics are always expressed in the language of differential equations"}{Steven Strogatz}

\section{RNN}
\subsection{Introduction}
Weather forecasting has traditionally been done by physical models of the atmosphere, which are unstable to perturbations, and thus are inaccurate for large periods of time\cite{why_rnn}. Since machine learning techniques are more sensitive to perturbations, it would be logical to combine a neural network with a physical model. Weather forecasting is a sequential data problem, therefore, a recurrent neural network is the most suitable option for this task.

\begin{definition}
A recurrent neural network is a class of artificial neural networks where connections between nodes form a directed graph along a temporal sequence.
\end{definition}

Before, we delve into the specific example of using a recurrent neural network to predict the future state of the atmosphere, it is necessary to review what a recurrent neural network is. Recurrent Neural Networks (RNNs) are neural networks that are used in situations where data is presented in a sequence. For example, let's say you want to predict the future position of a fast-moving ball. Without information on the previous position of the ball, it is only possible to make an inaccurate guess. If you had, however, a large number of snapshots of the previous position, you are then able to predict the future position of the ball with some certainty. RNNs excel at modelling sequential data such as this. This is due to sequential memory.

In order to intuitively understand sequential memory, the prime example would be the alphabet. While it is easy to say the alphabet from A-Z, it is much harder to go from Z-A. There is a logical reason why this is difficult. As a child, you learn the alphabet in a sequence. Sequential memory is a mechanism that makes it easier for your brain to recognise sequence patterns.

In a traditional neural network, there is an input layer, hidden layer, and an output layer. In a recurrent neural network, a loop is added to pass information forward as seen in the diagram below (provided by Towards Data Science)\cite{intro_rnn}:

\begin{figure}[H]
    \centering
    \includegraphics[width=.2\linewidth]{Images/rnn.png}
    \caption{Visualisation of a Recurrent Neural Network}
\end{figure}

The information that is forwarded is the hidden layer, which is a representation of previous inputs. How this works in practise is that you initialise your network layers and the initial hidden state. The shape and dimension of the hidden state will be dependent on the shape and dimension of your recurrent neural network. Then you loop through your inputs, pass the relevant parameter and hidden state into the RNN. The RNN returns the output and a modified hidden state. Last you pass the output of the hidden state to the output layer of the model, and it returns a prediction.

There is, however, a major problem known as short-term memory. Short-term memory is caused by something known as the vanishing gradient problem, which is also prevalent in other neural network architectures. As the RNN processes more steps, it has troubles retaining information from previous steps. Short-Term memory and the vanishing gradient is due to the nature of back-propagation. This can be comprehended through understanding how a neural network is trained\cite{intro_rnn}.

\begin{definition}
Back-propagation is an algorithm used to train and optimise neural networks.
\end{definition}

To train a recurrent neural network, you use an application of back-propagation called back-propagation through time. Training a neural network has three major steps. First, the relevant data vector is normalised between 0 and 1, the vector is fed into the RNN, and it goes through an activation function. The activation function utilised in the software is the rectified linear activation function\cite{lstm_rnn}.

\begin{definition}
The rectified linear activation function is a piece-wise linear function that will output the input directly if is positive, otherwise, it will output zero.
\end{definition}

The function is linear for values greater than zero, meaning it has a lot of the desirable properties of a linear activation function when training a neural network using back-propagation. Yet, it is a nonlinear function as negative values are always output as zero. As a result, the rectified function is linear for half of the input domain and nonlinear for the other half, it is referred to as a piece-wise linear function\cite{relu}. This nonlinear element is extremely important if the system has a nonlinear component, for example in predicting the evolution of the future state of the atmosphere.

\begin{figure}[H]
    \centering
    \includegraphics[width=.65\linewidth]{Images/relu.png}
    \caption{Sketch of the Rectified Linear Activation Function}
\end{figure}

Second, it outputs the results. Third, it compares the prediction to the observed state using a loss function.

\begin{definition}
A loss function outputs an error value which is an estimate of how poorly the network is performing.
\end{definition}

The lost function that will be utilised in the software will be the function for mean squared error. The reason for choosing this particular function is that it heavily penalises large errors, as it squares the difference between the predicted and actual value. A large error in a weather forecast is highly undesirable, hence, the use of this function. The function is represented below:

\begin{equation}
    MSE = \frac{1}{n}\sum_{i=1}^n(Y_i-\hat{Y_i})^2
\end{equation}

If  a vector of $n$ predictions is generated from a sample of $n$ data points on all variables, and $Y$ is the vector of observed values of the variable being predicted, with $\hat{Y_i}$ being the predicted values.

\begin{definition}
Mean squared error is the average squared difference between the estimated values and the actual value.
\end{definition}

Returning to the training of the RNN, it uses that error value from the loss function to do back propagation which calculates the gradients for each time step in the network. The gradient is the value used to adjust the networks internal weights, allowing the network to learn. The bigger the gradient, the bigger the adjustments and vice versa. Here is where the problem lies. When doing back propagation, the gradient of the current time step is calculated with respect to the effects of the gradients, in the time step before it. So if the adjustments to the time step before it is small, then adjustments to the current time step will be even smaller.  The gradient values will exponentially shrink as it propagates through each time step. That causes gradients to exponentially shrink as it back propagates down. The earlier layers fail to do any learning as the internal weights are barely being adjusted due to extremely small gradients.

Because of vanishing gradients, the RNN doesn’t learn the long-range dependencies across time steps. So not being able to learn on earlier time steps causes the network to have a short-term memory. In order to combat this, a long short-term memory is used\cite{intro_rnn}.

\subsection{LSTM}
LSTM's were created as a solution to the short-term memory problem. They have internal mechanisms called gates that can regulate the flow of information. These gates can learn which data in a sequence is important to keep or throw away. By doing that, it can pass relevant information down the long chain of sequences to make predictions. For example, if you were interested in buying a particular product, you might read a review in order to determine if the purchase of the product is a good decision. When you read a review, your brain subconsciously only remembers important keywords. You pick up words like ``amazing", ``superb", or ``awful", you don't remember words such as "the", "as", or "because". This is what an LSTM does, it learns to keep only the relevant information to make predictions.

An LSTM has a similar control flow as a recurrent neural network. It processes data passing on information as it propagates forward. The differences are the operations within the LSTM’s cells. The core concept of LSTM’s are the cell state, and it’s various gates. The cell state is the method by which information is transferred down the sequence chain. The cell state, in theory, can carry relevant information throughout the processing of the sequence. So even information from the earlier time steps can make its way to later time steps, reducing the effects of short-term memory. As the cell state goes on its journey, information gets added or removed to the cell state via gates\cite{lstm_rnn}.

\begin{definition}
A gate is an electric circuit with an output which depends on the combination of several inputs.
\end{definition}

Gates contain the sigmoid activation function. The sigmoid activation function squishes values between 0 and 1. That is helpful to update or forget data because any number getting multiplied by 0 is 0, causing values to disappears or be ``forgotten". Any number multiplied by 1 is the same value therefore that value stays the same or is ``kept".

\begin{figure}[H]
    \centering
    \includegraphics[width=.65\linewidth]{Images/sigmoid.png}
    \caption{Sketch of the Sigmoid Activation Function}
\end{figure}

There are three types of gates utilised within a neural network: a forget gate, an input gate, and an output gate. A forget gate decides what information should be thrown away or kept. Information from the previous hidden state and information from the current input is passed through the sigmoid function. An input gate is where the previous hidden state and current input are passed into a sigmoid function. The output gate decides what the next hidden state should be. The hidden state is also used for predictions. First, we pass the previous hidden state and the current input into a sigmoid function. Then we pass the newly modified cell state to the rectified linear activation function. We multiply the rectified linear activation function output with the sigmoid output to decide what information the hidden state should carry. The output is the hidden state. The new cell state and the new hidden state is then carried over to the next time step\cite{lstm_rnn}.

\subsection{Convolutional LSTM Network}
The formulation of a numerical weather prediction model is a spatiotemporal sequence forecasting problem that can be solved under a general sequence-to-sequence learning framework.

\begin{definition}
A spatiotemporal sequence is a sequence that contains both spatial and temporal information.
\end{definition}

In order to better model the spatiotemporal relationships, this project will utilise ConvLSTM layers; which was proposed by Xingjian Shi et. al\cite{convlstm}. This spatiotemporal sequence forecasting problem is different from the one-step time series forecasting problem because the prediction target of the problem is a sequence which contains both spatial and temporal structures. Although a LSTM layer has proven powerful for handling temporal correlation, it contains too much redundancy for spatial data. To address this problem, this project will use a ConvLSTM layer which has convolutional structures in both the input-to-state and state-to-state transitions\cite{convlstm}.

The major drawback of LSTMs is in the handling spatiotemporal data due to its usage of full connections in input-to-state and state-to-state transitions in which no spatial information is encoded. To overcome this problem, a distinguishing feature of a ConvLSTM cell is that all the inputs and gates of the ConvLSTM layer are 3D tensors whose last two dimensions are spatial dimensions. To get a better picture of the inputs and states, we may imagine them as vectors standing on a spatial grid. The ConvLSTM determines the future state of a certain cell in the grid by the inputs and past states of its local neighbour. This can easily be achieved by using a convolution operator in the state-to-state and input-to-state transitions\cite{convlstm}. This is what the ConvLSTM layer does.

\begin{figure}[H]
    \centering
    \includegraphics[width=.8\linewidth]{Images/convlstm.png}
    \caption{The architecture of a ConvLSTM cell.}
\end{figure}

\section{Dataset}
\subsection{ERA5 Atmospheric Reanalysis Dataset}\label{era5_dataset}
ERA5 provides hourly estimates of a large number of atmospheric, land and oceanic climate variables. The data covers the Earth on a 30km grid and resolves the atmosphere using 137 levels from the surface up to a height of 80km. ERA5 includes information about uncertainties for all variables at reduced spatial and temporal resolutions. Quality-assured monthly updates of ERA5 are published within 3 months of real time. Preliminary daily updates of the dataset are available to users within 5 days of real time. ERA5 combines vast amounts of historical observations into global estimates using advanced modelling and data assimilation systems\cite{era5}.

The ERA5 reanalysis dataset was used for training, validating and testing the performance of the neural network architecture. Reanalysis datasets provide the best guess of the atmospheric state at any point in time by combining a forecast model with the available observations. The raw data is available hourly for 40 years from 1979 to 2019 on a $0.25^{\circ}$ latitude-longitude grid ($721 \times 1440$ grid points) with 37 vertical levels. Since this raw dataset is quite significant, it is necessary to regrid the dataset to a lower resolution and use a smaller fraction of the available dataset\cite{rasp2020weatherbench}.  The poles were excluded from the dataset in order to avoid a singularity, and the potential negative impact that could have on predictive ability of the neural network.

It was ultimately decided to use a spatial resolution of $1^{\circ}$ ($179 \times 360$ grid points) and a temporal resolution of 2 hours. The data is split into yearly NetCDF files for each variable. The entire dataset at $0.25^{\circ}$ resolution has a size of 400GB, before the dataset was interpolated to a grid of a lower resolution. The prognostic variables of interest are temperature and geopotential, and were chosen based on meteorological considerations. Geopotential, and temperature are prognostic state variables in most physical numerical weather prediction and climate models\cite{rasp2020weatherbench}.

\subsection{Integrated Forecasting System}\label{ifs_section}
The Integrated Forecast System is a global numerical weather prediction system developed and maintained by the European Centre for Medium-Range Weather Forecasts organisation. The version of the IFS run at ECMWF is often referred to as the ``ECMWF" or the ``European model" in North America, to distinguish it from the American GFS. It comprises of a spectral atmospheric model with a terrain-following vertical coordinate system coupled to a 4D variational data assimilation system. In 1997 the IFS became the first operational forecasting system to use a 4D variational, data assimilation system

\begin{definition}
4D dimensional variational data assimilation system adjusts a short-range forecast, called the background, in space and time to bring it into closer agreement with meteorological observations.
\end{definition}

\begin{figure}[H]
    \centering
    \includegraphics[width=.8\linewidth]{Images/ifs.jpg}
    \caption{ECMWF Integrated Forecast System}
\end{figure}

It is one of the predominant global medium-range models in general use worldwide; its most prominent rival in the 6–10 day medium range include the American Global Forecast System, the Canadian Global Environmental Multiscale Model and the UK Met Office Unified Model. For context, Met Éireann utilises the IFS for medium-range forecasts; however for short-range forecasts, Met Éireann uses its own HARMONIE-AROME NWP model. The operational configuration consists of $1000 \times 900$ grid points in the horizontal at 2.5km resolution, with 65 levels in the vertical. A 54-hour forecast is produced four times a day, at 00Z, 06Z, 12Z and 18Z\cite{harmonie_arome_nwp}.

The IFS is the gold standard of medium-range numerical weather prediction. The current IFS deterministic forecast is computed on a cluster with 11,664 cores. One 10 day forecast at 10 km resolution takes around 1 hour of real time to compute\cite{rasp2020weatherbench}. The Integrated Forecasting System will be used as a comparison against the neural network.

To provide physical baselines more in line with the current resolution of the neural network, it was compared against the IFS model at two coarser horizontal resolutions, T42 (approximately $2.8^{\circ}$) with 62 vertical levels and T63 (approximately $1.9^{\circ}$) with 137 vertical levels. It must be noted that I personally did not generate such forecasts, or perform the analysis on said forecasts. It acquired the results from Weather Bench, a benchmark dataset for data-driven weather forecasting. According to said source; computationally, a single forecast takes 270 seconds for the T42 model and 503 seconds for the T64 model on a single XC40 node with 36 cores.

\section{Implementation}\label{implement_rnn}
Drawing from knowledge of current numerical weather prediction model frameworks, it may seem intuitive to train a machine learning model to produce the best possible single‐step forecast from a given atmospheric state. In practice, this can yield a model that performs well for short‐range forecasts but diverges from reality for longer range predictions. This is because there are no constraints on the CNN, physical or mathematical, that would prevent it from diverging from reality when its prediction fed back in as inputs no longer resemble an atmospheric state in the training data. In order to nudge the numerical weather prediction model toward learning to predict longer‐term weather and improve its long‐term stability, the model will be trained on multiple iterated predictive steps.

Initially, the model inputs included two time steps and it is tasked with predicting two output time steps. This resulted in the first couple of time step predictions producing relatively decent result, however, it quickly diverged from reality after this point. Hence, it was necessary to increase both the time steps and the output time steps to six. This increased the overall numerical stability of the model.

The dataset consists of three features: air temperature at a pressure surface of 850 hPa, geopotential at a pressure surface of 500 hPa, and air temperature at 2 metres above the surface. For a single day, there is twelve observations. The goal for this project will be to predict the relevant atmospheric parameter in 12 hours time given the last twelve hours of data. In order to make such predictions, it is necessary to create a window of the last 6 ($\frac{12}{2}$) observations to train the model\cite{time_series}. The neural network was trained on observational data from 2009 to 2015. The remainder of the dataset, 2016 to 2019, was preserved for validation, testing and benchmarking the neural network against physics-based models. The model was built using the open‐source Keras library for Python with Google's TensorFlow backend\cite{numerical_stability}.

\begin{minted}[mathescape,linenos,frame=lines]{python}
# Optimiser.
opt = Adam(lr=1e-3, decay=1e-5)
\end{minted}

Adam optimisation was chosen as the most appropriate optimiser for this particular model. This is a stochastic gradient descent method that is based on adaptive estimation of first-order and second-order moments. This method is computationally efficient, has little memory requirements, invariant to diagonal rescaling of gradients, and is well suited for problems that are large in terms of parameters.

\begin{minted}[mathescape,linenos,frame=lines]{python}
# First layer of model.
model.add(
    ConvLSTM2D(
        filters=64,
        kernel_size=(7, 7),
        input_shape=(6, 179, 360, 3),
        padding='same',
        return_sequences=True,
        activation='tanh',
        recurrent_activation='hard_sigmoid',
        kernel_initializer='glorot_uniform',
        unit_forget_bias=True,
        dropout=0.3,
        recurrent_dropout=0.3,
        go_backwards=True
    )
)
\end{minted}

The above code is the first layer of the model. The activation functions utilised in this layer are predefined within Tensorflow, and have been described at great length previously. The model consists of four ConvLSTM layers in total, with similar parameters as the aforementioned layer with a few variations.

\begin{minted}[mathescape,linenos,frame=lines]{python}
# Batch normalisation.
model.add(BatchNormalization())
\end{minted}

A consistent challenge in machine learning is that the model is updated layer-by-layer backward from the output to the input using an estimate of error that assumes the weights in the layers prior to the current layer are fixed. Because all layers are changed during an update, the update procedure is forever chasing a moving target. A batch normalisation layer occurs after each ConvLSTM layer to resolve this problem.

\begin{definition}
Batch normalization is a technique to help coordinate the update of multiple layers in the model.
\end{definition}

It does this by scaling the output of the layer, specifically by standardising the activation of each input variable per mini-batch, such as the activation of a node from the previous layer. Standardising the activation of the prior layer means that assumptions the subsequent layer makes about the spread and distribution of inputs during the weight update will not change. This has the effect of stabilising and speeding-up the training process of neural networks\cite{batch_normalization}.

\begin{minted}[mathescape,linenos,frame=lines]{python}
# Dropout.
model.add(Dropout(0.1))
\end{minted}

Each batch normalisation layer is then followed by a dropout. Dropout is a regularisation method that approximates training a large number of neural networks with different architectures in parallel. During training, some number of layer outputs are randomly ignored or “dropped out.” This has the effect of making the layer look-like and be treated-like a layer with a different number of nodes and connectivity to the prior layer. In effect, each update to a layer during training is performed with a different “view” of the configured layer. Dropout has the effect of making the training process noisy, forcing nodes within a layer to probabilistically take on more or less responsibility for the inputs. This conceptualisation suggests that perhaps dropout breaks-up situations where network layers co-adapt to correct mistakes from prior layers, in turn making the model more robust\cite{dropout}.

\begin{minted}[mathescape,linenos,frame=lines]{python}
# Add dense layer.
model.add(Dense(3))
\end{minted}

Following the final ConvLSTM layer, a dense layer is implemented. The dense layer is a neural network layer that is connected deeply, which means each neuron in the dense layer receives input from all neurons of its previous layer\cite{dense}. In our case, it results in the model outputting the expected shape by reducing the filters down to the three previously mentioned features. The code for the model in its entirety can be found in appendix \ref{model_code}.


\documentclass{article}

\title{\sc\LARGE CSCA67 Tutorial, Week 8\\
{\Large Nov. 2nd-Nov. 6th, 2015}}
\date{}
\author{\sc Compiled by {\em G. Singh Cadieux}\\[1ex]
\sc Adapted from\\
D. Cunningham. \textit{A logical introduction to proof}. Springer, 2012. \&\\
A. Bretscher, \href{http://www.utsc.utoronto.ca/~bretscher/a67/lectures/predicates.pdf}{\em CSCA67 Week 8 Lecture Notes}}

\usepackage{fullpage}
\usepackage{amsmath,amssymb}
\usepackage{color}
%\usepackage{multirow}
\usepackage{tikz}
\usepackage{hyperref}
%\usepackage{array}

\setlength{\parindent}{0pt}

\begin{document}
\maketitle

\section{\sc More formal logic problems}

\subsection*{Q: {\em Show that $(a\vee b)\wedge\neg(a\wedge b)$ is logically equivalent to $a\leftrightarrow \neg b$ using truth tables.}}
Using the method discussed last week, we construct a truth table for each of the statements.\\
(The truth table shown below merges the two truth tables into one.)
\begin{center}
\begin{tabular}{cc|c|c|c|c||c|c}
$a$&$b$&$a\wedge b$&$\neg(a\wedge b)$&$a\vee b$&$(a\vee b)\wedge\neg(a\wedge b)$&$\neg b$&$a\leftrightarrow \neg b$\\\hline
T&T&T&F&T&F&F&F\\
T&F&F&T&T&T&T&T\\
F&T&F&T&T&T&F&T\\
F&F&F&T&F&F&T&F
\end{tabular}
\end{center}
We have shown that the truth table for $(a\vee b)\wedge\neg(a\wedge b)$ and the truth table for $a\leftrightarrow \neg b$ are identical (the truth value of two statements is the same, given the same combination of truth values of $a$ and $b$). Thus, the two statements are logically equivalent.

\subsection*{Q: {\em Construct a truth table for $(p\vee r)\wedge(r\wedge q)$.}}
\begin{center}
\begin{tabular}{ccc|c|c|c}
$p$&$q$&$r$&$p\vee r$&$r\wedge q$&$(p\vee r)\wedge(r\wedge q)$\\\hline
T&T&T&T&T&T\\
T&T&F&T&F&F\\
T&F&T&T&F&F\\
T&F&F&T&F&F\\
F&T&T&T&T&T\\
F&T&F&F&F&F\\
F&F&T&T&F&F\\
F&F&F&F&F&F
\end{tabular}
\end{center}

\subsection*{Q: {\em Shade regions of a Venn diagram where $(p\vee r)\wedge(r\wedge q)$ is true.}}
Let $p$ be the statement ``$x\in P$", $q$ be the statement ``$x\in Q$", and $r$ be the statement ``$x\in R$", where $P,Q,R$ are sets.\\
Then $(p\vee r)\wedge(r\wedge q)$ represents the region(s) of the Venn diagram where $x\in P$ and $x\in Q$, or where $x\in R$ and $x\in Q$.\\[1ex]
Let us first shade in the region where $x\in P$ or $x\in R$ (red). Then let us shade in the region where $x\in R$ and $x\in Q$ (blue). The intersection between these regions represents our statement, since they are joined by an ``and."
\begin{center}
\begin{tikzpicture}
\filldraw[red!70] (0,0) circle[radius=0.5in];
\draw (0.75in,0) circle[radius=0.5in];
\filldraw[red!70] (0.375in,0.6in) circle[radius=0.5in];
\begin{scope}
\path[clip] (0.375in,0.6in) circle[radius=0.5in];
\draw[fill=blue!70,fill opacity=0.5] (0.75in,0) circle[radius=0.5in];
\end{scope}
\draw (-.65in,-.65in) rectangle (1.4in,1.25in);
\draw (0,0) circle[radius=0.5in];
\draw (0.375in,0.6in) circle[radius=0.5in];
\node at (0,0) {$P$};
\node at (0.75in,0) {$Q$};
\node at (0.375in,0.6in) {$R$};
\node at (-.5in,1.1in) {$\mathcal{U}$};
\end{tikzpicture}\qquad
\begin{tikzpicture}[a/.style={pin={[pin distance=.7in]above right:${\color{red}p,\,q,\,r}$}}]
\draw (0,0) circle[radius=0.5in];
\draw (0.75in,0) circle[radius=0.5in];
\draw (0.375in,0.6in) circle[radius=0.5in];
\draw (-.65in,-.65in) rectangle (1.4in,1.25in);
\node at (-.1in,-.1in) {${\color{red}p},\,q,\,r$};
\node at (0.85in,-.1in) {$p,\,{\color{red}q},\,r$};
\node at (0.375in,0.7in) {$p,\,q,\,{\color{red}r}$};
\node at (-.3in,1in) {$p,\,q,\,r$};
\node[pin={above right:$p,\,{\color{red}q,\,r}$}] at (.7in,.3in) {};
\node[pin={above left:${\color{red}p},\,q,\,{\color{red}r}$}] at (.05in,.3in) {};
\node[pin={below:${\color{red}p,\,q},\,r$}] at (.375in,-.2in) {};
\node[a] at (.3in,.2in) {};
\end{tikzpicture}
\end{center}
\textsc{Notice} that the Venn diagram and the truth table for this statement are equivalent. The regions that are shaded in the diagram correspond to the rows of the truth table where the statement is true, and the regions that are not shaded correspond to the rows where the statement is false.

\subsection*{{\normalsize Let $d$ represent ``David likes baseball."\\
Let $j$ represent ``Jaime likes gymnastics."\\
Let $a$ represent ``Anna likes hockey."}\\
Q: {\em Translate each of the following propositions into English.}}
\subsubsection*{a) $\neg d\wedge j\wedge a$}
We start with the most specific statement: $\neg d$ is the negation of ``David likes baseball", which we express in English as ``David does not like baseball".\\[1ex]
Then, $\neg d\wedge j\wedge a$ is the 3-way conjunction (``and") of ``David does not like baseball", ``Jaime likes gymnastics", and ``Anna likes hockey".\\[1ex]
Thus, in English, our statement is ``David does not like baseball and Jaime likes gymnastics and Anna likes hockey".
We might also express this as ``David does not like baseball \textit{but} Jaime likes gymnastics and Anna likes hockey", as well as a number of other sentence structures.

\subsubsection*{b) $\neg (d\wedge j)$}
Again, we start with the most specific statement: $d\wedge j$ is the conjunction of ``David likes baseball" and ``Jaime likes gymnastics", which we express as ``David likes baseball and Jaime likes gymnastics".\\[1ex]
Then, $\neg (d\wedge j)$ is the negation of ``David likes baseball and Jaime likes gymnastics".\\[1ex]
Thus, in English, our statement is ``It is not the case that David likes baseball and Jaime likes gymnastics".
We might express this more naturally as ``David does not like baseball or Jaime does not like gymnastics" (which is really the English translation of an equivalent but different logic statement).

\subsubsection*{c) $\neg d\vee\neg a$}
As above, we translate $\neg d$ to ``David does not like baseball", and similarly, $\neg a$ becomes ``Anna does not like hockey".\\[1ex]
Then, $\neg d\vee\neg a$ is the disjunction (``or") of ``David does not like baseball" and ``Anna does not like hockey".\\[1ex]
Thus, in English, our statement is ``David does not like baseball or Anna does not like hockey".

\subsubsection*{d) $(d\wedge a)\vee j$}
We start with the most specific statement: $d\wedge a$ is the conjunction of ``David likes baseball" and ``Anna likes hockey", which we express in English as ``David likes baseball and Anna likes hockey".\\[1ex]
Then, $(d\wedge a)\vee j$ is the disjunction of ``David likes baseball and Anna likes hockey", and ``Jaime likes gymnastics".\\[1ex]
Thus, in English, our statement is ``David likes baseball and Anna likes hockey, or Jaime likes gymnastics".\\
The comma separating ``David likes [\ldots]" and ``or Jaime likes gymnastics" is meant to indicate that these are two separate clauses of the sentence, so that it is not interpreted as a conjunction of ``David likes baseball" and ``Anna likes hockey, or Jaime likes gymnastics".

\subsection*{Q: {\em Translate each of the following English sentences into formal logic.}}
\subsubsection*{a) ``Jaime doesn't like gymnastics but David likes baseball."}

We start with the most general statement: from the language ``\ldots but\ldots", we know that the statement is a conjunction (``and").\\[1ex]
Then the first part of the conjunction ``Jaime doesn't like gymnastics" is the negation of ``Jaime likes gynmastics".\\[1ex]
Thus, in formal logic, our statement is $\neg j\wedge d$.

\subsubsection*{b) ``Either David likes baseball or Anna likes hockey, and Jaime likes gymnastics."}

Again, we start with the most general statement: from the language ``\ldots and\ldots", we know that the statement is a conjunction.\\[1ex]
Then the first part of the conjunction ``Either David likes baseball or Anna likes hockey" is the disjunction (``or") of ``David likes baseball" and ``Anna likes hockey".\\
However, we must be careful: in English, the use of the word ``either" implies exclusion. The ``$x$ or $y$" we have previously seen is true if $x$ or $y$ or \textit{both} are true (known as ``inclusive or"); this ``either $x$ or $y$" is true if $x$ or $y$ are true, but \textit{not} if both are true.\\
We can express this more clearly as ``David likes baseball and Anna doesn't like hockey, or David doesn't like baseball and Anna likes hockey".\\[1ex]
Thus, in formal logic, our statement is $((d\wedge\neg a)\vee(\neg d\wedge a))\wedge j$.\\[1ex]
Alternatively, because of the ambiguity of the English language, we may consider this statement to be a disjunction of ``David likes baseball" and ``Anna likes hockey, and Jaime likes gymnastics". The comma separating ``Either David likes baseball or Anna likes hockey" and `` and Jaime likes gymnastics" suggests that this is not the case, but it is possible.\\
Then our statement would be $(d\wedge\neg(a\wedge j))\vee(\neg d\wedge(a\wedge j))$.

\subsubsection*{c) ``If Anna likes hockey then David likes baseball."}

From the language ``if\ldots then\ldots", we know that the statement is an implication, with ``Anna likes hockey" as the antecedent (first half) and ``David likes baseball" as the consequent (second half).\\[1ex]
Thus, in formal logic, our statement is $a\to d$.

\section{\sc Predicate Logic}
So far, we have discussed something that we have referred to as ``formal logic." However, we have only studied one branch of formal logic, which we will now properly call \textsc{sentential logic} (also known as ``propositional logic").\\[1ex]
Here, we introduce another branch of formal logic: \textsc{predicate logic}.\\[1em]
Predicate logic addresses some of the limitations of sentential logic:
\begin{itemize}
\item Sentential logic cannot express quantitities/categories, eg. ``\textit{Every} person has a mother," ``\textit{Some} horses are white"
\item Sentential logic cannot express relationships between properties, eg. ``If X is married to Y, then Y is married to X"
\end{itemize}

Predicate logic uses \textsc{predicates} instead of statements/propositions. Predicates are sentences containing variables which can be assigned values. For example, ``$x>5$" is a predicate, where $x$ is the variable.\\[1ex]
Predicates are \textit{not} statements themselves because they have no inherent truth value.\\
\textsc{Consider}: can we say definitively that ``$x>5$" is true or false?\\[1ex]
However, once the variable(s) in the predicate have been assigned values, then the sentence becomes a statement. For example, if we let $x=3$, then ``$x>5$" is a false statement, and if we let $x=6$, then ``$x>5$" is a true statement.

\subsection{\em Syntax}

The language of sentential logic is composed of: sentence symbols, operators/connectives, and parentheses.\\
Since predicate logic uses predicates rather than statements, it uses predicate symbols rather than sentence symbols.\\ Predicate symbols typically follow mathematical function notation, eg. $\overbrace{P(\underbrace{x}_{\text{variable}})}^{\text{predicate}}$.\\
The name of the predicate is typically a single uppercase letter, and the variable is denoted as a single lowercase letter.\\[1ex]
\textsc{Note} that, as with a mathematical function, the letter chosen to represent the variable is simply a placeholder. $P(x)$ is identical to $P(y),\,P(z)$, etc.\\[1em]
We can then use the same connectives as in sentential logic to combine predicates and create compound predicates, eg.
\begin{itemize}
\item $P(x):\,2x=6$
	\begin{itemize}
	\item $P(3)$ is true
	\item $P(10)$ is false
	\end{itemize}
\item $Q(x,y):\,``x\text{ is larger than }y"$
	\begin{itemize}
	\item $Q(\text{Earth},\text{moon})$ is true
	\item $Q(\text{apple},\text{car})$ is false
	\end{itemize}
\item $R(x):\,x<10$
\item $P(x)\wedge R(x)\equiv ``x\text{ is between 5 and 10}"$
	\begin{itemize}
	\item $P(6)\wedge R(6)$ is true
	\item $P(14)\wedge R(14)$ is false
	\end{itemize}
\end{itemize}
\textsc{Note} that, although $x$ is a placeholder, $x$ represents the same value everywhere it is used within a predicate. For example, if we create the compound predicate $P(x)\wedge R(x)$, we cannot assign values such that we have $P(6)\wedge R(14)$.\\[1ex]
Conversely, different variables may represent different values. For example, if we create the compound predicate $P(x)\wedge R(y)$, we \textit{can} assign $x=6$ and $y=14$ such that we have $P(6)\wedge R(14)$.
\begin{itemize}
\item $S(x):\,x<20$
\item $\neg S(x)\equiv ``x$ is greater than or equal to 20"
\item $R(x)\to S(x)\equiv$ ``if $x$ is less than 10, then it is also less than 20"
	\begin{itemize}
	\item $R(3)\to S(3)$ is true
	\item $R(25)\to S(25)$ is (vacuously) true
	\end{itemize}
\end{itemize}
Predicate logic also adds operators called \textsc{quantifiers}, which allow us to describe ranges of values for variables, rather than assigning them individual values.\\[1em]
The \textsc{existential} quantifier is denoted $\exists$ and means ``(there) exists."\\[1ex]
For example, if $P(x):\,``x^2=5"$, then $\exists x(P(x))$ means ``There exists (at least) one value of $x$ for which $x^2=5$". The statement $\exists x(P(x))$ is true, since $P(\sqrt{5})$ and $P(-\sqrt{5})$ are both true.\\[1ex]
If $Q(x):\,``x^2=-5"$, then $\exists x(Q(x))$ means ``There exists (at least) one value of $x$ for which $x^2=-5$", which is false (assuming that $x$ is restricted to the set of reals).\\[1em]
The \textsc{universal} quantifier is denoted $\forall$ and means ``for all/every."\\[1ex]
For example, if $P(x):\,``x^2=5"$, then $\forall x(P(x))$ means ``For every possible value of $x$, $x^2=5$". The statement $\forall x(P(x))$ is false, since only $P(\sqrt{5})$ and $P(-\sqrt{5})$ are true, and $P(x)$ is false for any other value of $x$.\\[1ex]
If $R(x):\,x>5$ and $S(x):\,x>7$, then $\forall x(S(x)\to R(x))$ means ``For every possible value of $x$, if $x$ is greater than 7, $x$ is also greater than 5". We know that this statement is true, since $5<7$ and $7<x$.\\[1em]
As with sentential logic, we may use parentheses to group predicates and operators. When parentheses are omitted, the operators are applied according to the following precedence rules:
\begin{center}
\begin{tikzpicture}
\draw[<->] (0,0) node[left,align=center,font=\small] {highest\\precedence}
-- (.5,0) node[above] {$\neg$}
-- (1.5,0) node[above] {$\wedge$}
-- (2.5,0) node[above] {$\vee$}
-- (3.5,0) node[align=center] {$\exists$\\$\forall$}
-- (4.5,0) node[align=center] {$\to$\\$\leftrightarrow$}
-- (5,0) node[align=center,right,font=\small] {lowest\\precedence};
\end{tikzpicture}
\end{center}

\subsection{\em Ordering of quantifiers}

When we have a statement containing multiple, different quantifiers, such as $\forall x\exists y(P(x,y))$, the quantified variables are read left to right. For example, if $P(x,y):\,x\cdot y=5$, then $\forall x\exists y(P(x,y))$ means ``For every number $x$, there exists a number $y$ such that $x\cdot y=5$".\\[1ex]
\textsc{Does the} meaning of the sentence change if we reorder the quantifiers?\\
It depends upon the predicate.\\[1ex]
For example, if $P(x,y):\,x>y$, then $\forall x\exists y(P(x,y))$ means ``For every number $x$, there exists a number $y$ such that $x$ is greater than $y$".\\
We know that this statement is true because the set of integers (or reals, depending upon the allowed values of $x$) is infinite. Thus, by definition, there is always a number larger than any number we choose from the set.\\[1ex]
But $\exists y\forall x(P(x,y))$ means ``There exists a number $y$ such that, for every number $x$, $x$ is greater than $y$".\\
We know that this statement is false because it is saying that there is some minimum integer (since every other integer is larger than it). Since the integers are infinite, this is impossible.\\[1ex]
Here, the order of the quantifiers is significant.

\subsection{\em Negation of quantifiers}

In addition to negating predicates using the negation operator, we want to be able to negate quantifiers - that is, we want to be able to say ``There does \textit{not} exist\ldots" and ``\textit{It is not the case that} for all\ldots".\\[1ex]
For some predicate $P(x)$, ``There does \textit{not} exist an $x$ such that $P(x)$ is true" is equivalent to saying ``$P(x)$ is false for every value of $x$", which we can write formally as $\forall x(\neg P(x))$.\\[1ex]
For some predicate $P(x)$, ``\textit{It is not the case that}, for every value of $x$, $P(x)$ is true" is equivalent to saying ``There is some value of $x$ for which $P(x)$ is false", which we can write formally as $\exists x(\neg P(x))$.\\[1ex]
Thus, the negation of $\exists x(P(x))$ is $\forall x(\neg P(x))$ and the negation of $\forall x(P(x))$ is $\exists x(\neg P(x))$.\\[1em]
For example, if $P(x)$: ``$x$ is tall", where $x$ is a person, then
\begin{itemize}
\item $\exists x(P(x))$ means ``There exists a person who is tall"
\item $\forall x(\neg(P(x))$ means ``Every person is not tall", or ``There does not exist a person who is tall"
\item $\forall x(P(x))$ means ``All people are tall"
\item $\exists x(\neg P(x))$ means ``There exists a person who is not tall", or ``Not every person is tall"
\end{itemize}

\section{\sc Additional practice problems}
Let $P(x,y)$ be the predicate ``$x\cdot y=12$", where $x,y$ are integers.\\
{\bf Q: Which of the following statements is true?}
\begin{itemize}
\item $P(3,4)$
\item $P(3,5)$
\item $P(2,6)\vee P(3,7)$
\item $\forall x,\forall y(P(x,y)\to P(y,x))$
\item $\forall x,\exists y(P(x,y))$
\end{itemize}
\vspace{1ex}Given the predicates\\
$L(x)=$``$x$ is a lion."\\
$F(x)=$``$x$ is fuzzy."\\
where $x$ is a mammal,\\
{\bf Q: Translate ``All lions are fuzzy" into predicate logic.}\\[1ex]
{\bf Q: Translate ``Some lions are fuzzy" into predicate logic.}\\[1em]
Consider the three predicates\\
$P(x)$ symbolizes the statement ``$x$ is a prime number"\\
$E(x)$ symbolizes the statement ``$x$ is even"\\
$D(x, y)$ symbolizes the statement ``$x$ evenly divides $y$"\\
where $x$ and $y$ represent integers.\\
{\bf Q: Find some values for the variables that make the following logical formulas true, and others making them false.}
\begin{itemize}
\item $P(x) \wedge E(x)$
\item $E(x) \vee D(x, y)$
\item $\neg P(x) \wedge D(x, y)$
\item $D(x, y) \to\neg P(x)$
\end{itemize}
\vspace{1ex}\textit{Tarski's World} is a computer program that is meant to be an introduction to predicate logic.\\
You can build two-dimensional worlds of shapes, describe them using predicates, and test whether your predicates are true or false.\\
Experiment with Tarski's World using the following implementation: \href{http://courses.cs.washington.edu/courses/cse590d/03sp/tarski/tarski.html}{Tarski's World (Java Applet)}

\end{document}


\vsssub
\subsubsection{~$S_{in} + S_{ds}$: \wam\ cycle 4 (ECWAM)} \label{sec:ST3}
\vsssub

\opthead{ST3}{\wam\ model}{F. Ardhuin}

\noindent
The wind-wave interaction source terms described here are based on the wave
growth theory of \cite{art:Miles57}, modified by \cite{art:Jan82}. The
pressure-slope correlations that give rise to part of the wave generation are
parameterized following \cite{art:Jan91}.
%A wave dissipation term due to shear
%stresses variations in phase with the orbital velocity is added for the swell
%part of the spectrum, based on the swell decay observations of
%\cite{art:ACC09}.

This parameterization was further extended by \cite{rep:AB02} to take into
account a stronger gustiness in unstable atmospheric conditions. This effect is
included in the present parameterization and is activated with the optional
{\code STAB3} switch. The formula used in {\code STAB3} is not described herein.
For that, the reader is referred to \cite{rep:AB02}. If {\code STAB3} is used, the air-sea
temperature differences should be provided by the user, e.g. using {\file ww3\_prep}.

Efforts have been made to make the present implementation as
close as possible to the one in the ECWAM model \citep{rep:Bea05}, in
particular the stress lookup tables were verified to be identical.
Later modifications include the addition of a negative part in the wind input
to represent swell dissipation.

The source term reads \citep{bk:Jan04}
\begin{equation}
\cS_{in}(k,\theta) =
\frac{\rho_a}{\rho_w}\frac{\beta_{\mathrm{max}}}{\kappa^2}{\mathrm e}^{Z}Z^4
\left(\frac{u_\star}{C}+z_\alpha\right)^2 \cos^{p_{in}}(\theta - \theta_u) \sigma N
\left(k,\theta\right) + S_{out}(k,\theta),\label{eq:SinWAM4}
\end{equation}

\noindent where $\rho_a$ and $\rho_w$ are the air and water densities,
$\beta_{\mathrm{max}}$ is a non-dimensional growth parameter (constant),
$\kappa$ is von K\'{a}rm\'{a}n' constant, and $p_{in}$ is a constant that
controls the directional distribution of $\cS_{in}$. In the present
implementation the air/water density ratio ${\rho_a}/{\rho_w}$ is constant. We
define $Z=\log(\mu)$ where $\mu$ is given by \cite{art:Jan91}  Eq.~(16), and
corrected for intermediate water depths, so that

\begin{equation}
Z=\log(k z_1)+\kappa/\left[\cos\left(\theta - \theta_u\right)
\left(u_\star/C + z_\alpha \right)\right],
\end{equation}

\noindent
where $z_1$ is a roughness length modified by the wave-supported stress
$\tau_w$, and $z_\alpha$ is a wave age tuning parameter\footnote{Although this
tuning parameter $z_\alpha$ is not well described in WAM-Cycle4 documentation,
it has an important effect on wave growth. Essentially it shifts the wave age
of the long waves, which typically increases the growth, and even generates
waves that travel faster than the wind. This accounts for some gustiness in
the wind and should possibly be resolution-dependent. For reference, this
parameter was not properly set in early versions of the SWAN model, as
discovered by R. Lalbeharry.}.  The roughness $z_1$ is defined as,

\begin{eqnarray}
U_{10}&=&\frac{u_\star}{\kappa} \log\left(\frac{z_u}{z_1}\right) \\
z_1&=&\alpha_0 \frac{\tau}{ \sqrt{1-\tau_w/\tau}},
\end{eqnarray}

\noindent
where $\tau=u_\star^2$, and $z_u$ is the height at which the wind is
specified. These two equations provide an implicit functional dependence of
$u_\star$ on $U_{10}$ and $\tau_w/\tau$. This relationship is then tabulated
\citep{art:Jan91, rep:Bea07}.

An important part of the parameterization is the calculation of the
wave-supported stress $\tau_w$,

\begin{equation}
\tau_w=\left|\int_0^{k_{\max}} \int_0^{2 \pi} \frac{\cS_{in}(k',\theta)}{C}
\left(\cos \theta, \sin \theta \right)  {\mathrm d} k' \mathrm d \theta +
\tau_{\mathrm{hf}}(u_\star,\alpha) \left(\cos \theta_u, \sin \theta_u \right)
\right|,\label{eq:tauwint}
\end{equation}

\noindent
which includes the resolved part of the spectrum, up to $k_{\max}$, as well as
the stress supported by shorter waves, $\tau_{\mathrm{hf}}$. Assuming a
$f^{-X}$ diagnostic tail beyond the highest frequency, $\tau_{\mathrm{hf}}$ is
given by

\begin{eqnarray}
\tau_{\mathrm{hf}}(u_\star,\alpha)&= &\frac{u_{\star}^2}{g^2}
\frac{\sigma_{\max}^X 2 \pi \sigma }{2 \pi C_g(k_{\max})} \int_0^{2 \pi} N
\left(k_{\max},\theta \right)
\max\left\{0,\cos\left(\theta-\theta_u\right)\right\}^3 d \theta \nonumber \\
& & \times \frac{\beta_{\mathrm{max}}}{\kappa^2}
\int_{\sigma_{\max}}^{0.05*g/u_\star} \frac{{\mathrm
e}^{Z_{\mathrm{hf}}}Z_{\mathrm{hf}}^4}{\sigma^{X-4}} {\mathrm d} \sigma
\label{eq:tauhfint}
\end{eqnarray}

\noindent
where the second integral is a function of $u_\star$ and the Charnock
coefficient $\alpha$ only, which is easily tabulated. In practice the
calculation is coded with $X=5$, and the variable $Z_{\mathrm{hf}}$ is defined
by,

\begin{equation}
Z_{\mathrm{hf}}(\sigma)=\log(k z_1)+\min\left\{\kappa/\left(u_\star/C +
z_\alpha \right),20\right\}.
\end{equation}

\noindent
This parameterization is sensitive to the spectral level at $k_{\max}$.
A higher spectral level will lead to a larger value of $u_\star$ and thus
positive feedback on the wind input via $z_1$. This sensitivity is exacerbated
by the sensitivity of the high-frequency spectral level to the presence of
swell via the dissipation term.

\begin{table}[htb]
\begin{center}
\begin{tabular}{|l|c|c|c|c|c|} \hline \hline
Par.         &  WWATCH var.           & namelist & WAM4 & BJA   & Bidlot 2012 \\
\hline
  $z_u$ &  ZWND                       & SIN3 & 10.0    & 10.0   & 10.0   \\
  $\alpha_0$ &  ALPHA0                & SIN3 & 0.01    & 0.0095 &  0.0095 \\
  $\beta_{\mathrm{max}}$ & BETAMAX    & SIN3 & 1.2     & 1.2    & 1.2  \\
  $p_{\mathrm{in}}$ &  SINTHP         & SIN3 & 2       & 2      & 2  \\
  $z_\alpha$ &  ZALP                  & SIN3 & 0.0110  & 0.0110 &  0.0080 \\
  $s_1$ &  SWELLF                     & SIN3 & 0.0     & 0.0    & 1.0   \\
\hline
\end{tabular} \end{center}
\caption{Parameter values for WAM4, BJA and the 2012 update in the ECWAM model. Source term
  parameterizations that can be reset via the {\F SIN3} and {\F SDS3} namelist. BJA is
  generally better than WAM4. The default parameters in ST3 corresponds to BJA. Please
  note that the names of the variables only apply to the namelists. In the source
  term module the names are slightly different, with a doubled first letter, in
  order to differentiate the variables from the pointers to these variables.} \label{tab:WAM4_parSIN}
\botline
\end{table}

\begin{table}[htb]
\begin{center}
\begin{tabular}{|l|c|c|c|c|c|} \hline \hline
Par.                               &  WWATCH var.         & namelist & WAM4 & BJA   & Bidlot 2012 \\
\hline
  $C_{\mathrm{ds}}$                 &  SDSC1          & SDS3 & -4.5 & -2.1& -1.33       \\
  $p$                               &  WNMEANP        & SDS3 & -0.5 & 0.5 &  0.5        \\
  $p_{\mathrm{tail}}$               &  WNMEANPTAIL    & SDS3 & -0.5 & 0.5 &  0.5        \\
  $\delta_1$                        &  SDSDELTA1      & SDS3 & 0.5  & 0.4 &  0.5        \\
  $\delta_2$                        &  SDSDELTA2      & SDS3 & 0.5  & 0.6 &  0.5 \\
  \hline \hline
\end{tabular} \end{center}
\caption{Parameter values for WAM4, BJA and the update by \cite{pro:Bid12}. Source term
parameterizations that can be reset via the {\F SDS3} namelist. BJA is generally
better than WAM4. Please note that the
names of the variables only apply to the namelists. In the source term module
the names are slightly different, with a doubled first letter, in order to
differentiate the variables from the pointers to these variables.} \label{tab:WAM4_parSDS}
\botline
\end{table}

A linear damping of swells was introduced in the operational ECWAM model in September 2009. It takes
the form given by \cite{bk:Jan04}

\begin{equation}
S_{out}(k,\theta)= 2 s_1  \kappa \frac{\rho_a }{\rho_w} \left(\frac{u_\star}{C}\right)^2
\left[\cos \left(\theta - \theta_u\right) - \frac{\kappa C}{u_\star \log(k z_0)}\right]
\end{equation}

\noindent where $s_1$ is set to 1 when this damping is used and 0 otherwise. For $s_1=0$
the parameterization is the WAM4 or BJA parameterization (see Table \ref{tab:WAM4_parSIN}).

Due to the increase in high-frequency input compared to WAM3, the dissipation
function was adapted by Janssen (1994) from the WAM3 dissipation, and later
reshaped by \cite{rep:Bea05}. That later modification is referred to as "BJA" for Bidlot,
Janssen and Abdallah. A more recent modification, strongly improved the model results for
Pacific swells, at the price of an underestimation of the highest sea states. This
corresponds to the ECMWF WAM model contained in the IFS version CY38R1 \citep{pro:Bid12}. Note that these parameters were optimized for use of neutral winds from the
operational ECMWF analysis. Using these with other wind products may require a re-tuning of these coefficients. For example, with NCEP or CFSRR winds, the value of BETAMAX
should probably be reduced or ZWND increased.

The generic form of the WAM4 dissipation term is,

\begin{equation}
S_{ds}\left(k,\theta\right)^{\mathrm{WAM}} = C_{ds} \overline{\alpha}^2
 \overline{\sigma} \left[\delta_1 \frac{k}{\overline{k}} + \delta_2
\left(\frac{k}{\overline{k}}\right)^2\right]\label{eq:SdsWAM4}
N\left(k,\theta\right)
\end{equation}

\noindent
where $C_{ds}$ is a non-dimensional constant $\delta_1$ and $\delta_2$ are
weight parameters,

\begin{equation}
\overline{k}=\left[\frac{\int k^p N\left(k,\theta\right) {\mathrm d}
\theta}{\int N\left(k,\theta\right) {\mathrm d} \theta}\right]^{1/p}
\end{equation}

\noindent
with $p$ a constant power. Similarly, the mean frequency is defined as

\begin{equation}
\overline{\sigma}=\left[\frac{\int \sigma^p N\left(k,\theta\right) {\mathrm d}
\theta}{ \int N\left(k,\theta\right) {\mathrm d} \theta}\right]^{1/p},
\end{equation}

\noindent
so that the mean steepness is $\overline{\alpha}=E \overline{k}^2$.

The mean frequency also occurs in the definition of the maximum frequency of
prognostic integration of the source terms. Since the definition of that
frequency may be different from that of the source term it is defined with
another exponent $p_{\mathrm{tail}}$.

Unfortunately these parameterizations are sensitive to swell. An increase in
swell height typically reduces dissipation at the windsea peak because the mean wavenumber $\overline{k}$ and
thus the mean steepness $\overline{\alpha}$ are reduced. For $p< 2$, as in the WAM-Cycle 4 and BJA
parameterizations, this sensitivity is much larger and opposite to the expected effect of
short wave modulation by long waves.

The source term code was generalized to
allow the use of WAM4, BJA or others ECWAM parameterization, via a simple change of
the parameters in the namelists {\F SIN3} and {\F SDS3}, see Tables \ref{tab:WAM4_parSIN} and \ref{tab:WAM4_parSDS}. At present, the default values of the namelist
parameters correspond to BJA \citep{rep:Bea05}.


\documentclass{report}
\usepackage{amsfonts, amsmath, amssymb, hyperref}
\renewcommand{\chaptername}{}
\setlength{\parindent}{0pt}

\title{Solutions to exercises in SICP (2e)}
\author{Paul Tan (\href{http://pauljxtan.com}{website}, \href{https://github.com/pauljxtan}{github})}

\begin{document}

\maketitle
\tableofcontents

\chapter{Exercise 1.10}

This is a pretty fun exercise in rendering Scheme procedures in mathematical notation. Personally, I find it helps with visualizing the recursive process. \\

Ackermann's function is defined in the question as

\begin{equation}
  A(x, y) = \begin{cases}
    0 & \text{if } y = 0 \\
    2y & \text{if } x = 0 \\
    2 & \text{if } y = 1 \\
    A(x - 1, A(x, y - 1)) & \text{otherwise}
  \end{cases}
\end{equation}

For each of the given expressions, we have

\begin{align*}
  A(1, 10) &= A(0, A(1, 9)) \\
           &= A(0, A(0, A(1, 8))) \\
           &= A(0, A(0, A(0, A(1, 7)))) \\
           &= A(0, A(0, A(0, A(0, A(1, 6))))) \\
           &= A(0, A(0, A(0, A(0, A(0, A(1, 5)))))) \\
           &= A(0, A(0, A(0, A(0, A(0, A(0, A(1, 4))))))) \\
           &= A(0, A(0, A(0, A(0, A(0, A(0, A(0, A(1, 3)))))))) \\
           &= A(0, A(0, A(0, A(0, A(0, A(0, A(0, A(0, A(1, 2))))))))) \\
           &= A(0, A(0, A(0, A(0, A(0, A(0, A(0, A(0, A(0, A(1, 1)))))))))) \\
           &= A(0, A(0, A(0, A(0, A(0, A(0, A(0, A(0, A(0, 2))))))))) \\
           &= A(0, A(0, A(0, A(0, A(0, A(0, A(0, A(0, 2\cdot2)))))))) \\
           &= A(0, A(0, A(0, A(0, A(0, A(0, A(0, 2\cdot2\cdot2))))))) \\
           &= A(0, A(0, A(0, A(0, A(0, A(0, 2\cdot2\cdot2\cdot2)))))) \\
           &= A(0, A(0, A(0, A(0, A(0, 2\cdot2\cdot2\cdot2\cdot2))))) \\
           &= A(0, A(0, A(0, A(0, 2\cdot2\cdot2\cdot2\cdot2\cdot2)))) \\
           &= A(0, A(0, A(0, 2\cdot2\cdot2\cdot2\cdot2\cdot2\cdot2))) \\
           &= A(0, A(0, 2\cdot2\cdot2\cdot2\cdot2\cdot2\cdot2\cdot2)) \\
           &= A(0, 2\cdot2\cdot2\cdot2\cdot2\cdot2\cdot2\cdot2\cdot2) \\
           &= 2\cdot2\cdot2\cdot2\cdot2\cdot2\cdot2\cdot2\cdot2\cdot2 \\
           &= 2^{10}
\end{align*}

In general, it appears that for $n > 0$ we have

\begin{equation}
  A(1, n) = 2^n
\end{equation}

(Recall that $A(1, 0) = 0$ by definition.) \\

Next expression:

\begin{align*}
  A(2, 4) &= A(1, A(2, 3)) \\
          &= A(1, A(1, A(2, 2))) \\
          &= A(1, A(1, A(1, A(2, 1)))) \\
          &= A(1, A(1, A(1, 2))) \\
          &= A(1, A(1, 2^2)) \quad &\text{using the result found above} \\
          &= A(1, 2^{2^2}) \quad &\text{ditto} \\
          &= 2^{2^{2^2}} \quad &\text{and again} \\
          &= 2^{2^4} \\
          &= 2^{16} \\
          &= 65536
\end{align*}

And one more:

\begin{align*}
  A(3, 3) &= A(2, A(3, 2)) \\
          &= A(2, A(2, A(3, 1))) \\
          &= A(2, A(2, 2)) \\
          &= A(2, A(1, A(2, 1)) \\
          &= A(2, A(1, 2)) \\
          &= A(2, A(0, A(1, 1))) \\
          &= A(2, A(0, 2)) \\
          &= A(2, 4) \\
          &= A(1, A(2, 3)) \\
          &= A(1, A(1, A(2, 2))) \\
          &= A(1, A(1, A(1, A(2, 1)))) \\
          &= A(1, A(1, A(1, 2))) \\
          &= A(1, A(1, A(0, A(1, 1)))) \\
          &= A(1, A(1, A(0, 2))) \\
          &= A(1, A(1, 4)) & \text{familiar pattern emerging\dots} \\
          &= A(1, A(0, A(1, 3))) \\
          &= A(1, A(0, A(0, A(1, 2)))) \\
          &= A(1, A(0, A(0, A(0, A(1, 1))))) \\
          &= A(1, A(0, A(0, A(0, 2)))) \\
          &= A(1, A(0, A(0, 4))) \\
          &= A(1, A(0, 8)) \\
          &= A(1, 16) \\
          &= 2^{16} \\
          &= 65536 \\
\end{align*}

(Gotta say, the way the equations ``fan'' in and out like that is pretty neat. First it fans out 3 times, then 2, then just once on the final expansion.) \\

Just to check, I computed these values in \textbf{sicp\_1-10\_check.scm} and they are indeed correct. Phew! \\

The last part is essentially the same idea, but with a variable parameter $n$. Let's see\dots

\begin{align*}
  f(n) &= A(0, n) \\
       &= 2n
\end{align*}

\begin{align*}
  g(n) &= A(1, n) \\
       &= A(0, A(1, n - 1)) \\
       &= 2 \cdot A(1, n - 1) \\
       &= 2 \cdot A(0, A(1, n - 2)) \\
       &= 2 \cdot 2 \cdot A(1, n - 2) \\
       &= \dots & \text{follow the pattern\dots} \\
       &= 2^{n - 1} \cdot A(1, 1) \\
       &= 2^{n - 1} \cdot 2 \\
       &= 2^n & \text{as found earlier!}
\end{align*}

\begin{align*}
  h(n) &= A(2, n) \\
       &= A(1, A(2, n - 1)) \\
       &= A(1, A(1, A(2, n - 2))) \\
       &= A(1, A(1, A(1, A(2, n - 3)))) \\
\end{align*}

At this point, note that the levels of nesting is equal to the number $m$ in $n - m$ at the end\dots

\begin{align*}
  \Rightarrow h(n)
       &= A(1, A(1, A(1, \dots A(1, A(2, 1)) \dots ))) \\
       &= A(1, A(1, A(1, \dots A(1, 2) \dots ))) & \text{We know what $A(1, n)$ is from earlier!} \\
       &= A(1, A(1, A(1, \dots 2^2 \dots ))) & \text{Now we "fold" it back in $n - 1$ times\dots} \\
       &= 2^{2^{2^{\dots}}} & \text{with $n$ "2"s} \\
\end{align*}

That is,

\begin{align*}
  A(2, 1) &= 2 \\
  A(2, 2) &= 2^2 = 4 \\
  A(2, 3) &= 2^{2^2} = 16 \\
  A(2, 4) &= 2^{2^{2^2}} = 65536 \quad \text{(as we found earlier)}
\end{align*}

and so on. To be honest, I have no idea how to formulate this as a ``concise mathematical definition''. Any suggestions welcome\dots

\end{document}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{graphicx}
\usepackage{bussproofs}
\usepackage{amssymb}
\usepackage{hyperref}
\usepackage[a4paper, total={7in, 10in}]{geometry}

\newenvironment{scprooftree}[1]%
  {\gdef\scalefactor{#1}\begin{center}\proofSkipAmount \leavevmode}%
  {\scalebox{\scalefactor}{\DisplayProof}\proofSkipAmount \end{center} }

\begin{document}

{\Huge Natural Deduction on Predicate Logic} \\

All this work is based on the book \textit{Logic and Structure} by Dirk Van Dalen.

\section*{Strategy to build correct derivations in Predicate Logic}

\noindent\fbox{%
    \parbox{\textwidth}{%
\textbf{From an university test:} \\ \\

Let $\Gamma = \{(\exists x)P_1(x), (\exists x)(\exists y)(\neg P_2 (x, y)), (\forall x)(P_1 (x) \rightarrow (\exists y)P_2 (x,y)) \}$ \\

Build a derivation that proves the following: \\
$$\Gamma \cup \{ (\forall x)(\forall y)(P_3 (x,y) \leftrightarrow P_2 (x,y)) \} \vdash (\exists x)(\exists y)(\neg P_3 (x, y))$$
    }%
}
\newline
\newline\newline
\noindent\fbox{%
    \parbox{\textwidth}{%
First of all build the derivation without canceling any hypotheses:\\
    }%
}
\begin{scprooftree}{0.8}
  \AxiomC{$(\exists x)(\exists y)(\neg P_2 (x, y))$}
  \AxiomC{$(\exists y)(\neg P_2 (x, y))$}
  \AxiomC{$\neg P_2 (x, y)$}
  \AxiomC{$(\forall x)(\forall y)(P_3 (x,y) \leftrightarrow P_2 (x,y))$}
  \RightLabel{$E_{\forall *6}$}
  \UnaryInfC{$(\forall y)(P_3 (x,y) \leftrightarrow P_2 (x,y))$}
  \RightLabel{$E_{\forall *5}$}
  \UnaryInfC{$P_3 (x,y) \leftrightarrow P_2 (x,y)$}
  \AxiomC{$P_3 (x, y)$}
  \RightLabel{$E_{\leftrightarrow 1}$}
  \BinaryInfC{$P_2 (x, y)$}
  \RightLabel{$E_{\neg}$}
  \BinaryInfC{$\bot$}
  \LeftLabel{(3)}
  \RightLabel{$I_{\neg}$}
  \UnaryInfC{$\neg P_3 (x, y)$}
  \RightLabel{$I_{\exists *4}$}
  \UnaryInfC{$(\exists y)(\neg P_3 (x, y))$}
  \RightLabel{$I_{\exists *3}$}
  \UnaryInfC{$(\exists x)(\exists y)(\neg P_3 (x, y))$}
  \LeftLabel{(2)}
  \RightLabel{$E_{\exists *2}$}
  \BinaryInfC{$(\exists x)(\exists y)(\neg P_3 (x, y))$}
  \LeftLabel{(1)}
  \RightLabel{$E_{\exists *1}$}
  \BinaryInfC{$(\exists x)(\exists y)(\neg P_3 (x, y))$}
\end{scprooftree}

During the construction the following hypotheses were generated:
\begin{enumerate}
\item $(\exists y)(\neg P_2 (x, y))$
\item $\neg P_2 (x, y)$
\item $P_3 (x, y)$
\end{enumerate}

\noindent\fbox{%
    \parbox{\textwidth}{%
Next, start checking every atomic step top-down. In each atomic step, if the rule needs proofs we write them down. If the rule generated hypotheses we cancel them. If you can give a correct proof for all rules that need it, the derivation is correct. \\
    }%
}
\begin{itemize}
  \item [$E_{\forall *6}$]:
  \item This rule doesn't generate hypotheses.
  \item \textbf{*6 justification:} $x$ is free for $x$ in any formula (in particular, in: $(\forall y)(P_3 (x,y) \leftrightarrow P_2 (x,y))$).
\end{itemize}

\begin{itemize}
  \item [$E_{\forall *5}$:]
  \item This rule doesn't generate hypotheses.
  \item \textbf{*5 justification:} $y$ is free for $y$ in any formula (in particular, in $(P_3 (x,y) \leftrightarrow P_2 (x,y))$).
\end{itemize}

\begin{itemize}
  \item [$E_{\leftrightarrow 1}$:]
  \item This rule doesn't generate hypotheses and doesn't need justification.
\end{itemize}

\begin{itemize}
  \item [$E_{\neg 1}$:]
  \item This rule doesn't generate hypotheses doesn't need justification.
\end{itemize}
\newpage

\begin{itemize}
  \item [$I_{\neg 1}$:]
  \item This rule generates the hypotheses ($P_3 (x, y)$), se let's cancel it.
  \item This rule doesn't need justification.\\
\end{itemize}

The derivation at this step is: \\

\begin{scprooftree}{0.8}
  \AxiomC{$(\exists x)(\exists y)(\neg P_2 (x, y))$}
  \AxiomC{$(\exists y)(\neg P_2 (x, y))$}
  \AxiomC{$\neg P_2 (x, y)$}
  \AxiomC{$(\forall x)(\forall y)(P_3 (x,y) \leftrightarrow P_2 (x,y))$}
  \RightLabel{$E_{\forall *6}$}
  \UnaryInfC{$(\forall y)(P_3 (x,y) \leftrightarrow P_2 (x,y))$}
  \RightLabel{$E_{\forall *5}$}
  \UnaryInfC{$P_3 (x,y) \leftrightarrow P_2 (x,y)$}
  \AxiomC{$[P_3 (x, y)]^3$}
  \RightLabel{$E_{\leftrightarrow 1}$}
  \BinaryInfC{$P_2 (x, y)$}
  \RightLabel{$E_{\neg}$}
  \BinaryInfC{$\bot$}
  \LeftLabel{(3)}
  \RightLabel{$I_{\neg}$}
  \UnaryInfC{$\neg P_3 (x, y)$}
  \RightLabel{$I_{\exists *4}$}
  \UnaryInfC{$(\exists y)(\neg P_3 (x, y))$}
  \RightLabel{$I_{\exists *3}$}
  \UnaryInfC{$(\exists x)(\exists y)(\neg P_3 (x, y))$}
  \LeftLabel{(2)}
  \RightLabel{$E_{\exists *2}$}
  \BinaryInfC{$(\exists x)(\exists y)(\neg P_3 (x, y))$}
  \LeftLabel{(1)}
  \RightLabel{$E_{\exists *1}$}
  \BinaryInfC{$(\exists x)(\exists y)(\neg P_3 (x, y))$}
\end{scprooftree}

\begin{itemize}
  \item [$I_{\exists *4}$:]
  \item This rule doesn't generate hypotheses.
  \item \textbf{*4 justification:} $y$ is free for $y $ in any formula (in particular, in $\neg P_3 (x,y)$).
\end{itemize}

\begin{itemize}
  \item [$I_{\exists *3}$:]
  \item This rule doesn't generate hypotheses.
  \item \textbf{*3 justification:} $x$ is free for $x$ in any formula (in particular, in $(\exists y)(\neg P_3 (x,y))$).
\end{itemize}

\begin{itemize}
  \item [$E_{\exists *2}$:]
  \item This rule generates the hypotheses $\neg P_2 (x, y)$, let's cancel it.
  \item \textbf{*2 justification:} $y \not\in$ $FV((\exists x)(\exists y)(\neg P_3 (x, y)))$, $y \not\in$ $FV((\forall x)(\forall y)(P_3 (x,y) \leftrightarrow P_2 (x,y)))$. \\
  \textbf{Note that there's no need to check if other hypotheses contain $y$ free: ($P_3 (x,y)$ ni en $\neg P_2 (x,y)$) as in this step they are canceled.}\\
\end{itemize}

The derivation at this step is: \\
\begin{scprooftree}{0.8}
  \AxiomC{$(\exists x)(\exists y)(\neg P_2 (x, y))$}
  \AxiomC{$(\exists y)(\neg P_2 (x, y))$}
  \AxiomC{$[\neg P_2 (x, y)]^2$}
  \AxiomC{$(\forall x)(\forall y)(P_3 (x,y) \leftrightarrow P_2 (x,y))$}
  \RightLabel{$E_{\forall *6}$}
  \UnaryInfC{$(\forall y)(P_3 (x,y) \leftrightarrow P_2 (x,y))$}
  \RightLabel{$E_{\forall *5}$}
  \UnaryInfC{$P_3 (x,y) \leftrightarrow P_2 (x,y)$}
  \AxiomC{$[P_3 (x, y)]^3$}
  \RightLabel{$E_{\leftrightarrow 1}$}
  \BinaryInfC{$P_2 (x, y)$}
  \RightLabel{$E_{\neg}$}
  \BinaryInfC{$\bot$}
  \LeftLabel{(3)}
  \RightLabel{$I_{\neg}$}
  \UnaryInfC{$\neg P_3 (x, y)$}
  \RightLabel{$I_{\exists *4}$}
  \UnaryInfC{$(\exists y)(\neg P_3 (x, y))$}
  \RightLabel{$I_{\exists *3}$}
  \UnaryInfC{$(\exists x)(\exists y)(\neg P_3 (x, y))$}
  \LeftLabel{(2)}
  \RightLabel{$E_{\exists *2}$}
  \BinaryInfC{$(\exists x)(\exists y)(\neg P_3 (x, y))$}
  \LeftLabel{(1)}
  \RightLabel{$E_{\exists *1}$}
  \BinaryInfC{$(\exists x)(\exists y)(\neg P_3 (x, y))$}
\end{scprooftree}

\newpage

\begin{itemize}
\item [$E_{\exists *1}$:]
\item This rule generates the hypotheses: $(\exists y)(\neg P_2 (x, y))$, let's cancel it.
\item \textbf{*1 justification:} $x \not\in$ $FV((\exists x)(\exists y)(\neg P_3 (x, y)))$, $x \not\in$ $FV((\forall x)(\forall y)(P_3 (x,y) \leftrightarrow P_2 (x,y)))$.\\
\textbf{Let's note that for this case also there's no need to check if any other hypotheses contain $x$ free: ($P_3 (x,y)$ ni en $\neg P_2 (x,y)$) because they are all canceled.}\\
\end{itemize}

The derivation at this step is: \\

\begin{scprooftree}{0.8}
  \AxiomC{$(\exists x)(\exists y)(\neg P_2 (x, y))$}
  \AxiomC{$[(\exists y)(\neg P_2 (x, y))]^1$}
  \AxiomC{$[\neg P_2 (x, y)]]^2$}
  \AxiomC{$(\forall x)(\forall y)(P_3 (x,y) \leftrightarrow P_2 (x,y))$}
  \RightLabel{$E_{\forall *6}$}
  \UnaryInfC{$(\forall y)(P_3 (x,y) \leftrightarrow P_2 (x,y))$}
  \RightLabel{$E_{\forall *5}$}
  \UnaryInfC{$P_3 (x,y) \leftrightarrow P_2 (x,y)$}
  \AxiomC{$[P_3 (x, y)]^3$}
  \RightLabel{$E_{\leftrightarrow 1}$}
  \BinaryInfC{$P_2 (x, y)$}
  \RightLabel{$E_{\neg}$}
  \BinaryInfC{$\bot$}
  \LeftLabel{(3)}
  \RightLabel{$I_{\neg}$}
  \UnaryInfC{$\neg P_3 (x, y)$}
  \RightLabel{$I_{\exists *4}$}
  \UnaryInfC{$(\exists y)(\neg P_3 (x, y))$}
  \RightLabel{$I_{\exists *3}$}
  \UnaryInfC{$(\exists x)(\exists y)(\neg P_3 (x, y))$}
  \LeftLabel{(2)}
  \RightLabel{$E_{\exists *2}$}
  \BinaryInfC{$(\exists x)(\exists y)(\neg P_3 (x, y))$}
  \LeftLabel{(1)}
  \RightLabel{$E_{\exists *1}$}
  \BinaryInfC{$(\exists x)(\exists y)(\neg P_3 (x, y))$}
\end{scprooftree}

And this, together with the justifications is the complete finished derivation.

\newpage

\section{How to use identity rules.}

Special thanks to Prof. Luis Sierra (FIng UdelaR) for the corrections. \\

\noindent\fbox{%
    \parbox{\textwidth}{%
\textbf{From an university test:} \newline \newline
Build a derivation that proves:
$$(\forall x)P(x, f(x)), (\exists y)f(y) =' y \vdash (\exists z)P(f(z), z)$$
    }%
}
\newline \newline

First of all, looking at the premises we can figure out that to reach the conclusion, using substition in the application of the rule $E_{\forall}$ won't lead us anywhere. For example, if we wanted to reach $(\exists z)P(f(z), z)$ with the premise $(\forall x)P(x, f(x))$, we should apply the rule $E_{\forall}$. If we wanted to obtain $f(z)$ instead of $x$, when substituting, we would get $P(f(z), f(f(z)))$. The only way to swawp the arguments of $P$ is by using the rule $RI4'$. \\

Let's build the derivation: \\

All we can do with the premises we have is an existential elimination:

\begin{prooftree}
\AxiomC{$(\exists y)f(y) =' y$}
\AxiomC{$(\exists z)P(f(z), z)$}
\LeftLabel{(1)}
\RightLabel{$E_{\exists *1}$}
\BinaryInfC{$(\exists z)P(f(z), z)$}
\end{prooftree}

Here, we obtained the hypotheses $f(y) =' y$. Checking the premises again we can see that with $(\forall x)P(x, f(x))$, by applying the rule $E_{\forall}$ and substituting adequately we obtain $P(y, f(y))$. If we manage to substitute in this formula in a way that we get $P(f(y), y)$, applying the rule $I_{\exists}$ and substituting adequately we can obtain $(\exists z)P(f(z), z)$. We can do this by using the identity rule $RI4'$.
Doing this, the resulting derivation is:

\begin{prooftree}
  \AxiomC{$(\exists y)f(y) =' y$}
  \AxiomC{$(\forall x)P(f(x), x)$}
  \UnaryInfC{$P(f(y), y)$}
  \AxiomC{$f(y) =' y$}
  \AxiomC{$f(y) =' y$}
  \UnaryInfC{$y =' f(y)$}
  \TrinaryInfC{$P(f(y), y)$}
  \UnaryInfC{$(\exists z)P(f(z), z)$}
  \LeftLabel{(1)}
  \RightLabel{$E_{\exists *1}$}
  \BinaryInfC{$(\exists z)P(f(z), z)$}
\end{prooftree}

By adding the rules in each step and using the strategy of the previous example, we have built the derivation: \\

\noindent\fbox{%
    \parbox{\textwidth}{%
 \textbf{*4}: $y$ free for $x$ in $P(f(x), x).$ \\
 \textbf{*3}: $y$ free for $z_1$; $f(y)$ free for $z_2$ en $P(z_1,z_2)$. \\
 $f(y)$ free for $z_1$; $y$ free for $z_2$ en $P(z_1,z_2)$.\\
 \textbf{*2}: $y$ free for $z$ en $P(f(z), z)$.\\
 \textbf{*1}: $y$ $\not \in$ $FV(\{(\exists z)P(f(z), z), (\forall x)P(f(x), x)\}).$
    }%
}

\begin{prooftree}
  \AxiomC{$(\exists y)f(y) =' y$}
  \AxiomC{$[f(y) =' y]^1$}
  \RightLabel{$RI2$}
  \UnaryInfC{$y =' f(y)$}
  \AxiomC{$[f(y) =' y]^1$}
  \AxiomC{$(\forall x)P(x, f(x))$}
  \RightLabel{$E_{\forall *4}$}
  \UnaryInfC{$P(z_1,z_2)[y/z_1,f(y)/z_2])$}
  \RightLabel{$RI4'_{*3}$}
  \TrinaryInfC{$P(z_1,z_2)[f(y)/z_1,y/z_2)]$}
  \RightLabel{$I_{\exists *2}$}
  \UnaryInfC{$(\exists z)P(f(z), z)$}
  \LeftLabel{(1)}
  \RightLabel{$E_{\exists *1}$}
  \BinaryInfC{$(\exists z)P(f(z), z)$}
\end{prooftree}

\end{document}


\section{Thermalization via a Nonlinear Boson Diffusion Equation (NBDE)}

\begin{frame}{Deriving the Nonlinear Boson Diffusion Equation I}
\vspace{0.5em}
The following derivation follows reference \cite{Wolschin2018}. \\[0.5em]
\begin{itemize}
\item The starting point for our investigation is the \alert{Boltzmann eqn.} (cf. Pavel's talk). %write it down
\item For \alert{spatial homogeneity} of the the boson distribution function $f(\mathbf{x}, \mathbf{p}, t)$ and a \alert{spherically symmetric momentum dependence} the equation for the single-particle occupation numbers  $n_j \equiv n_{\mathrm{th}}(\varepsilon_j,t)$ reads:
\begin{align}
\frac{\partial n_1}{\partial t} &= \sum_{\varepsilon_2,\varepsilon_3,\varepsilon_4}\langle V^{\phantom{.}2}\rangle G(\varepsilon_1+\varepsilon_2,\varepsilon_3+\varepsilon_4)\\
&\times \left[(1+n_1)(1+n_2)n_3n_4 - (1+n_3)(1+n_4)n_1n_2\right]
\end{align}

\item The \alert{collision term} can be written in the form of a \alert{Master eqn.}:
\begin{equation}
\frac{\partial n_1}{\partial_t} = (1+n_1)\sum_{\varepsilon_4}W_{4\rightarrow 1}n_4	- \sum_{\varepsilon_4}W_{1\rightarrow 4}(1+n_4)
\end{equation}
with
\begin{equation}
W_{4\rightarrow 1}=  W_{41}g_1 = \sum_{\varepsilon_2, \varepsilon_3} \langle V^{\phantom{.}2}\rangle G(\varepsilon_1+\varepsilon_2,\varepsilon_3+\varepsilon_4)(1+n_2)n_3
\end{equation}
\end{itemize}
\end{frame}

\begin{frame}{Deriving the Nonlinear Boson Diffusion Equation II}
\begin{itemize}
	\item In continuum $\sum\rightarrow\int$ and introduce \alert{density of states} $g_j \equiv g(\varepsilon_j)$.
	\item If $G$ acquires a width in a finite system:
	\begin{equation}
		W_{14}=W_{41}=W\left[\frac{1}{2}(\varepsilon_4+\varepsilon_1),\underbrace{\abs{\varepsilon_4-\varepsilon_1}}_{=: x}\right]
	\end{equation}
	\item Perform a \alert{gradient expansion} of $n_4$ and $g_4n_4$ around $x\approx 0$.
	\item Introduce \alert{transport coefficients} via moments of the transition probability:
\begin{align}
	D &= \frac{g_1}{2}\int\limits_0^{\infty}\dd x\ W(\varepsilon_1,x) \ x^2 \\
	v &= g_1^{-1}\frac{d}{d\varepsilon_1}(g_1D)
\end{align}
\end{itemize}
\end{frame}

\begin{frame}{Deriving the Nonlinear Boson Diffusion Equation III}
\begin{itemize}
	\item Nonlinear partial differential equation for $n \equiv n(\varepsilon_1,t) = n(\varepsilon,t)$:
		\begin{equation}
			\frac{\partial n}{\partial t} = -\frac{\partial}{\partial\varepsilon}\left[v\cdot n(1+n) + n\frac{\partial n}{\partial\varepsilon}\right] + \frac{\partial^2}{\partial\varepsilon^2}\left[Dn\right]\label{eqn:nbde1}
		\end{equation}

	\item Consider the limit of constant transport coefficients:
		\begin{equation}
			\frac{\partial n}{\partial t} = -v\frac{\partial}{\partial\varepsilon}\left[n(1+n)\right] + D\frac{\partial^2 n}{\partial\varepsilon^2}\label{eqn:nbde2}
		\end{equation}

\item Thermal \alert{Bose-Einstein distribution} provides stationary solution:
\begin{equation}
	n_{\mathrm{eq}}(\varepsilon) = \frac{1}{\exp(\frac{\varepsilon-\mu}{T}) - 1}
\end{equation}
\end{itemize}
\end{frame}

\begin{frame}{Some Remarks}
\begin{itemize}
	\item The present model does \alert{not} resolve the 2nd-order phase transition.
	\item The effects of condensation are included (cf. the following figures).
	\item A treatment resolving the singularity at $\epsilon=\mu$ is presented later.
\end{itemize}
\end{frame}

\begin{frame}{Linear Relaxation-Time Approximation (RTA)}
\begin{itemize}
	\item Given some initial distribution $n_{\mathrm{i}}(\varepsilon)$ we find an approximated solution for the thermalization process via the RTA:
\begin{equation}
	\frac{\partial n_{\mathrm{rel}}}{\partial t} = \frac{(n_{\mathrm{eq}} - n_{\mathrm{
rel}})}{\tau_{\mathrm{eq}}}
\end{equation}
with solution:
\begin{equation}
	n_{\mathrm{rel}}(\varepsilon,t) = n_{\mathrm{i}}(\varepsilon)\cdot\exp\left(-\frac{t}{\tau_{\mathrm{eq}}}\right) +  n_{\mathrm{eq}}(\varepsilon)\left(1-\exp\left(-\frac{t}{\tau_{\mathrm{eq}}}\right)\right)
\end{equation}
where $\tau_{\mathrm{eq}} = 4D/(9v^2)$.
\item Motivated by the study of early stages of RHICs, the initial distribution is chosen such that:
\begin{equation}
	n_{\mathrm{i}}(\varepsilon) = N_{\mathrm{i}}\cdot\theta\left(1-\varepsilon/Q_{\mathrm{s}}\right)\cdot\theta(\varepsilon) \label{eqn:rta_initial}
\end{equation}
with limiting momentum $Q_{\mathrm{s}} \sim \tau_0^{-1} \approx 1\ \mathrm{GeV}$.\mycite{Mueller1999}
\end{itemize}
\end{frame}

\begin{frame}{Results for the RTA}
\begin{figure}[H]
\centering
\includegraphics[width=0.8\textwidth]{figures/rta}
\caption{Relaxation of a finite Bose system towards the equilibrium. \cite{Wolschin2018} \\
Here $T = -D/v \simeq 0.4\ \mathrm{GeV}$, $\tau_{\mathrm{eq}} = 4D/(9v^2) = 0.33\cdot 10^{-23} \mathrm{s} \simeq 1\ \mathrm{fm/c}$ and the timesteps are $\left\{0.1, 0.25, 0.5,\infty\right\}$ (in units of $10^{-23}s$) from top to bottom.}
\end{figure}
\end{frame}
%TODO: Maybe add another slide about conservation etc.

\begin{frame}{Exact Solution of the Nonlinear Boson Diffusion Equation}
\begin{itemize}
	\item To solve eqn. (\ref{eqn:nbde1}) analytically, we perform the following \alert{nonlinear transformation:}
	\begin{equation}
		n(\varepsilon,t) = -\frac{D}{v}\frac{\partial \ln \mathcal{Z}(\varepsilon,t)}{\partial\varepsilon}
	\end{equation}
	which reduces our problem to a \alert{linear diffusion eqn.} for $ \mathcal{Z}(\varepsilon,t)$:
	\begin{equation}
		\frac{\partial  \mathcal{Z}}{\partial t} = -v\frac{\partial  \mathcal{Z}}{\partial \varepsilon} +  D\frac{\partial^2  \mathcal{Z}}{\partial \varepsilon^2}
	\end{equation}
	\item Solutions to this equation can be written as:
 \begin{equation}
n(\varepsilon, t)=\frac{1}{2 v} \frac{\int_{-\infty}^{+\infty} \frac{\varepsilon-x}{t} F(x)\cdot G_{\mathrm{free}}(\varepsilon-x,t)\ \dd x}{\int_{-\infty}^{+\infty} F(x)\cdot G_{\mathrm{free}}(\varepsilon-x,t)\ \dd x}-\frac{1}{2}
\end{equation}
\end{itemize}
\end{frame}

\begin{frame}{Additional Definitions}
\begin{itemize}
\item The quantities appearing in the solution are the \alert{free Green's function}
\begin{align}
	G_{\mathrm{free}}(\varepsilon-x,t) = \exp\left[-\frac{(\varepsilon-x)^2}{4Dt}\right],\\
\end{align}
and the implementation of the \alert{initial conditions}
\begin{equation}
	    F(x)  = \exp\left[-\frac{1}{2D}(vx+2v\int_0^x n_{\mathrm{i}}(y) \dd y) \right].
\end{equation}
\item They define the \alert{free partition function} via:
\begin{equation}
	\mathcal{Z}(\varepsilon,t) = a(t)\cdot\int_{\infty}^{\infty} G_{\mathrm{free}}(\varepsilon,x,t)\cdot F(x)\ \dd x
\end{equation}
\end{itemize}
\end{frame}

\begin{frame}{Results for the Solution of the NBDE I}
\begin{figure}[H]
\centering
\includegraphics[width=0.8\textwidth]{figures/nbde_positive_range}
\caption{Equilibration of a finite Bose system from the NBDE. \cite{Wolschin2018} \\
The integration range is restricted to $x \geq 0$. Here $T\simeq 0.4\ \mathrm{GeV}$, $\tau_{\mathrm{eq}} =  0.33\cdot 10^{-23} \mathrm{s}$ and the timesteps are $\left\{0.005, 0.05, 0.15,0.5\right\}$ (in units of $10^{-23}s$) from top to bottom.}
\end{figure}
\end{frame}

\begin{frame}{Results for the Solution of the NBDE II}
\begin{figure}[H]
\centering
\includegraphics[width=0.8\textwidth]{figures/nbde_full_range}
\caption{Equilibration of a finite Bose system from the NBDE. \cite{Wolschin2018} \\
The integration range is extended to $-\infty \leq x \leq \infty$. Here $T\simeq 0.4\ \mathrm{GeV}$, $\tau_{\mathrm{eq}} =  0.33\cdot 10^{-23} \mathrm{s}$ and the timesteps are $\left\{0.005, 0.05, 0.15,0.5\right\}$ (in units of $10^{-23}s$) from top to bottom.}
\end{figure}
\end{frame}

\begin{frame}{Results for the Solution of the NBDE III}
\begin{figure}[H]
\centering
\includegraphics[width=0.8\textwidth]{figures/nbde_gaussian}
\caption{Equilibration of a finite Bose system from the NBDE for Gaussian initial conditions $n_{\mathrm{i}}(\varepsilon) = N_{\mathrm{i}}\left(\sqrt{2\pi}\sigma\right)^{-1}\exp\left((\varepsilon - \langle\varepsilon\rangle)/(2\sigma^2)\right)$ with $\sigma = 0.04\ \mathrm{GeV}$. \cite{Wolschin2018} \\
Here $T\simeq 0.4\ \mathrm{GeV}$, $\tau_{\mathrm{eq}} =  0.33\cdot 10^{-23} \mathrm{s}$ and the timesteps are $\left\{0.002, 0.006, 0.02, 0.2\right\}$ (in units of $10^{-23}s$) from top to bottom.}
\end{figure}
\end{frame}

\begin{frame}{Treating the Singularity}
This part is based on the publication \cite{Wolschin2020_1} which provides an extension of \cite{Wolschin2018} and was published just recently.\\[0.5em]
\begin{itemize}
	\item To account for the singularity at $\varepsilon=\mu < 0$ we have to modify the initial distribution given before (eqn. (\ref{eqn:rta_initial})) as follows:
	\begin{equation}
		\tilde{n_{\mathrm{i}}}(\varepsilon) = n_{\mathrm{i}}(\varepsilon) + \frac{1}{\exp\left(\frac{\varepsilon-\mu}{T}\right)-1}
	\end{equation}
	\item The chemical potential $\mu$ has to be treated as a fixed parameter.
	\item Considering the limit $\lim_{\varepsilon\rightarrow\mu^{+}}\ n(\varepsilon,t) = \infty\ \forall t$ yields $\mathcal{Z}(\mu,t) = 0$.
	\item This results in a modified expression for the Green's function
	\begin{equation}
		G(\varepsilon,x,t) = G_{\mathrm{free}}(\varepsilon-\mu,x,t) - G_{\mathrm{free}}(\varepsilon-\mu,-x,t)
	\end{equation}
\end{itemize}
\end{frame}

\begin{frame}{Results for the RTA for the modified Initial Conditions}
\begin{figure}[H]
\centering
\includegraphics[width=0.8\textwidth]{figures/rta_full}
\caption{Local thermalization of gluons in the linear RTA for $\mu<0$. \cite{Wolschin2020_1} \\
Here $T \simeq 513\ \mathrm{MeV}$ and the timesteps are $\left\{0.02, 0.08, 0.15,0.3,0.6\right\}$ (in units of $\mathrm{fm}/c$) from top to bottom.}
\end{figure}
\end{frame}
%TODO: Elaborate on Solutions#2
\begin{frame}{Results for the full Solution of the NBDE}
\begin{figure}[H]
\centering
\includegraphics[width=0.8\textwidth]{figures/nbde_full_result}
\caption{Local thermalization of gluons from the time-dependent solutions of the NBDE for $\mu<0$. \cite{Wolschin2020_1} \\
Here $T \simeq 513\ \mathrm{MeV}$ and the timesteps are $\left\{6\cdot10^{-5}, 6\cdot10^{-4}, 6\cdot10^{-3},0.12,0.36\right\}$ (in units of $\mathrm{fm}/c$) from top to bottom.}
\end{figure}
\end{frame}


\input{../header.tex}
\title{\vspace{-2cm}INF3490/INF4490 Mandatory Assignment 1:\\
Travelling Salesman Problem}
\author{Eivind Samuelsen\input{../author_footnote.tex}}
\date{\today}

% Removing paragraph indents is sometimes useful:
%\setlength\parindent{0pt}
% ==============================================================================

% ================================= DOCUMENT ===================================
\begin{document}
    \renewcommand\marginsymbol[1][0pt]{%
  \tabto*{0cm}\makebox[-1cm][c]{$\mathbb{P}$}\tabto*{\TabPrevPos}}

\maketitle
\section*{Rules}
Before you begin the exercise, review the rules at this website:
\begin{center}
\url{http://www.uio.no/english/studies/admin/compulsory-activities/mn-ifi-mandatory.html}
\end{center}
(This is an individual assignment.
You are not allowed to deliver together or copy/share source-code/answers with others.)

\section*{Delivering}
\textbf{Deadline:} 2016-09-23 23:59:00. All deliveries must be made through
\href{https://devilry.ifi.uio.no}{\textbf{Devilry}}.
\section*{What to deliver?}
Deliver one single zipped folder (.zip, .tgz or .tar.gz) which includes:
\begin{itemize}
    \item PDF report containing:
    \begin{itemize}
        \item Your name and username (!)
        \item Instructions on how to run your program.
        \item Answers to all questions from assignment.
        \item \emph{Brief} explanation of what you've done.
    \end{itemize}
    \item Source code
    \item The european cities file so the program will run right away.
    \item Any files needed for the group teacher to easily run your program on
          IFI linux machines.
\end{itemize}
\emph{Important: } if you weren't able to finish the assignment, use the PDF
report to elaborate on what you've tried and what problems you encountered.
Students who have made an effort and attempted all parts of the assignment
will get a second chance even if they fail initially. This exercise will be
graded PASS/FAIL.
\section*{Introduction}
In this exercise, you will attempt to solve an instance of the traveling
salesman problem (TSP) using different methods. The goal is to become familiar
with evolutionary algorithms and to appreciate their effectiveness on a
difficult search problem. You may use
whichever programming language you like, but we strongly suggest that you try
to use Python, since you will be required to write the second assignment in
Python. You must write your program from scratch (but you may use
non-EA-related libraries).

\section*{Problem}

The traveling salesman, wishing to disturb the residents of the major cities
in some region of the world in the shortest time possible, is faced with the
problem of finding the shortest tour among the cities. A tour is a path that
starts in one city, visits all of the other cities, and then returns to the
starting point. The relevant pieces of information, then, are the cities and
the distances between them. In this instance of the TSP, a number of European
cities are to be visited. Their relative distances are given in the data file
found at
\url{http://www.uio.no/studier/emner/matnat/ifi/INF3490/h16/assignment-1/european_cities.csv}.\\
(You will use permutations to represent tours in your programs. If you use
Python, the itertools module provides a permutations function that returns
successive permutations, this is useful for exhaustive search)

\begin{table}[]
\centering
  \begin{tabular}{|c|c|c|c|c|c|c|}
    \hline
              & Barcelona & Belgrade & Berlin  & Brussels & Bucharest & Budapest \\ \hline
    Barcelona & 0         & 1528.13  & 1497.61 & 1062.89  & 1968.42   & 1498.79  \\ \hline
    Belgrade  & 1528.13   & 0        & 999.25  & 1372.59  & 447.34    & 316.41   \\ \hline
    Berlin    & 1497.61   & 999.25   & 0       & 651.62   & 1293.40   & 689.06   \\ \hline
    Brussels  & 1062.89   & 1372.59  & 651.62  & 0        & 1769.69   & 1131.52  \\ \hline
    Bucharest & 1968.42   & 447.34   & 1293.40 & 1769.69  & 0         & 639.77   \\ \hline
    Budapest  & 1498.79   & 316.41   & 689.06  & 1131.52  & 639.77    & 0        \\ \hline
  \end{tabular}
  \caption{First 6 cities from csv file.}
  \label{tab:cities}
\end{table}

\section*{Exhaustive Search}
First, try to solve the problem by inspecting every possible tour. Start by
writing a program to find the shortest tour among a subset of the cities (say,
6 of them). Measure the amount of time your program takes. Incrementally add
more cities and observe how the time increases.\\

What is the shortest tour (i.e., the actual sequence of cities, and its
length) among the first 10 cities (that is, the cities starting with B,C,D,H
and I)? How long did your program take to find it? How long would you expect
it to take with all 24 cities?

\section*{Hill Climbing}
Then, write a simple hill climber to solve the TSP. How well does the hill
climber perform, compared to the result from the exhaustive search for the
first 10 cities? Since you are dealing with a stochastic algorithm, you should
run the algorithm several times to measure its performance. Report the length
of the tour of the best, worst and mean of 20 runs (with random starting
tours), as well as the
\href{https://en.wikipedia.org/wiki/Standard_deviation}{\textbf{standard deviation}}
of the runs, both with the 10 first cities, and with all 24 cities.

\section*{Genetic Algorithm}
Next, write a genetic algorithm (GA) to solve the problem. Choose mutation
and crossover operators that are appropriate for the problem (see chapter 3
of the Eiben and Smith textbook). Choose three different values for the
population size. Define and tune other parameters yourself and make
assumptions as necessary (and report them, of course).\\

For all three variants: As with the hill climber, report best, worst, average
and deviation of tour length out of 20 runs of the algorithm (of the best
individual of last generation). Also, find and plot the average fitness of
the best fit individual in each generation (average across runs), and include
a figure with all three curves in the same plot in the report. Conclude which
is best in terms of tour length and number of generations of evolution time.\\

Among the first 10 cities, did your GA find the shortest tour (as found by the
exhaustive search)? Did it come close? For both 10 and 24 cities: How did the
running time of your GA compare to that of the exhaustive search? How many
tours were inspected by your GA as compared to by the exhaustive search?

\section*{Hybrid Algorithm (INF4490 only)}
Implement a hybrid algorithms to solve the TSP: Couple your GA and hill
climber by running the hill climber a number of iterations on each individual
in the population as part of the evaluation. Test both Lamarckian and
Baldwinian learning models and report the results of both variants in the same
way as with the pure GA (min, max, mean and std.dev. of the end result and an
averaged generational plot). How does the results compare to that of the pure
GA, considering the number of evaluations done?

\input{../contact.tex}
\end{document}
% ==============================================================================


\chapter{PV Panel Curves and simulation program} \label{ap:pvPanelCurves}

	This appendix shows the simulation curves of the photovoltaic panel static equation \eqref{eq:PVPanelModelModified}. It also shows listing \ref{lst:pvPanelCurves}, the Python program used to obtain those curves. The parameter values of the photovoltaic panels are denoted in table \ref{tab:panelParameters}.

\begin{figure}[h]
	\centering
	\includegraphics[angle = -90, width = 0.9\textwidth]{../images/pvPanelCurves/ivCurve.pdf}
	\caption{Panel transconductance (continuous) and MPP (dashed) curves with fixed temperature $\theta_{SRC}$ and varying irradiance.}
	\label{fig:ivCurve}
\end{figure}

\begin{figure}[h]
	\centering
	\includegraphics[angle = -90, width = 0.9\textwidth]{../images/pvPanelCurves/pvCurve.pdf}
	\caption{Panel power(continuous) and MPP (dashed) curves with fixed temperature $\theta_{SRC}$ and varying irradiance.}
	\label{fig:pvCurve}
\end{figure}

\begin{figure}[h]
	\centering
	\includegraphics[angle = -90, width = 0.9\textwidth]{../images/pvPanelCurves/ivCurveVaryingTemperature.pdf}
	\caption{Panel transconductance (continuous) and MPP (dashed) curves with fixed irradiation $\phi_{SRC}$ and varying temperature.}
	\label{fig:ivCurveVaryingTemperature}
\end{figure}

\begin{figure}[h]
	\centering
	\includegraphics[angle = -90, width = 0.9\textwidth]{../images/pvPanelCurves/pvCurveVaryingTemperature.pdf}
	\caption{Panel power (continuous) and MPP (dashed) curves with fixed irradiation $\phi_{SRC}$ and varying temperature.}
	\label{fig:pvCurveVaryingTemperature}
\end{figure}

\begin{figure}[h]
	\centering
	\includegraphics[angle = -90, width = 0.9\textwidth]{../images/pvPanelCurves/vocVersusIrradiance.pdf}
	\caption{Panel open-circuit voltage versus irradiance and temperature.}
	\label{fig:vocVersusIrradiance}
\end{figure}

	\lstinputlisting[caption = {Python PV panel simulation program developed to generate figures \ref{fig:ivCurve} to \ref{fig:pvCurveVaryingTemperature}}, label = {lst:pvPanelCurves}, style = apaListing, language = Python]{../images/pvPanelCurves/mppTracking.py}


\begin{comment}

 Licensed to the Apache Software Foundation (ASF) under one
 or more contributor license agreements.  See the NOTICE file
 distributed with this work for additional information
 regarding copyright ownership.  The ASF licenses this file
 to you under the Apache License, Version 2.0 (the
 "License"); you may not use this file except in compliance
 with the License.  You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing,
 software distributed under the License is distributed on an
 "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 KIND, either express or implied.  See the License for the
 specific language governing permissions and limitations
 under the License.

\end{comment}

\subsection{Principal Component Analysis}
\label{pca}

\noindent{\bf Description}

Principal Component Analysis (PCA) is a simple, non-parametric method to transform the given data set with possibly correlated columns into a set of linearly uncorrelated or orthogonal columns, called {\em principal components}. The principal components are ordered in such a way that the first component accounts for the largest possible variance, followed by remaining principal components in the decreasing order of the amount of variance captured from the data. PCA is often used as a dimensionality reduction technique, where the original data is projected or rotated onto a low-dimensional space with basis vectors defined by top-$K$ (for a given value of $K$) principal components.
\\

\noindent{\bf Usage}

\begin{tabbing}
\texttt{-f} \textit{path}/\texttt{PCA.dml -nvargs}
\=\texttt{INPUT=}\textit{path}/\textit{file}
  \texttt{K=}\textit{int} \\
\>\texttt{CENTER=}\textit{0/1}
  \texttt{SCALE=}\textit{0/1}\\
\>\texttt{PROJDATA=}\textit{0/1}
  \texttt{OFMT=}\textit{csv}/\textit{text}\\
\>\texttt{MODEL=}\textit{path}$\vert$\textit{file}
  \texttt{OUTPUT=}\textit{path}/\textit{file}
\end{tabbing}

\noindent{\bf Arguments}

\begin{itemize}
\item INPUT: Location (on HDFS) to read the input matrix.
\item K: Indicates dimension of the new vector space constructed from $K$ principal components. It must be a value between $1$ and the number of columns in the input data.
\item CENTER (default: {\tt 0}): Indicates whether or not to {\em center} input data prior to the computation of principal components.
\item SCALE (default: {\tt 0}): Indicates whether or not to {\em scale} input data prior to the computation of principal components.
\item PROJDATA: Indicates whether or not the input data must be projected on to new vector space defined over principal components.
\item OFMT (default: {\tt csv}): Specifies the output format. Choice of comma-separated values (csv) or as a sparse-matrix (text).
\item MODEL: Either the location (on HDFS) where the computed model is stored; or the location of an existing model.
\item OUTPUT: Location (on HDFS) to store the data rotated on to the new vector space.
\end{itemize}

\noindent{\bf Details}

Principal Component Analysis (PCA) is a non-parametric procedure for orthogonal linear transformation of the input data to a new coordinate system, such that the greatest variance by some projection of the data comes to lie on the first coordinate (called the first principal component), the second greatest variance on the second coordinate, and so on. In other words, PCA first selects a normalized direction in $m$-dimensional space ($m$ is the number of columns in the input data) along which the variance in input data is maximized -- this is referred to as the first principal component. It then repeatedly finds other directions (principal components) in which the variance is maximized. At every step, PCA restricts the search for only those directions that are perpendicular to all previously selected directions. By doing so, PCA aims to reduce the redundancy among input variables. To understand the notion of redundancy, consider an extreme scenario with a data set comprising of two variables, where the first one denotes some quantity expressed in meters, and the other variable represents the same quantity but in inches. Both these variables evidently capture redundant information, and hence one of them can be removed. In a general scenario, keeping solely the linear combination of input variables would both express the data more concisely and reduce the number of variables. This is why PCA is often used as a dimensionality reduction technique.

The specific method to compute such a new coordinate system is as follows -- compute a covariance matrix $C$ that measures the strength of correlation among all pairs of variables in the input data; factorize $C$ according to eigen decomposition to calculate its eigenvalues and eigenvectors; and finally, order eigenvectors in the decreasing order of their corresponding eigenvalue. The computed eigenvectors (also known as {\em loadings}) define the new coordinate system and the square root of eigen values provide the amount of variance in the input data explained by each coordinate or eigenvector.
\\

%As an example, consider the data in Table~\ref{tab:pca_data}.
\begin{comment}
\begin{table}
\parbox{.35\linewidth}{
\centering
\begin{tabular}{cc}
  \hline
  x & y \\
  \hline
  2.5 & 2.4  \\
  0.5 & 0.7  \\
  2.2 & 2.9  \\
  1.9 & 2.2  \\
  3.1 & 3.0  \\
  2.3 & 2.7  \\
  2 & 1.6  \\
  1 & 1.1  \\
  1.5 & 1.6  \\
  1.1 & 0.9  \\
	\hline
\end{tabular}
\caption{Input Data}
\label{tab:pca_data}
}
\hfill
\parbox{.55\linewidth}{
\centering
\begin{tabular}{cc}
  \hline
  x & y \\
  \hline
  .69  & .49  \\
  -1.31  & -1.21  \\
  .39  & .99  \\
  .09  & .29  \\
  1.29  & 1.09  \\
  .49  & .79  \\
  .19  & -.31  \\
  -.81  & -.81  \\
  -.31  & -.31  \\
  -.71  & -1.01  \\
  \hline
\end{tabular}
\caption{Data after centering and scaling}
\label{tab:pca_scaled_data}
}
\end{table}
\end{comment}

\noindent{\bf Returns}
When MODEL is not provided, PCA procedure is applied on INPUT data to generate MODEL as well as the rotated data OUTPUT (if PROJDATA is set to $1$) in the new coordinate system.
The produced model consists of basis vectors MODEL$/dominant.eigen.vectors$ for the new coordinate system; eigen values MODEL$/dominant.eigen.values$; and the standard deviation MODEL$/dominant.eigen.standard.deviations$ of principal components.
When MODEL is provided, INPUT data is rotated according to the coordinate system defined by MODEL$/dominant.eigen.vectors$. The resulting data is stored at location OUTPUT.
\\

\noindent{\bf Examples}

\begin{verbatim}
hadoop jar SystemML.jar -f PCA.dml -nvargs
            INPUT=/user/biuser/input.mtx  K=10
            CENTER=1  SCALE=1
            OFMT=csv PROJDATA=1
				    # location to store model and rotated data
            OUTPUT=/user/biuser/pca_output/
\end{verbatim}

\begin{verbatim}
hadoop jar SystemML.jar -f PCA.dml -nvargs
            INPUT=/user/biuser/test_input.mtx  K=10
            CENTER=1  SCALE=1
            OFMT=csv PROJDATA=1
				    # location of an existing model
            MODEL=/user/biuser/pca_output/
				    # location of rotated data
            OUTPUT=/user/biuser/test_output.mtx
\end{verbatim}