\lab{Python}{Array Programming}{Array Programming}
\label{lab:Python_Vectorization}

NumPy allows Python to behave like an array programming language.
It does this by allowing operations that normally apply only to scalars to also work seamlessly with NumPy arrays.
Some of the benefits of allowing these types of operations are concise code and fast execution.
Array programming allows operations on entire arrays without explicit \li{for} loops.
In Python this is desirable because explicit \li{for} loops are generally slow.
This lab is some fun exercises using array programming!

\section*{Shuffle}
This exercise is to write a program that shuffles a deck of a cards how a human would shuffle them, as opposed to having a random number generator put them in a random arrangement.
In order to this, first, cut the deck in half.
Say that either the top card in the first half goes down then the top card in the second half or vice-versa.
Which one comes first should be random.
Follow the same procedure for the rest of the cards in the two halves.
The following is the code implemented using a loop.
\begin{lstlisting}
from random import randint
def shuffle(deck):
    size = deck.size
    newdeck = np.empty_like(deck)
    for i in xrange(size/2):
         if randint(0,1) == 0:
            newdeck[i*2] = deck[i]
            newdeck[i*2+1] = deck[size/2+i]
         else:
            newdeck[i*2] = deck[size/2+i]
            newdeck[i*2+1] = deck[i]
    return newdeck
\end{lstlisting}

\begin{problem}
Write a function that shuffles a deck the same way as the function above using only array operations.
\end{problem}

\section*{Image Editor}
An image is often represented as a 3D array where the first and second dimensions are the height and width respectively and the third dimension represents the intensity of each of three color channels: red, green, and blue.
The code below can be used to read and display images in Python.
\begin{lstlisting}
import matplotlib.pyplot as plt
img = plt.imread(<image-file>)
plt.imshow(img)
plt.show()
\end{lstlisting}

Matplotlib can read in \li{.png} images on its own, otherwise it will read in images using the Python Image Library.
Depending on the format of the image, it may be read as an array of floating point values between 0 and 1 or it may be read as an array of integer values between 0 and 255.
In this lab we will assume the latter.

\begin{problem}
Write a function that edits an array representing an image.
Include options that allow you to  invert the image, change it to grayscale, or add a motion blur (definitions below).
The function should take in a image, a parameter that tells how to modify the image, and a optional parameter for the motion blur.
It should then plot the image.
\end{problem}

\begin{itemize}
\item Invert:
Every color value for every pixel is changed to its inverse value. For example, 0 becomes 255, 230 becomes 25, and 127 becomes 128.
Remember that the minimum color value is 0 and the maximum is 255.

\item Grayscale:
To convert an image to grayscale, each pixel’s color value is changed to the average of
the pixel’s red, green, and blue value. For example, if the pixel values are:
Red: 225 Green: 30 Blue: 131, we convert the

Grayscale coversion: $\frac{225 + 30 + 131}{3} = 128$ (using integer division)
Red: 128 Green: 128 Blue: 128

\item Motion Blur:
An additional parameter $n>0$ will be used for motion blur.
The value of each color of each pixel is the average of that color value for $n$ pixels (from
the current pixel to $n-1$) horizontally. So pixel \li{[x,y,0]} would turn in to the average of
pixel \li{[ x, y,0 ]} to pixel \li{[ x,y+n-1,0]}.
Note: You will need to use one \li{for} loop here.
Be sure to account for the situations where one or more of the values used in the computing the average do not exist. For example, if an image has width w and we are considering the pixel on row r, column c, if c + n >= w, then we only average the pixels up to w. (Proper array slicing should take care of this case without any extra code.)

\end{itemize}


\section{Related Work}
\label{sec:rw}
% (max 1 page. You briefly describe for the methods you will use, what
% they do and what is their role. E.g. you describe what clustering does and what
% techniques exist for clustering.
In this chapter we introduce some concepts and techniques used to preprocess
the dataset and to find popular consistent topics in the tweets.
We also briefly introduce what other methods could be used to find
popular topics.

\mathchardef\mhyphen="2D
\subsection{TF-IDF}

In several applications, such as the one treated in this paper,
it can be useful to categorize documents by their topic. Typically, this
is done by finding, for each document, the words that characterize it.

Term Frequency - Inverse Document Frequency (TF-IDF) is a numerical statistic
that is intended to reflect how important a word is for a document in a collection.~\cite{rw:tfidf}
This measure consists of a product of two components: TF and IDF.

\begin{definition}
    The \emph{Term Frequency} (\emph{TF}) of a word $i$ in a document $j$ is defined as
    \begin{equation*}
        \mathit{TF}_{ij} = \frac{\mathit{f}_{ij}}{\max_k{\mathit{f}_{kj}}}
    \end{equation*}
    where
    $\mathit{f}_{ij}$ is the number of times $i$ appears in $j$.
\end{definition}

\begin{definition}
    The \emph{Inverse Document Frequency} (\emph{IDF}) of a word $i$ in a document $j$ is defined as
    \begin{equation*}
        \mathit{IDF}_{ij} = \log{\frac{N}{n_i}}
    \end{equation*}
    where $N$ is the total number of documents and $n_i$ is the number of documents the word $i$
    appears in.
\end{definition}

\begin{definition}
    The \emph{Term Frequency - Inverse Document Frequency} (\emph{TF-IDF}) of a word $i$ in a document
    $j$ is defined as
    \begin{equation*}
        \mathit{TF \mhyphen IDF}_{ij} = \mathit{TF}_{ij} * \mathit{IDF}_{ij}
    \end{equation*}
\end{definition}

As we discuss in Section~\ref{sec:ds}, this technique can be used to preprocess the tweets in order to
keep only relevant terms for each tweet. The objective is to obtain a more
suitable dataset, excluding words that are used frequently but
do not characterize the topic the tweets are about. We do this in order to obtain higher quality
results and also to reduce the dimension of the input, so to speed up the execution time.

\subsection{Finding Frequent Itemsets}
A subtask of the problem consists in finding popular topics in a period of time. This task can
be reduced to the more general scenario in which we are given a list of
transactions, each containing a set of items. The problem to face is to find frequent
itemsets, that are sets of items that appear
together in many transactions, in order to find, for instance, association rules.
The fraction of transactions in which an itemset $i$ is present is called \emph{support}
of $i$, and an itemset is said to be frequent if its support is at least $s$, where
$s$ is a threshold set in advance.
An algorithm that solves this task efficiently is A-Priori.

\subsubsection{A-Priori}
The algorithm was first introduced in 1994 by Agrawal R.S. and Srikant R. and
it is a very fast algorithm to discover frequent itemsets in a
list of transactions.~\cite{rw:apriori}

The efficiency of the algorithm is based on the following observation:
if an itemset is frequent, then all its subsets are frequent too.
This property of monotonicity is exploited by the algorithm by
seeing the implication in reverse way, that is itemsets whose
subsets are not all frequent can not become frequent, so a great
quantity of potential candidates is discarded.

The idea of A-Priori is shown in Algorithm~\ref{alg:apriori_basic}.
\input{algorithms/apriori_basic.tex}

At each iteration we have a set of candidate itemsets of size $k$ and
we find among them frequent itemsets of size $k$.
Initially, frequent itemsets of size $1$ are the sets containing a single item.
At each iteration, we get the frequent itemsets of size $k$, which are
the itemsets among the candidates with a support at least $s$.
Then, we build the candidates of size $k+1$, by merging two frequent itemsets
of size $k$ and checking if all the subsets of size $k$ of the new candidate
are frequent. We repeat this process until the set of candidates is not empty.

Though A-Priori avoids counting many itemsets, there are numerous optimizations
that make a better use of the main memory.

Among these, PCY, the Multistage and the Multihash algorithms try to
further reduce the memory used to count frequent itemsets
that are known an advance not to be candidates to
become frequent.~\cite{alg:pcy}\cite{alg:multi}

The SON algorithm applies a different heuristic, by dividing the file in
chunks that can be stored in main memory to individuate
the candidate frequent itemsets more rapidly.~\cite{alg:son}

In this paper we focus just on the A-Priori algorithm, although the other
algorithms could be used to deal with larger datasets.


\subsubsection{Yee Cell and Leapfrog Method}

According to ref.~\cite{yee} Eq.~\ref{eq:maxwell} is discretized with $t=q\Delta_t$ and $x=m\Delta_x$ where $q,m\in\left\{\dots,-1,-\frac{1}{2},0,\frac{1}{2},1,\dots\right\}$ and $\Delta_x, \Delta_t$ being the division of our space and time grid.

Taking equation~\ref{eq:maxwell} and using the time steps $q+\frac{1}{2}$ and $q-\frac{1}{2}$ and the partition steps $m+1$ and $m$ this gives

\begin{equation}
  \mu\frac{H_y\left[\left(q+\frac{1}{2}\right)\Delta_t, (m+\frac{1}{2})\Delta_x\right]-H_y\left[\left(q-\frac{1}{2}\right)\Delta_t, (m+\frac{1}{2})\Delta_x\right]}{\Delta_t}=\frac{E_z\left[q\Delta_t, (m+1)\Delta_x)-E_z(q\Delta_t, m\Delta_x)\right]}{\Delta_x}.
\end{equation}

Solving for future $H_y$ yields

\begin{multline}
  H_y\left[\left(q+\frac{1}{2}\right)\Delta_t, \left(m+\frac{1}{2}\right)\Delta_x\right]=H_y\left[\left(q-\frac{1}{2}\right)\Delta_t, \left(m+\frac{1}{2}\right)\Delta_x\right] +\\
  \underbrace{\frac{\Delta_t}{\mu\Delta_x}\left(E_z\left[q\Delta_t, (m+1)\Delta_x\right]-E_z\left[q\Delta_t, m\Delta_x\right]\right)}_\text{update function for $H_y$}.
\end{multline}

This means the value of the magnetic field at a given point $(m+1/2)\Delta_x$ can be calculated from it's last value and the surrounding electric fields at a prior time. The difference between $H_y((q+\frac{1}{2})\Delta_t)$ and $H_y((q-\frac{1}{2})\Delta_t)$ is called the update function.

Likewise these calculations can be done for $E_z$ to get an update function to calculate the next value of $E_z$ from it's old value and the surrounding two $H_y$ values.

\begin{figure}[!h]
  \centering
  \includegraphics[width=0.5\textwidth]{./images/space-time-cell.png}
  \caption{Each value of $\mathbf{E}$ at a specific time can be calculated by the former value at the same position in space one time step before and the surrounding values of $\mathbf{H}$ half a timestep before. Following this algorithm the diagram is built from bottom to top line by line which is the reason this method is called leap frog method.}
  \label{fig:leapfrog}
\end{figure}

To get the electric field of any given system at a specific time this algorithm calculates $E_z$ and $H_y$ with $q$ starting at 0 and increasing by $\frac{1}{2}$ until the required time $t = q\Delta_t$ is reached, as schematically shown in figure~\ref{fig:leapfrog}. The only required input is the initial field of $E_z$ at $q = 0$. Because this method calculates $E$ and $H$ in turns it is also called the "leap frog method".

To account for different materials within the system, $\varepsilon$ can be replaced by a location or even time dependent $\varepsilon(t, x)$.

\begin{figure}[!h]
  \centering
  \includegraphics[width=0.8\textwidth]{./images/yeecell.jpg}
  \caption{To visualize the different points that are calculated during the process for three dimensions there is the so called Yee Cell that is formed by taking the points for $\mathbf{E}$ at $q=0$ and $\mathbf{H}$ at $q=\frac{1}{2}$. Each value for $\mathbf{H}$ can be calculated by its surrounding values of $\mathbf{E}$ at the time step $q-\frac{1}{2}$.}
  \label{fig:yeecell}
\end{figure}

Analogous to the calculation done in one dimension, the same algorithm can be applied to three dimensional space to obtain $\mathbf{E}(\mathbf{r}, t)$ and $\mathbf{H}(\mathbf{r}, t)$. By using the points $E_x$, $E_y$ and $E_z$ with the corresponding points $H_x$, $H_y$ and $H_y$ a base cell for a grid is formed. This cell is called "Yell Cell" and can be seen in a bigger lattice in figure~\ref{fig:yeecell}.


\section{Infinite Sets}
\subsection{Countability}
Why $\setn$ being infinite implies $\setn - \{0\}$ being so?
\begin{proof}
\begin{align*}
&X \text{ being finite} \to X \cup \{x\} \text{ being finite} \tag{3.6.14(a)} \\
\equiv &X \cup \{x\} \text{ being infinite} \to X  \text{ being infinite}
\end{align*}
\end{proof}

\paragraph{Examples 8.1.3}
Why $f: n \mapsto 2n$ gives a bijection? Injectivity: $m \ne n \to 2m \ne 2n$; Surjectivity: $\forall n \in \setn, 2n \in f(\setn)$.

\declareexercise{8.1.2}
Proof using induction:
\begin{proof}
Suppose that for $S \subseteq \setn$, there is no smallest element in $S$. We now prove that $S$ must be empty.

First, $0 \notin S$, otherwise, 0 would be the smallest element.

Now suppose that for some $N$, $\forall n \le N, n \not in S$, then $N+1$ must not be in $S$, otherwise it would be the smallest number.

We can now close the induction.
\end{proof}

Proof using the least upper bound property:
\begin{proof}
It is obvious that the nonempty set $S \subseteq \setn$ has a lower bound. For example, $0$ is one. Therefore, $\exists L \in \setr$ to be the greatest lower bound of $S$. We now show that $L$ is the smallest element of $S$.

$L$ can only be an integer. Otherwise, all real numbers between $L$ and $\ceiling{L}$ would be a lower bound bigger than $L$. In addition, $L \in S$, otherwise, $L+1$ would be a lower bound ($\forall x \in S, x>L \to x \ge L+1$). Therefore, $L$ is the smallest number.
\end{proof}

\declareexercise{8.1.3}
Gap Number
\begin{enumerate}
\item $X$ must be unbounded, for any $S \subseteq \setn$ bounded above by $M \in setn$ cannot have a cardinality bigger than $M+1$. Therefore $X\setminus\{a_m,m\le n\}$ is also unbounded, and is thus infinite (Remark 3.6.13).
\item Denote the set $\{x \in X: x \neq a_m \forall m < n\}$ as $X_n$, and we have $X_{n} \subseteq X_{n-1}$. Since that $a_n$ is the smallest element in $X_n$, all elements in $X_m$ for $m > n$ is bigger than $a_n$. Thus $a_{n-1} < a_n$ follows from there.
\item There is no equality in the previous relation.
\item This is nearly obvious because we are selecting elements from $X$ and its subsets.
\item $\forall n \forall m < n, a_n \ne x \to x \ne a_m$.
\item We have both $a_0 \ge 0$ and $a_n < a_{n+1} \to a_{n+1} \ge a_n + 1$. Thus $a_n > n$ can be easily shown with induction.
\item Otherwise $g$ could not be both bijective and increasing. If $g(m) \ne a_m$, then $g(m) > a_m$ since $a_m$ is the smallest element in the remaining set. $g$ is a bijection, so $a_m$ must be equaled by $g(n)$ with some $n > m$, a contradiction to the fact that $g$ is increasing.
\end{enumerate}

\declareexercise{8.1.4}
\begin{proof}
It is obvious that when restricted to $A$, $f$ becomes injective. Now we show that $\forall x \in f(N), \exists n \in A, f(n) = x$.

Suppose for sake of contradiction that there are elements $y \in f(N)$ such that $\forall x \in A, f(x) \ne y$. Note that we must also have $\exists n \in \setn \setminus A, f(n) = y$, which means the set $S := \setn \setminus A$ is non-empty. $S$ is also a subset of $\setn$, so it has a smallest element, and let's call it $m$.

All numbers between 0 and $m$ thus become elements in $A$. By definition, $m$ cannot equal to any of them, that is, $\forall 0\le x \le m, f(x) \ne f(m)$. But this implies that $m$ is an element of $A$, a contradiction.

Therefore, $f: A \to f(N)$ is a bijection. $f(N)$ then is proved to be at most countable since $A$ is a subset of $\setn$.
\end{proof}

\declareexercise{8.1.5}
\begin{proof}
Since $X$ is countable, there is a bijection $g: \setn \to X$, which gives $X = g(\setn)$. So $f(X) = f(g(\setn)) = f \circ g (\setn)$. Therefore Corollary 8.1.9 follows from Proposition 8.1.8.
\end{proof}

\declareexercise{8.1.6}
\begin{proof}
If $A$ is finite, then there exists a bijection $f: A \to S = \{m\in \setn : 1 \le m \le \#A\}$. Since that $S$ is a subset of $\setn$, $f:A \to \setn$ is injective.

If $A$ is countable, then there is a bijection $f: A \to \setn$, which is itself injective.

We have proved that if $A$ is at most countable, then there is an injective function $f: A \to \setn$.

On the other hand, if there is an injective map $f: A \to \setn$, then $f: A \to f(A)$ is a bijection. Since that $f(A)$ is a subset of $\setn$, it is at most countable, which implies that $A$ is at most countable.
\end{proof}

\declareexercise{8.1.7}
\begin{proof}
Since $f$ is bijective, for all $x \in X, \exists n \in \setn, f(n) = X$. But for all $n$, $h(2n) = f(n)$, so it means $h$ iterates every element of $X$. Similarly, $h$ iterates every element of $Y$. Thus, $X\cup Y \subseteq h(\setn)$. But by definition, $h(\setn) \subseteq X \cup Y$, so we must have $h(\setn) = X \cup Y$.

Both $X$ and $Y$ are infinite, so their union cannot be finite. According to Proposition 8.1.8, $X\cup Y$ thus can only be countable.
\end{proof}

\declareexercise{8.1.8}
\begin{proof}
Since that $X,Y$ are countable, there are two bijections: $f: \setn \to X, g: \setn \to Y$. Define $h:(m,n) \mapsto (f(m),g(n))$, and we can see that $h$ is a bijection from $\setn \times \setn$ to $X \times Y$.

Because $\setn \times \setn$ is countable, so is $X \times Y$.
\end{proof}


\chapter{Silhouettes}
\label{ch:silhouettes}

\newthought{Consider a two-feature data set} which we have painted in the \widget{Paint Data} widget. We send it to the k-means clustering, tell it to find three clusters, and display the clustering in the scatter plot.\marginnote{Don't get confused: we paint data and/or visualize it with Scatter plots, which show only two features. This is just for an illustration! Most data sets contain many features and methods like k-Means clustering take into account all features, not just two.}

\begin{marginfigure}
    \centering
    \includegraphics[width=\linewidth]{paint-and-kmeans.png}
    \caption{$\;$}
\end{marginfigure}

\begin{figure}[h]
    \centering
    \includegraphics[width=\linewidth]{silhouette-from-paint.png}
    \caption{$\;$} % empty caption for proper pagesetting
\end{figure}

The data points in the green cluster are well separated from those in the other two. Not so for the blue and red points, where several points are on the border between the clusters. We would like to quantify the degree of how well a data point belongs to the cluster to which it is assigned.

\begin{marginfigure}
    \centering
    \includegraphics[scale=0.25]{silhouette-pt1.png}
    \caption{Average distance A.}
\end{marginfigure}

We will invent a scoring measure for this and we will call it a silhouette (because this is how it's called). Our goal: a silhouette of 1 (one) will mean that the data instance is well rooted in the cluster, while the score of 0 (zero) will be assigned to data instances on the border between two clusters.

For a given data point (say the blue point in the image on the left), we can measure the distance to all the other points in its cluster and compute the average. Let us denote this average distance with A. The smaller the A, the better.

On the other hand, we would like a data point to be far away from the points in the closest neighboring cluster. The closest cluster to our blue data point is the red cluster. We can measure the distances between the blue data point and all the points in the red cluster, and again compute the average. Let us denote this average distance as B. The larger the B, the better.

\begin{marginfigure}
    \centering
    \includegraphics[scale=0.25]{silhouette-pt2.png}
    \caption{Average distance B.}
\end{marginfigure}

The point is well rooted within its own cluster if the distance to the points from the neighboring cluster (B) is much larger than the distance to the points from its own cluster (A), hence we compute B-A. We normalize it by dividing it with the larger of these two numbers, S  = (B -A) / max{A, B}. Voilá, S is our silhouette score.

Orange has a \widget{Silhouette Plot} widget that displays the values of the silhouette score for each data instance. We can also choose a particular data instance in the silhouette plot and check out its position in the scatter plot.

\marginnote{C3 is the green cluster, and all its points have large silhouettes. Not so for the other two.}

\begin{figure}[h]
    \centering
    \includegraphics[width=\linewidth]{silhouette-workflow.png}
    \caption{$\;$} % empty caption for proper pagesetting
\end{figure}

This of course looks great for data sets with two features, where the scatter plot reveals all the information. In higher-dimensional data, the scatter plot shows just two features at a time, so two points that seem close in the scatter plot may be actually far apart when all features - perhaps thousands of gene expressions - are taken into account. \marginnote{We selected three data instances with the worst silhouette scores. Can you guess where they lie in the scatter plot?}

\begin{figure}[h!]
    \centering
    \includegraphics[width=\linewidth]{silhouette-outliers.png}
    \caption{$\;$} % empty caption for proper pagesetting
\end{figure}

\newpage

The total quality of clustering - the silhouette of the clustering - is the average silhouette across all points. When the \widget{k-Means} widget searches for the optimal number of clusters, it tries a different number of clusters and displays the corresponding silhouette scores.
Ah, one more thing: Silhouette Plot can be used on any data, not just on data sets that are the output of clustering. We could use it with the iris data set and figure out which class is well separated from the other two and, conversely, which data instances from one class are similar to those from another.

We don't have to group the instances by the class. For instance, the silhouette on the left would suggest that the patients from the heart disease data with typical anginal pain are similar to each other (with respect to the distance/similarity computed from all features), while those with other types of pain, especially non-anginal pain are not clustered together at all.

\begin{marginfigure}
    \centering
    \includegraphics[width=\linewidth]{silhouette-chest-pain.png}
    \caption{$\;$}
\end{marginfigure}

\begin{figure}[h]
    \centering
    \includegraphics[width=\linewidth]{kmeans-silhouette-scores.png}
    \caption{$\;$} % empty caption for proper pagesetting
\end{figure}


\chapter{Variational Principles and Lagrange's Equations}
\section{Hamilton's Principle}
\subsection{Configuration Space}
\begin{itemize}
	\item The Instantaneous configuration of a system is described by the values of $n$ generalized coordinates $q_1,    q_n$ and corresponds to a particular point in a cartesian hyperspace where the q's from the $n$  diamensional space is known as configuration space.
	\item As time goes by, the state of the system changes and the system point moves in configuration space tracing out a curve, described as "the path of motion of the system".
	\item The motion of the system then refered to the motion of the system point along this path in configuration space
	\item Time can be considered formally as a parameter of the curve to each point on the path there is associated one or more values of the time. Each point on the path represent the entire system  configuration at some given instant of time.\\\\
	\textbf{Monogenic System}: The mechanical systems whose motion for which all forces  (except the forces of constraint) are derivable from a generalized scalar potential that may be a function of the coordinates, velocities and time.
\end{itemize}
	\subsection{Hamilton's Principle}
	The motion of the system from time $t_1 $ to time $t_2$ is such that the line integral called the avtion or the action integral.
	\begin{equation}
	I=\int\limits_{t_1}^{t_2}Ldt
	\end{equation}
	where $L$ is $T-V$ has stationary value for the actual path of the motion.\\\\
	ie. out of all possible path by which the system point could travel from it's position at time $t_1$ to its position at time $t_2$. It will travel along that path for which the value of integral is stationary.\\\\
	$\therefore$ Hamilton's principle can summarize by saying that the motion is such that the variation of the line integral $I$ for fixed $t_1$ and $t_2$ is zero.
	\begin{equation}
		\delta I=\delta\int\limits_{t_1}^{t_2}L(q_1,.....q_n,\dot{q}_1.....\dot{q}_n,t)dt=0
	\end{equation}
	Where the system constraints are holonomic, Hamilton's principle is both a necessary and suffitient condition for lagrange's equations.
	\section{Calculus of Variation}
	Consider a one diamensional problem, we have a function $f(y,\dot{y},x)$ defined on a path $y-y(x)$  between two value $x_1$ and $x_2$ where $\dot{y}$ is the derivative of $y$ with respect to $x$.\\\\
	To find a particular path y(x) such that the line integral $y$ of the function $f$ between $x_1$ and $x_2$\\
	$ \dot{y}\equiv \frac{dy}{dx}$
	\begin{equation}
	J=\int\limits_{x_1}^{x_2}f(y,\dot{y},x)dx
	\end{equation}
 	has stationary value relative to paths differing infinitesimally from the correct function $y(x)$.\\\\
 	Since $J$ must have a stationary value for the correct path relative to any neighboring path, the variation must be zero relative to some particular set of neighboring paths labeled by an infinitesimal parameter $\alpha$.\\
 	such a set of varied path given by
 	\begin{equation}
 	y(x,\alpha)=y(x,0)+\alpha \eta(x)\label{VP-eq04}
 	\end{equation}
	$y(x,0)$- Correct path\\
	$\eta(x)$-any function vanishes at $x=x_1$ and $x=x_2$\\\\
	Assume both the correct path $y(x)$ and the auxillary function $\eta(x)$ are well behaved functions between $x_1$ and $x_2$. For such any parametric family of curves, $J$ is also a function of $\alpha$
	\begin{equation}
	J(\alpha)=\int\limits_{x_1}^{x_2}f(y(x,\alpha),\dot{y}(x,\alpha),x)dx
	\end{equation}
	 Condition for obtaining stationary points is
	 \begin{equation}
	 \left( \frac{dJ}{d\alpha}\right) _{\alpha=0}=0
	 \end{equation}
	differentiating under integral sign
	\begin{equation}
	\frac{d J}{d \alpha}=\int_{x_{1}}^{x_{2}}\left(\frac{\partial f}{\partial y} \frac{\partial y}{\partial \alpha}+\frac{\partial f}{\partial \dot{y}} \frac{\partial \dot{y}}{\partial \alpha}\right) d x
	\end{equation}
	For the second term, integrating by parts\\
	\begin{align*}
		\int_{x_{1}}^{x_{2}} \frac{\partial f}{\partial \dot{y}} \frac{\partial \dot{y}}{\partial \alpha} d x&=\int_{x_{1}}^{x_{2}} \frac{\partial f}{\partial \dot{y}} \frac{\partial^{2} y}{\partial x \partial \alpha} d x\\
		&=\left.\frac{\partial f}{\partial \dot{y}} \frac{\partial y}{\partial \alpha}\right|_{x_{1}} ^{x_{2}}-\int_{x_{1}}^{x_{2}} \frac{d}{d x}\left(\frac{\partial f}{\partial \dot{y}}\right) \frac{\partial y}{\partial \alpha} d x
	\end{align*}
All the varied curves pass through $(x_1,y_1),(x_1,y_1)$ and hence the partial derivative of $y$ with respect to  $\alpha$ $x_1$ and $x_2$ must vanish.
\begin{equation}
\therefore \frac{d J}{d \alpha}=\int_{x_{1}}^{x_{2}}\left(\frac{\partial f}{\partial y}-\frac{d}{d x} \frac{\partial f}{\partial \dot{y}}\right) \frac{\partial y}{\partial \alpha} d x
\end{equation}
Therefore, the condition for stationary value
\begin{equation}
\left(\frac{dJ}{d\alpha} \right)_{\alpha=0} =\int_{x_{1}}^{x_{2}}\left(\frac{\partial f}{\partial y}-\frac{d}{d x} \frac{\partial f}{\partial \dot{y}}\right) \frac{\partial y}{\partial \alpha} d x=0\label{VP-eq09}
\end{equation}
Where $\left( \frac{\partial y}{\partial \alpha}\right) $ is a function of $x$ that is arbitrary except for continuity and end point conditions.
\begin{itemize}
	\item \textit{Fundemental lemma} of calculus of variation
	\begin{equation}
	\text{if} \int\limits_{x_1}^{x_2}M(x)\eta(x)dx=0
	\end{equation}
	\textit{For all arbitrary functions $\eta (x)$ continuous through the second derivative, then $M(x)$ must identically vanish in the interval $(x_1,x_2)$}
\end{itemize}
	For a particular parametric family of varied paths given by \ref{VP-eq04}
	$$\left(\frac{\partial y}{\partial\alpha} \right)_0 =\eta(x)\text{the arbitrary function}$$
	$\therefore$ from \ref{VP-eq09} using fundamental lemma, we get
	\begin{equation}
	\frac{\partial f}{dy}-\frac{d}{dx}\left( \frac{\partial f}{\partial\dot{y}}\right) =0\label{VP-11}
	\end{equation}
	\begin{itemize}
		\item The infinitesimal departure of the varied path from the correct path $y(x)$ at the point $x$ and thus corresponds to virtual displacement $\delta y$ is
		$$\left( \frac{\partial y}{\partial\alpha}\right)_0 d\alpha=\delta y$$
\item Similarly, infinitesimal variation of $y$ above the correct path is given as
		$$\left( \frac{\partial y}{\partial\alpha}\right)_0 d\alpha=\delta J$$
		$$\delta J=\int_{x_{1}}^{12}\left(\frac{\partial f}{\partial y}-\frac{d}{d x} \frac{\partial f}{\partial \dot{y}}\right)\delta y\ dx=0$$
	\end{itemize}
\textbf{Euler-Lagrange Differential Equation}\\
Let $f$ is a function of many independent variables $y_i$, and their relatives $\dot{y}_i$. Where $y_i$ and $\dot{y}_i$ are functions of parametric variable $x$.
\begin{equation}
\delta J=\delta \int_{1}^{2} f\left(y_{1}(x) ; y_{2}(x), \ldots, \dot{y}_{1}(x) ; \dot{y}_{2}(x), \cdots, x\right)_{d_{x}}\label{VP-12}
\end{equation}
\begin{align*}
y_{1}(x, \alpha)&=y_{1}(x, 0)+\alpha \eta_{1}(x),\\
y_{2}(x, \alpha)&=y_{2}(x, 0)+\alpha \eta_{2}(x)\\
\vdots\quad&\hspace{1cm}\vdots\hspace{1.2cm}\vdots
\end{align*}
By using fundamental lemma, the condition that $\delta J$ is zero requires coeffitient of $\delta y_i$ seperately vanish
\begin{equation}
\frac{\partial f}{\partial y_{i}}-\frac{d}{d x} \frac{\partial f}{\partial \dot{y}_{i}}=0\label{VP-13}
\end{equation}
This is the appropriate generalization of equation \ref{VP-11} to several variables and known as the Euler-Lagrange differential equation.\\\\
\textbf{Langrange's Equation}\\
For the integral in Hamilton principle
$$I=\int_{1}^{2} L\left(q_{i}, \dot{q}_{i}, t\right) d t$$
have same form as equation \ref{VP-12}, with transformation
$$x\rightarrow t$$
$$y_i\rightarrow q_i$$
$$f(y_i,\dot{y}_i,x)\rightarrow L(q_i,\dot{q}_i,t)$$
as we assumed $y_i$ variables are independent the corresponding $q_i$ generalized coordinates are independent which requires that the constraints be holonomic.\\
The Euler-Lagrange equation corresponding to the integral $I$ then become the Lagrange equations of motion
$$\frac{d}{d t} \frac{\partial L}{\partial \dot{q}_{i}}-\frac{\partial L}{\partial q_{i}}=0$$
Lagrange's equations follow Hamilton's principle for monogenic systems with holonomic constraints \\\\
\textbf{Lagrange Undetermined Multipliers}\\
used to solve systems with holonomic constraints as well as certain types of non-holonomic systems. \\
If there are $n$ variables and $m$ constraint equations $f\alpha$ of the form
$$f(r_1,r_2,r_3....t)=0$$
the extra virtual displacements are eliminated by the method of Lagrange undetermined multipliers.
$$I=\int_{1}^{2}\left(L+\sum_{\alpha=1}^{m} \lambda_{\alpha} f_{a}\right) d t$$
allow the $q_2$ and the $\lambda_\alpha$ to be vary independently to obtain $n+m$ equations.\\
variations of $\lambda_\alpha$ gives the $m$ constraint equations the variations of the $q-i$'s give
$$\delta I=\int_{1}^{2} d t\left(\sum_{i=1}^{n}\left(\frac{d}{d t} \frac{\partial L}{\partial \dot{q}_{i}}-\frac{\partial L}{\partial q_{i}}+\sum_{\alpha=1}^{m} \lambda_{a} \frac{\partial f_{a}}{\partial q_{i}}\right) \delta q_{i}\right)=0$$
The $\delta q_i$'s are not independent we choose the $\lambda_{\alpha}$'s so that $m$ of the $n$ equations are satisfied for arbitrary $\delta q_i$, and then choose variations of the $\delta q_i$ in the remaining $n-m$ equations independently.\\
Thus we obtain $m$ equations of the form
$$\frac{d}{d t} \frac{\partial L}{\partial \dot{q}_{k}}-\frac{\partial L}{\partial q_{k}}+\sum_{\alpha=1}^{m} \lambda_{a} \frac{\partial f_{a}}{\partial q_{k}}=0$$
$Q_k$ are generalized forces\\
$Q_k$ have the magnitute of the forces needed to produce the individual constraints.\\\\
\section{Generalized Momentum}
If $L$ is the Lagrangian of a system then the generalized momentum associated with the coordinate $q_j$ shall be defined as
$$P_j=\frac{\partial L}{\partial \dot{q}_j}$$
The terms canonical momentum and conjugate momentum are often also used for $P_j$\\
If $q_j$ is not cartisian coordinate $P_j$ does not necessarilu have the diamensios of a linear momentum. If there is a velocity dependent potential then even with a cartesian coordinate $q_j$ the associated generalized momentum will not be indentical with mechanical momentum.
\section{Cyclic coordinate and Generalized Momentum Conjugate }
If the Lagrangian of a system does not contain a given generalized coordinate $q_j$ explicitly (although it may contain the corresponding velocity $\dot{q}_j$ ) then the coordinate is said to be cyclic or ignorable.
\begin{align*}
\intertext{The Lagrange equation of motion}
\frac{d}{dt}\left( \frac{\partial L}{\partial \dot{q}_j}\right)-\frac{\partial L}{\partial q_j} &=0
\intertext{reduces for cyclic coordinate, to}
\frac{d}{dt}\left(\frac{\partial L}{\partial \dot{q}_j} \right) &=0
\intertext{or}
\frac{dP_j}{dt}&=0
\intertext{Which means}
P_j&=\text{constant}
\therefore& \text{The generalized momentum conjugate to a cyclic coordinate is conserved}
\end{align*}
\begin{itemize}
	\item This general rule for cyclic coordinates contain all the conservation theorems discussed before
	\item With proper restriction it should reduces to the conservation theorems.
	\item The consevation theorems ate closely related to the symmetry properties of the system.
\end{itemize}
\begin{note}
	If any system or any function representing a property of the system does not change under some operation carried on the system, the system is said to possess symmetry with respect to that operation
\end{note}
\section{Conservation of Linear Momentum}
Consider a generalized coordinate $q_j$ for which $dq_j$ represents a translation of the system as a swhole in some given direction.
\begin{align*}
\intertext{Then $q_j$ cannot appear in $T$ for velocities are not affected by a shift in the origin}
\therefore \frac{\partial T}{\partial q_j}=0
\intertext{assume a consevative system for which $V$ is not function of the velocities}
\therefore \text{Lagrange's equation of motion}\\
\frac{d}{dt}\left(\frac{\partial  T}{\partial \dot{q}_j} \right) \equiv\dot{P}_j=\frac{-\partial V}{\partial q_j}=Q_j
\intertext{Where $Q_j$ is the component of the total force along the direction of translation of $q_j$ and $P_j$ is the component of total linear momentum.}
\text{if}q_j\text{ is cyclic, then}\\
\frac{-\partial V}{\partial q_j}\equiv Q_j=0
\end{align*}
ie. conservation theorem if a geven component of the total applied force vanishes, the corresponding component of the linear momentum is conserved\\
\textit{If the system is invarient under translation along a given direction (homogeneily of space) the corresponding linear momentum is conserved}\\
Then the physical properties of a closed system are not affected by an arbitrary displacement\\\\
\section{Conservation of Angular Momentum}
If a cyclic coordinate $q_j$ is such that $dq_j$ corresponds to a rotation of the system of particles around some axis, then the consevation of it's conjugate momentum corresponds to consetvation of angular momentum.\\
$T$ cannot cantain $q_j$ for a rotation of the coordinate system cannot affect the magnitude of velocities.
$$\therefore \frac{\partial T}{\partial q_j}=0$$
$V$ is independent of $\dot{q}_j$ as before\\
If the rotation coordinate $q_j$ is cyclic then $Q_j$, which is the component of applied torque along $\hat{n}$ vanishes and the component of $L$ along $n$ is constant. ie angular momentum conservation.\\
\textit{If the generalized rotation coordinate is cyclic, (and conjugate angular momentum is conserved) the system is invarient under rotation (homogeneily of angle) ie properties of closed system will not change due to arbitrary rotation about the origin of the frame of reference.}
\section{Aspects of Lagrangian Formulation}
A system with $n$ degrees of freesom possess $x$ equations of motion of the form
$$\frac{d}{dt}\left( \frac{\partial L}{\partial \dot{q}_1}\right)-\frac{\partial L}{\partial q_1}=0 $$
\begin{itemize}
	\item Equations are of second order
	\item motion of the system is determined for all time only when $2n$ initial values are specified
	\item State of the system represented by a point in aa $n$ dimentional configuration space whose coordinaes are the $n$ generalized coordinates $q_j$
	\item The Lagrangian view point a system with $n$ independent degrees of freedom is a problem in $n$ independent variables $q_i(1), $ and  $\dot{q}_i$ appears only as a shorthand for the time derivative of $q_i$
	\item All $n$ coordinates must be independent
\end{itemize}
\newpage
\begin{abox}
	Practise set-01
\end{abox}
\begin{enumerate}
	\item A double pendulum consists of two point masses $m$ attached by strings of length $l$ as shown in the figure: The kinetic energy of the pendulum is
	{\exyear{NET/JRF(DEC-2011)}}
	\begin{figure}[H]
		\centering
		\includegraphics[height=4cm,width=4.5cm]{diagram-20210926(1)-crop}
	\end{figure}
	\begin{tasks}(1)
		\task[\textbf{A.}] $\frac{1}{2} m l^{2}\left[\dot{\theta}_{1}^{2}+\dot{\theta}_{2}^{2}\right]$
		\task[\textbf{B.}] $\frac{1}{2} m l^{2}\left[2 \dot{\theta}_{1}^{2}+\dot{\theta}_{2}^{2}+2 \dot{\theta}_{1} \dot{\theta}_{2} \cos \left(\theta_{1}-\theta_{2}\right)\right]$
		\task[\textbf{C.}] $\frac{1}{2} m l^{2}\left[\dot{\theta}_{1}^{2}+2 \dot{\theta}_{2}^{2}+2 \dot{\theta}_{1} \dot{\theta}_{2} \cos \left(\theta_{1}-\theta_{2}\right)\right]$
		\task[\textbf{D.}] $\frac{1}{2} m l^{2}\left[2 \dot{\theta}_{1}^{2}+\dot{\theta}_{2}^{2}+2 \dot{\theta}_{1} \dot{\theta}_{2} \cos \left(\theta_{1}+\theta_{2}\right)\right]$
	\end{tasks}
	\item A particle of mass $m$ moves inside a bowl. If the surface of the bowl is given by the equation $z=\frac{1}{2} a\left(x^{2}+y^{2}\right)$, where $a$ is a constant, the Lagrangian of the particle is
	{\exyear{NET/JRF(DEC-2011)}}
	\begin{tasks}(2)
		\task[\textbf{A.}] $\frac{1}{2} m\left(\dot{r}^{2}+r^{2} \dot{\phi}^{2}-g a r^{2}\right)$
		\task[\textbf{B.}] $\frac{1}{2} m\left[\left(1+a^{2} r^{2}\right) \dot{r}^{2}+r^{2} \dot{\phi}^{2}\right]$
		\task[\textbf{C.}]  $\frac{1}{2} m\left(\dot{r}^{2}+r^{2} \dot{\theta}^{2}+r^{2} \sin ^{2} \theta \dot{\phi}^{2}-g a r^{2}\right)$
		\task[\textbf{D.}]  $\frac{1}{2} m\left[\left(1+a^{2} r^{2}\right) \dot{r}^{2}+r^{2} \dot{\phi}^{2}-g a r^{2}\right]$
	\end{tasks}
	\item The Lagrangian of a particle of mass $m$ moving in one dimension is given by
	$$
	L=\frac{1}{2} m \dot{x}^{2}-b x
	$$
	where $b$ is a positive constant. The coordinate of the particle $x(t)$ at time $t$ is given by: (in following $c_{1}$ and $c_{2}$ are constants)
	{\exyear{NET/JRF(JUNE-2013)}}
	\begin{tasks}(2)
		\task[\textbf{A.}] $-\frac{b}{2 m} t^{2}+c_{1} t+c_{2}$
		\task[\textbf{B.}] $c_{1} t+c_{2}$
		\task[\textbf{C.}] $c_{1} \cos \left(\frac{b t}{m}\right)+c_{2} \sin \left(\frac{b t}{m}\right)$
		\task[\textbf{D.}] $c_{1} \cosh \left(\frac{b t}{m}\right)+c_{2} \sinh \left(\frac{b t}{m}\right)$
	\end{tasks}
	\item A particle moves in a potential $V=x^{2}+y^{2}+\frac{z^{2}}{2} .$ Which component(s) of the angular momentum is/are constant(s) of motion?
	{\exyear{NET/JRF(DEC-2013)}}
	\begin{tasks}(4)
		\task[\textbf{A.}] None
		\task[\textbf{B.}] $L_{x}, L_{y}$ and $L_{z}$
		\task[\textbf{C.}]  only $L_{x}$ and $L_{y}$
		\task[\textbf{D.}] only $L_{z}$
	\end{tasks}
	\item A pendulum consists of a ring of mass $M$ and radius $R$ suspended by a massless rigid rod of length $l$ attached to its rim. When the pendulum oscillates in the plane of the ring, the time period of oscillation is
	{\exyear{NET/JRF(DEC-2013)}}
	\begin{tasks}(2)
		\task[\textbf{A.}] $2 \pi \sqrt{\frac{l+R}{g}}$
		\task[\textbf{B.}] $\frac{2 \pi}{\sqrt{g}}\left(l^{2}+R^{2}\right)^{1 / 4}$
		\task[\textbf{C.}] $2 \pi \sqrt{\frac{2 R^{2}+2 R l+l^{2}}{g(R+l)}}$
		\task[\textbf{D.}] $\frac{2 \pi}{\sqrt{g}}\left(2 R^{2}+2 R l+l^{2}\right)^{1 / 4}$
	\end{tasks}
	\item Consider a particle of mass $m$ attached to two identical springs each of length $l$ and spring constant $k$ (see the figure). The equilibrium configuration is the one where the springs are unstretched. There are no other external forces on the system. If the particle is given a small displacement along the $x$-axis, which of the following describes the equation of motion for small oscillations?
	{\exyear{NET/JRF(DEC-2013)}}
	\begin{figure}[H]
		\centering
		\includegraphics[height=4.5cm,width=5cm]{diagram-20210926(18)-crop}
	\end{figure}
	\begin{tasks}(4)
		\task[\textbf{A.}] $m \ddot{x}+\frac{k x^{3}}{l^{2}}=0$
		\task[\textbf{B.}]  $m \ddot{x}+k x=0$
		\task[\textbf{C.}] $m \ddot{x}+2 k x=0$
		\task[\textbf{D.}] $m \ddot{x}+\frac{k x^{2}}{l}=0$
	\end{tasks}
	\item The equation of motion of a system described by the time-dependent Lagrangian
	$$
	L=e^{\gamma t}\left[\frac{1}{2} m \dot{x}^{2}-V(x)\right] \text { is }
	$$
	{\exyear{NET/JRF(DEC-2014)}}
	\begin{tasks}(2)
		\task[\textbf{A.}] $m \ddot{x}+\gamma m \dot{x}+\frac{d V}{d x}=0$
		\task[\textbf{B.}] $m \ddot{x}+\gamma m \dot{x}-\frac{d V}{d x}=0$
		\task[\textbf{C.}]  $m \ddot{x}-\gamma m \dot{x}+\frac{d V}{d x}=0$
		\task[\textbf{D.}] $m \ddot{x}+\frac{d V}{d x}=0$
	\end{tasks}
	\item A particle of unit mass moves in the $x y$-plane in such a way that $\dot{x}(t)=y(t)$ and $\dot{y}(t)=-x(t) .$ We can conclude that it is in a conservative force-field which can be derived from the potential
	{\exyear{NET/JRF(JUNE-2015)}}
	\begin{tasks}(4)
		\task[\textbf{A.}] $\frac{1}{2}\left(x^{2}+y^{2}\right)$
		\task[\textbf{B.}] $\frac{1}{2}\left(x^{2}-y^{2}\right)$
		\task[\textbf{C.}] $x+y$
		\task[\textbf{D.}] $x-y$
	\end{tasks}
	\item The Lagrangian of a particle moving in a plane s given in Cartesian coordinates as
	$$
	L=\dot{x} \dot{y}-x^{2}-y^{2}
	$$
	In polar coordinates the expression for the canonical momentum $p_{r}$ (conjugate to the radial coordinate $r$ ) is
	{\exyear{NET/JRF(DEC-2015)}}
	\begin{tasks}(2)
		\task[\textbf{A.}] $\dot{r} \sin \theta+r \dot{\theta} \cos \theta$
		\task[\textbf{B.}]  $\dot{r} \cos \theta+r \dot{\theta} \sin \theta$
		\task[\textbf{C.}] $2 \dot{r} \cos \theta-r \dot{\theta} \sin 2 \theta$
		\task[\textbf{D.}] $\dot{r} \sin 2 \theta+r \dot{\theta} \cos 2 \theta$
	\end{tasks}
	\item The dynamics of a particle governed by the Lagrangian
	$$
	L=\frac{1}{2} m \dot{x}^{2}-\frac{1}{2} k x^{2}-k x \dot{x} t \text { describes }
	$$
	{\exyear{NET/JRF(DEC-2016)}}
	\begin{tasks}(1)
		\task[\textbf{A.}] An undamped simple harmonic oscillator
		\task[\textbf{B.}] A damped harmonic oscillator with a time varying damping factor
		\task[\textbf{C.}]  An undamped harmonic oscillator with a time dependent frequency
		\task[\textbf{D.}] A free particle
	\end{tasks}
	\item The parabolic coordinates $(\xi, \eta)$ are related to the Cartesian coordinates $(x, y)$ by $x=\xi \eta$ and $y=\frac{1}{2}\left(\xi^{2}-\eta^{2}\right)$. The Lagrangian of a two-dimensional simple harmonic oscillator of mass $m$ and angular frequency $\omega$ is
	{\exyear{NET/JRF(DEC-2016)}}
	\begin{tasks}(2)
		\task[\textbf{A.}] $\frac{1}{2} m\left[\dot{\xi}^{2}+\dot{\eta}^{2}-\omega^{2}\left(\xi^{2}+\eta^{2}\right)\right]$
		\task[\textbf{B.}] $\frac{1}{2} m\left(\xi^{2}+\eta^{2}\right)\left[\left(\dot{\xi}^{2}+\dot{\eta}^{2}\right)-\frac{1}{4} \omega^{2}\left(\xi^{2}+\eta^{2}\right)\right]$
		\task[\textbf{C.}] $\frac{1}{2} m\left(\xi^{2}+\eta^{2}\right)\left[\dot{\xi}^{2}+\dot{\eta}^{2}-\frac{1}{2} \omega^{2} \xi \eta\right]$
		\task[\textbf{D.}] $\frac{1}{2} m\left(\xi^{2}+\eta^{2}\right)\left[\dot{\xi}^{2}+\dot{\eta}^{2}-\frac{1}{4} \omega^{2}\right]$
	\end{tasks}
	\item The spring constant $k$ of a spring of mass $m_{s}$ is determined experimentally by loading the spring with mass $M$ and recording the time period $T$, for a single oscillation. If the experiment is carried out for different masses, then the graph that correctly represents the result is
	{\exyear{NET/JRF(DEC-2017)}}
	\begin{tasks}(2)
		\task[\textbf{A.}] \begin{figure}[H]
			\centering
			\includegraphics[height=3cm,width=4.5cm]{diagram-20210926(38)-crop}
		\end{figure}
		\task[\textbf{B.}] \begin{figure}[H]
			\centering
			\includegraphics[height=3cm,width=4.5cm]{diagram-20210926(39)-crop}
		\end{figure}
		\task[\textbf{C.}] \begin{figure}[H]
			\centering
			\includegraphics[height=3cm,width=4.5cm]{diagram-20210926(40)-crop}
		\end{figure}
		\task[\textbf{D.}] \begin{figure}[H]
			\centering
			\includegraphics[height=3cm,width=4.5cm]{diagram-20210926(41)-crop}
		\end{figure}
	\end{tasks}
	\item The motion of a particle in one dimension is described by the Langrangian $L=\frac{1}{2}\left(\left(\frac{d x}{d t}\right)^{2}-x^{2}\right)$ in suitable units. The value of the action along the classical path from $x=0$ at $t=0$ to $x=x_{0}$ at $t=t_{0}$, is
	{\exyear{NET/JRF(DEC-2018)}}
	\begin{tasks}(4)
		\task[\textbf{A.}] $\frac{x_{0}^{2}}{2 \sin ^{2} t_{0}}$
		\task[\textbf{B.}] $\frac{1}{2} x_{0}^{2} \tan t_{0}$
		\task[\textbf{C.}] $\frac{1}{2} x_{0}^{2} \cot t_{0}$
		\task[\textbf{D.}] $\frac{x_{0}^{2}}{2 \cos ^{2} t_{0}}$
	\end{tasks}
	\item Two particles of masses $m_{1}$ and $m_{2}$ are connected by a massless thread of length $l$ as shown in figure below.\\
	The particle of mass in on the plane undergoes a circular motion with radius $r_{0}$ and angular momentum $L$. When a small radial displacement $\in$ (whew $\in \ll<r_{0}$ ) is applied, its radial coordinate is found to oscillate about $r_{0}$. The frequency of the oscillations is
	{\exyear{NET/JRF(JUNE-2019)}}
	\begin{tasks}(2)
		\task[\textbf{A.}] $\sqrt{\frac{7 m_{2} g}{\left(m_{1}+\frac{m_{2}}{2}\right) r_{0}}}$
		\task[\textbf{B.}] $\sqrt{\frac{7 m_{2} g}{\left(m_{1}+m_{2}\right) r_{0}}}$
		\task[\textbf{C.}] $\sqrt{\frac{3 m_{2} g}{\left(m_{1}+\frac{m_{2}}{2}\right) r_{0}}}$
		\task[\textbf{D.}] $\sqrt{\frac{3 m_{2} g}{\left(m_{1}+m_{2}\right) r_{0}}}$
	\end{tasks}
	\item Which of the following terms, when added to the Lagrangian $L(x, y, \dot{x}, \dot{y})$ of a system with two degrees of freedom will not change the equations of motion?\\
	(check question )
	{\exyear{NET/JRF(DEC-2019)}}
	\begin{tasks}(4)
		\task[\textbf{A.}] $x \ddot{x}-y \ddot{y}$
		\task[\textbf{B.}] $x \ddot{y}-y \ddot{x}$
		\task[\textbf{C.}] $x \dot{y}-y \dot{x}$
		\task[\textbf{D.}] $y \dot{x}^{2}+x \dot{y}^{2}$
	\end{tasks}
	\item A point mass $m$, is constrained to move on the inner surface of a paraboloid of revolution $x^{2}+y^{2}=a z$ (where $a>0$ is a constant). When it spirals down the surface, under the influence of gravity (along $-z$ direction), the angular speed about the $z$-axis is proportional to
	{\exyear{NET/JRF(JUNE-2020)}}
	\begin{tasks}(2)
		\task[\textbf{A.}] 1 (independent of $z$ )
		\task[\textbf{B.}] $z$
		\task[\textbf{C.}]  $z^{-1}$
		\task[\textbf{D.}] $z^{-2}$
	\end{tasks}
\end{enumerate}
\newpage
\begin{abox}
	Practise set-02
\end{abox}
\begin{enumerate}
	\item A particle of mass $m$ slides under the gravity without friction along the parabolic path $y=a x^{2}$, as shown in the figure. Here $a$ is a constant.\\
	\begin{figure}[H]
		\centering
		\includegraphics[height=4.5cm,width=5cm]{diagram-20210915(2)-crop}
	\end{figure}
	The Lagrangian for this particle is given by
	{	\exyear{GATE 2012}}
	\begin{tasks}(2)
		\task[\textbf{A.}] $L=\frac{1}{2} m \dot{x}^{2}-m g a x^{2}$
		\task[\textbf{B.}] $L=\frac{1}{2} m\left(1+4 a^{2} x^{2}\right) \dot{x}^{2}-m g a x^{2}$
		\task[\textbf{C.}] $L=\frac{1}{2} m \dot{x}^{2}+m g a x^{2}$
		\task[\textbf{D.}] $L=\frac{1}{2} m\left(1+4 a^{2} x^{2}\right) \dot{x}^{2}+m g a x^{2}$
	\end{tasks}
	\item  The Lagrange's equation of motion of the particle for above question is given by
	{\exyear{GATE 2012}}
	\begin{tasks}(2)
		\task[\textbf{A.}] $\ddot{x}=2 g a x$
		\task[\textbf{B.}] $m\left(1+4 a^{2} x^{2}\right) \ddot{x}=-2 m g a x-4 m a^{2} x \dot{x}^{2}$
		\task[\textbf{C.}] $m\left(1+4 a^{2} x^{2}\right) \ddot{x}=2 m g a x+4 m a^{2} x \dot{x}^{2}$
		\task[\textbf{D.}] $\ddot{x}=-2 g a x$
	\end{tasks}
	\begin{answer}
		\begin{align*}
		\frac{d}{d t}\left(\frac{d L}{d \dot{x}}\right)&=\frac{d L}{d x} \Rightarrow m \ddot{x}\left(1+4 a^{2} x^{2}\right)\\&=-4 m a^{2} x \dot{x}^{2}-2 m g a x
		\end{align*}
		So the correct answer is \textbf{Option (B)}
	\end{answer}
	\item The Lagrangian of a system with one degree of freedom $q$ is given by $L=\alpha \dot{q}^{2}+\beta q^{2}$, where $\alpha$ and $\beta$ are non-zero constants. If $p_{q}$ denotes the canonical momentum conjugate to $q$ then which one of the following statements is CORRECT?
	{\exyear{GATE 2013}}
	\begin{tasks}(1)
		\task[\textbf{A.}] $p_{q}=2 \beta q$ and it is a conserved quantity.
		\task[\textbf{B.}]  $p_{q}=2 \beta q$ and it is not a conserved quantity.
		\task[\textbf{C.}] $p_{q}=2 \alpha \dot{q}$ and it is a conserved quantity.
		\task[\textbf{D.}]  $p_{q}=2 \alpha \dot{q}$ and it is not a conserved quantity.
	\end{tasks}
	\item A bead of mass $m$ can slide without friction along a massless rod kept at $45^{\circ}$ with the vertical as shown in the figure. The rod is rotating about the vertical axis with a constant angular speed $\omega$. At any instant $r$ is the distance of the bead from the origin. The momentum conjugate to $r$ is
	{\exyear{GATE 2014}}
	\begin{figure}[H]
		\centering
		\includegraphics[height=5cm,width=7cm]{diagram-20210915(4)-crop}
	\end{figure}
	\begin{tasks}(4)
		\task[\textbf{A.}] $m \dot{r}$
		\task[\textbf{B.}] $\frac{1}{\sqrt{2}} m \dot{r}$
		\task[\textbf{C.}] $\frac{1}{2} m \dot{r}$
		\task[\textbf{D.}] $\sqrt{2} m \dot{r}$
	\end{tasks}
	\item The Lagrangian of a system is given by
	$L=\frac{1}{2} m l^{2}\left[\dot{\theta}^{2}+\sin ^{2} \theta \dot{\varphi}^{2}\right]-m g l \cos \theta$, where $m, l$ and $g$ are constants.
	Which of the following is conserved?
	{\exyear{GATE 2016}}
	\begin{tasks}(4)
		\task[\textbf{A.}] $\dot{\varphi} \sin ^{2} \theta$
		\task[\textbf{B.}] $\dot{\varphi} \sin \theta$
		\task[\textbf{C.}] $\frac{\dot{\varphi}}{\sin \theta}$
		\task[\textbf{D.}] $\frac{\dot{\varphi}}{\sin ^{2} \theta}$
	\end{tasks}
	\item If the Lagrangian $L_{0}=\frac{1}{2} m\left(\frac{d q}{d t}\right)^{2}-\frac{1}{2} m \omega^{2} q^{2}$ is modified to $L=L_{0}+\alpha q\left(\frac{d q}{d t}\right)$, which one of the following is TRUE?
	{\exyear{GATE 2017}}
	\begin{tasks}(1)
		\task[\textbf{A.}] Both the canonical momentum and equation of motion do not change
		\task[\textbf{B.}] Canonical momentum changes, equation of motion does not change
		\task[\textbf{C.}] Canonical momentum does not change, equation of motion changes
		\task[\textbf{D.}] Both the canonical momentum and equation of motion change
	\end{tasks}
	\item  A double pendulum consists of two equal masses $m$ suspended by two strings of length $l$. What is the Lagrangian of this system for oscillations in a plane? Assume the angles $\theta_{1}, \theta_{2}$ made by the two strings are small (you can use $\cos \theta=1-\theta^{2} / 2$ ).\\
	Note: $\omega_{0}=\sqrt{g / l}$.
	{\exyear{JEST 2014}}
	\begin{tasks}(1)
		\task[\textbf{A.}] $L \approx m l^{2}\left(\dot{\theta}_{1}^{2}+\frac{1}{2} \dot{\theta}_{2}^{2}-\omega_{0}^{2} \theta_{1}^{2}-\frac{1}{2} \omega_{0}^{2} \theta_{2}^{2}\right)$
		\task[\textbf{B.}]  $L \approx m l^{2}\left(\dot{\theta}_{1}^{2}+\frac{1}{2} \dot{\theta}_{2}^{2}+\dot{\theta}_{1} \dot{\theta}_{2}-\omega_{0}^{2} \theta_{1}^{2}-\frac{1}{2} \omega_{0}^{2} \theta_{2}^{2}\right)$
		\task[\textbf{C.}] $L \approx m l^{2}\left(\dot{\theta}_{1}^{2}+\frac{1}{2} \dot{\theta}_{2}^{2}-\dot{\theta}_{1} \dot{\theta}_{2}-\omega_{0}^{2} \theta_{1}^{2}-\frac{1}{2} \omega_{0}^{2} \theta_{2}^{2}\right)$
		\task[\textbf{D.}]  $L \approx m l^{2}\left(\frac{1}{2} \dot{\theta}_{1}^{2}+\frac{1}{2} \dot{\theta}_{2}^{2}+\dot{\theta}_{1} \dot{\theta}_{2}-\omega_{0}^{2} \theta_{1}^{2}-\omega_{0}^{2} \theta_{2}^{2}\right)$
	\end{tasks}
	\item A bike stuntman rides inside a well of frictionless surface given by $z=a\left(x^{2}+y^{2}\right)$, under the action of gravity acting in the negative $z$ direction. $\vec{g}=-g \hat{z} .$ What speed should be maintain to be able to ride at a constant height $z_{0}$ without falling down?
	{\exyear{JEST 2015}}
	\begin{tasks}(1)
		\task[\textbf{A.}] $\sqrt{g z_{0}}$
		\task[\textbf{B.}] $\sqrt{3 g z_{0}}$
		\task[\textbf{C.}] $\sqrt{2 g z_{0}}$
		\task[\textbf{D.}] The biker will not be able to maintain a constant height, irrespective of speed.
	\end{tasks}
	\item The Lagrangian of a particle is given by $L=\dot{q}^{2}-q \dot{q}$. Which of the following statements is true?
	{\exyear{JEST 2015}}
	\begin{tasks}(1)
		\task[\textbf{A.}]  This is a free particle
		\task[\textbf{B.}] The particle is experiencing velocity dependent damping
		\task[\textbf{C.}] The particle is executing simple harmonic motion
		\task[\textbf{D.}] The particle is under constant acceleration.
	\end{tasks}
	\begin{answer}
		\begin{align*}
		\because L&=\dot{q}^{2}-q \dot{q} \Rightarrow \frac{\partial L}{\partial \dot{q}}\\&=2 \dot{q}-q \Rightarrow \frac{d}{d t}\left(\frac{\partial L}{\partial \dot{q}}\right)\\&=2 \ddot{q}-\dot{q}\\
		\because \frac{d}{d t}\left(\frac{\partial L}{\partial \dot{q}}\right)-\frac{\partial L}{\partial q}&=0
		\Rightarrow 2 \ddot{q}-\dot{q}+\dot{q}\\&=0 \Rightarrow 2 \ddot{q}=0 \Rightarrow \frac{d^{2} q}{d t^{2}}\\&=0 \Rightarrow \frac{d q}{d t}=C \Rightarrow q=C t+\alpha
		\end{align*}
		So the correct answer is \textbf{Option (A)}
	\end{answer}
	\item A hoop of radius a rotates with constant angular velocity $\omega$ about the
	vertical axis as shown in the figure. A bead of mass $m$ can slide on the
	hoop without friction. If $g<\omega^{2} a$ at what angle $\theta$ apart from 0 and $\pi$ is the bead stationary (i.e., $\frac{d \theta}{d t}=\frac{d^{2} \theta}{d t^{2}}=0$ )?
	{\exyear{JEST 2016}}
	\begin{figure}[H]
		\centering
		\includegraphics[height=4.2cm,width=3.5cm]{diagram-20210809(11)-crop}
		\caption{}
		\label{}
	\end{figure}
	\begin{tasks}(4)
		\task[\textbf{A.}] $\tan \theta=\frac{\pi g}{\omega^{2} a}$
		\task[\textbf{B.}] $\sin \theta=\frac{g}{\omega^{2} a}$
		\task[\textbf{C.}] $\cos \theta=\frac{g}{\omega^{2} a}$
		\task[\textbf{D.}] $\tan \theta=\frac{g}{\pi \omega^{2} a}$
	\end{tasks}
	\item A bead of mass $M$ slides along a parabolic wire described by $z=2\left(x^{2}+y^{2}\right)$. The wire rotates with angular velocity $\Omega$ about the $z$ - axis. At what value of $\Omega$ does the bead maintain a constant nonzero height under the action of gravity along $-\hat{z}$ ?
	{\exyear{JEST 2017}}
	\begin{tasks}(4)
		\task[\textbf{A.}] $\sqrt{3 g}$
		\task[\textbf{B.}] $\sqrt{g}$
		\task[\textbf{C.}] $\sqrt{2 g}$
		\task[\textbf{D.}] $\sqrt{4 g}$
	\end{tasks}
	\item A possible Lagrangian for a free particle is
	{\exyear{JEST 2017}}
	\begin{tasks}(4)
		\task[\textbf{A.}] $L=\dot{q}^{2}-q^{2}$
		\task[\textbf{B.}] $L=\dot{q}^{2}-q \dot{q}$
		\task[\textbf{C.}] $L=\dot{q}^{2}-q$
		\task[\textbf{D.}] $L=\dot{q}^{2}-\frac{1}{q}$
	\end{tasks}
	\begin{answer}
		\begin{align*}
		\frac{d}{d t}\left(\frac{\partial L}{\partial \dot{q}}\right)-\left(\frac{\partial L}{\partial q}\right)&=0 \Rightarrow 2 \ddot{q}-\dot{q}+\dot{q}\\&=0 \Rightarrow \ddot{q}=0
		\end{align*}
		So the correct answer is \textbf{Option (B)}
	\end{answer}
	\item  A rod of mass $m$ and length $l$ is suspended from two massless vertical springs with a spring constants $k_{1}$ and $k_{2} .$ What is the Lagrangian for the system, if $x_{1}$ and $x_{2}$ be the displacements from equilibrium position of the two ends of the rod?
	{\exyear{JEST 2017}}
	\begin{tasks}(2)
		\task[\textbf{A.}] $\frac{m}{8}\left(\dot{x}_{1}^{2}+2 \dot{x}_{1} \dot{x}_{2}+\dot{x}_{2}^{2}\right)-\frac{1}{2} k_{1} x_{1}^{2}-\frac{1}{2} k_{2} x_{2}^{2}$
		\task[\textbf{B.}] $\frac{m}{2}\left(\dot{x}_{1}^{2}+\dot{x}_{1} \dot{x}_{2}+\dot{x}_{2}^{2}\right)-\frac{1}{4}\left(k_{1}+k_{2}\right)\left(x_{1}^{2}+x_{2}^{2}\right)$
		\task[\textbf{C.}] $\frac{m}{6}\left(\dot{x}_{1}^{2}+x_{1} \dot{x}_{2}+\dot{x}_{2}^{2}\right)-\frac{1}{2} k_{1} x_{1}^{2}-\frac{1}{2} k_{2} x_{2}^{2}$
		\task[\textbf{D.}] $\frac{m}{2}\left(\dot{x}_{1}^{2}-2 \dot{x}_{1} \dot{x}_{2}+\dot{x}_{2}^{2}\right)-\frac{1}{4}\left(k_{1}-k_{2}\right)\left(x_{1}^{2}+x_{2}^{2}\right)$
	\end{tasks}
	\item Consider the Lagrangian
	$$L=1-\sqrt{1-\dot{q}^{2}}-\frac{q^{2}}{2}$$
	of a particle executing oscillations whose amplitude is $A$. If $p$ denotes the momentum of the particle, then $4 p^{2}$ is
	{\exyear{JEST 2015}}
	\begin{tasks}(2)
		\task[\textbf{A.}] (a) $\left(A^{2}-q^{2}\right)\left(4+A^{2}-q^{2}\right)$
		\task[\textbf{B.}] $\left(A^{2}+q^{2}\right)\left(4+A^{2}-q^{2}\right)$
		\task[\textbf{C.}] $\left(A^{2}-q^{2}\right)\left(4+A^{2}+q^{2}\right)$
		\task[\textbf{D.}] $\left(A^{2}+q^{2}\right)\left(4+A^{2}+q^{2}\right)$
	\end{tasks}
	\begin{answer}
		So the correct answer is \textbf{Option (A)}
	\end{answer}
	\item Consider the motion of a particle in two dimensions given by the Lagrangian $$L=\frac{m}{2}\left(\dot{x}^{2}+\dot{y}^{2}\right)-\frac{\lambda}{4}(x+y)^{2}$$
	where $\lambda>0$. The initial conditions are given as $y(0)=0, x(0)=42$ meters, $\dot{x}(0)=\dot{y}(0)=0$. What is the value of $x(t)-y(t)$ at $t=25$ seconds in meters?
	{\exyear{JEST 2019}}
\end{enumerate}
\newpage
\begin{abox}
	Practise set-03
\end{abox}
\begin{enumerate}
	\item A particle of mass $m$ moves inside a bowl under gravity. If the surface of the bowl is given by the equation $z=\frac{1}{2} a\left(x^{2}+y^{2}\right)$, where $a$ is a constant.\\
		(a) Write down Lagrangian of the system in cylindrical coordinate.\\
		(b) Identify the cyclic coordinate and law of conservation of momentum.\\
		(c) Write down the equation of motion
	\begin{answer}
		\begin{align*}
		\text{There is only one particle, so }N&=1\\
		\text{Cartesian coordinate kinetic energy is }T&=\frac{1}{2} m\left(\dot{x}^{2}+\dot{y}^{2}+\dot{z}^{2}\right)\text{ and potential energy is} V=m g z
		\intertext{Equation of Constraint is -}
		\intertext{1. Particle is constrained to move on surface of a bowl, so the equation of constraint is}
		z&=\frac{1}{2} a\left(x^{2}+y^{2}\right) \text {, so } k=1\\
		\text{Degree of freedom }( DOF) &=3 N-k=3 \times 1-1=2
		\intertext{Hence, there are two degree of freedom, so we need two generalized coordinates.}
		\intertext{Now transforming the Cartesian coordinate to cylindrical coordinate, $x=r \cos \theta, y=r \sin \theta$, z=z}
		\intertext{So, the kinetic energy is $T=\frac{1}{2} m\left(\dot{r}^{2}+r^{2} \dot{\theta}^{2}+\dot{z}^{2}\right)$. And the equation of constraint in cylindrical coordinate $\left(z=\frac{1}{2} a r^{2}\right)$ is written as $\dot{z}=a r \dot{r}$}
		\intertext{So, kinetic energy for two degree of freedom with suitable coordinate is given by, $T=\frac{m}{2}\left(\dot{r}^{2}+r^{2} \dot{\theta}^{2}+a^{2} r^{2} \dot{r}^{2}\right)$ and potential energy $V=\frac{m g a r^{2}}{2}$}
		T=\frac{m}{2}\left(\dot{r}^{2}+r^{2} \dot{\theta}^{2}+a^{2} r^{2} \dot{r}^{2}\right)&\text{ and potential energy }V=\frac{m g a r^{2}}{2}\\
		\text{So, Lagrangian is }L&=\frac{m}{2}\left[\dot{r}^{2}\left(1+a^{2} r^{2}\right)+r^{2} \dot{\theta}^{2}-a g r^{2}\right]
		\intertext{(b) Hence, $\frac{\partial L}{\partial \theta}=0$ so, $\theta$ is cyclic coordinate hence $p_{\theta}=\frac{\partial L}{\partial \dot{\theta}}=m r^{2} \dot{\theta}$ is a constant of motion, which is identified as angular momentum of the system. So angular momentum of the system is conserved.}
		\text{	(c) If Lagrangian of the system is } L&=\frac{m}{2}\left[\dot{r}^{2}\left(1+a^{2} r^{2}\right)+r^{2} \dot{\theta}^{2}-a g r^{2}\right]\text{ then}\\
		\text{Equation of Motion is }&\frac{d}{d t}\left(\frac{\partial L}{\partial \dot{r}}\right)-\frac{\partial L}{\partial r}=0\\
		\left(\frac{\partial L}{\partial r}\right)&=m \dot{r}^{2} a^{2} r+m r \dot{\theta}^{2}-m a g r\text{ and }\left(\frac{\partial L}{\partial \dot{r}}\right)=m \dot{r}\left(1+a^{2} r^{2}\right)\\
		\Rightarrow \frac{d}{d t}\left(\frac{\partial L}{\partial \dot{r}}\right)&=m \ddot{r}\left(1+a^{2} r^{2}\right)+m \dot{r} a^{2} 2 r \dot{r}=m \ddot{r}\left(1+a^{2} r^{2}\right)+2 m a^{2} r \dot{r}^{2}\\
		\frac{d}{d t}\left(\frac{\partial L}{\partial \dot{r}}\right)-\frac{\partial L}{\partial r}&=0 \Rightarrow m \ddot{r}\left(1+a^{2} r^{2}\right)+2 m a^{2} r \dot{r}^{2}-m a^{2} r \dot{r}^{2}-m r \dot{\theta}^{2}+m a g r=0\\
		\Rightarrow m \ddot{r}\left(1+a^{2} r^{2}\right)&+m a^{2} r^{2}-m r \dot{\theta}^{2}+m a g r=0
		\end{align*}
	\end{answer}
	\item A mass $m$ is attached to one end of spring with spring constant $k$ and natural length $l$. The other end is pivoted at point $O$ as shown in figure. If the particle is constrained to move in $x-y$ plane under gravity (in $y$ direction)\\
		(a) Write down the expression for kinetic energy and potential energy.\\
		(b) Write down Lagarngian in Cartesian coordinate system.\\
		(c) Transform the Lagrangian in suitable coordinate system.\\
		\begin{figure}[H]
			\centering
			\includegraphics[height=3.8cm,width=5.5cm]{VP-01}
		\end{figure}
	\begin{answer}
		\begin{align*}
		\intertext{ (a) Number of particle $N=1$, kinetic energy is given by $T=\frac{1}{2} m\left(\dot{x}^{2}+\dot{y}^{2}+\dot{z}^{2}\right)$, potential energy is given by $V=-m g y .$}
		\intertext{Equation of Constraint is -}
		\text{The particle is constrained to move in }&x-y\text{ plane so equation of constraint }z=0 \Rightarrow k=1\\
		\text{So, degree of freedom }( \text{ DOF } )&=3 N-K=3 \times 1-1=2
		\intertext{If degree of freedom is two then there must be two independent motion.}
		z&=0 \Rightarrow \dot{z}=0
		\intertext{So, $T=\frac{1}{2} m\left(\dot{x}^{2}+\dot{y}^{2}\right)$ and $V=-m g y+\frac{1}{2} k\left[\sqrt{\left(x^{2}+y^{2}\right)}-l\right]^{2}$ (potential energy due to gravity and stored energy due to spring).}
		\intertext{Lagrangian in the Cartesian coordinate is $L=\frac{1}{2} m\left(\dot{x}^{2}+\dot{y}^{2}\right)-\left(-m g y+\frac{1}{2} k\left[\sqrt{\left(x^{2}+y^{2}\right)}-l\right]^{2}\right)$,
			$$
			L=\frac{1}{2} m\left(\dot{x}^{2}+\dot{y}^{2}\right)+m g y-\frac{1}{2} k\left[\sqrt{\left(x^{2}+y^{2}\right)}-l\right]^{2}
			$$}
		\end{align*}
		(b) These two independent motions are in (i) radial direction and (ii) angular direction in plane so best coordinate system to define the system is circular polar coordinate.\\
		But one should express the Lagrangian into suitable coordinate which must be independent of radial and angular variable.
		\begin{align*}
		\text{Put }x&=r \sin \theta, y=r \cos \theta,\text{ so, Lagrangian is given by -}\\
		L&=\frac{1}{2} m\left(\dot{r}^{2}+r^{2} \dot{\theta}^{2}\right)+m g r \cos \theta-\frac{1}{2} k(r-l)^{2}\\
		\text{The equation of }&\text{motion in terms of variable $r$ is }\frac{d}{d t}\left(\frac{\partial L}{\partial \dot{r}}\right)-\frac{\partial L}{\partial r}=0\\
		&\left(\frac{\partial L}{\partial \dot{r}}\right)=m \dot{r} \Rightarrow \frac{d}{d t}\left(\frac{\partial L}{\partial \dot{r}}\right)=m \ddot{r} \text { and }\left(\frac{\partial L}{\partial r}\right)=m r \dot{\theta}^{2}+m g \cos \theta-k(r-l) \\
		&\frac{d}{d t}\left(\frac{\partial L}{\partial \dot{r}}\right)-\frac{\partial L}{\partial r}=0 \Rightarrow m \ddot{r}-m r \dot{\theta}^{2}-m g \cos \theta+k(r-l)=0
		\intertext{The equation of motion in terms of variable $\theta$}
		\left(\frac{\partial L}{\partial \dot{\theta}}\right)&=m r^{2} \dot{\theta}\left(\frac{\partial L}{\partial \theta}\right)=-m g r \sin \theta\\
		\frac{d}{d t}\left(\frac{\partial L}{\partial \dot{\theta}}\right)&=m r^{2} \ddot{\theta}+2 m r \dot{r} \dot{\theta}\\
		\frac{d}{d t}\left(\frac{\partial L}{\partial \dot{\theta}}\right)-\frac{\partial L}{\partial \theta}&=0 \Rightarrow m r^{2} \ddot{\theta}+2 m r \dot{r} \dot{\theta}+m g r \sin \theta=0
		\end{align*}
	\end{answer}
	\item A spherical pendulum of length $a$ moves in a space about fixed point $O$.\\
		(a) Write down Lagrangian of the system.\\
		(b) Identify cyclic Coordinate\\
		(c) Solve Lagrangian equation of motion.\\
		\begin{figure}[H]
			\centering
			\includegraphics[height=5cm,width=6cm]{VP-02}
		\end{figure}
	\begin{answer}
		\begin{align*}
		\text{Number of particle }N&=1\text{ so }T=\frac{1}{2} m\left(\dot{x}^{2}+\dot{y}^{2}+\dot{z}^{2}\right)\text{ and potential}\\
		\text{energy of system is }V&=-m g z
		\intertext{Equation of Constraint is -}
		\text{(1) The length of pendulum is fixed }&\sqrt{x^{2}+y^{2}+z^{2}}=a\text{ so }K=1\\
		\text{So, degree of freedom }&(\mathrm{DOF})=3 . N-K=3.1-1=2
		\intertext{So there is a need of two generalized coordinates to solve the equation of motion. There are two independent motion (i) angular motion of particle from $z$-axis (ii) angular motion in $x-y$ plane about $z$ axis.}
		\intertext{So, suitable coordinate is spherical coordinate.}
		x&=r \sin \theta \cos \phi, y=r \sin \theta \sin \phi, z=r \cos \theta\\
		\text{	From equation }&\text{of constraint, }r=a \Rightarrow \dot{r}=0\\
		\text{Kinetic energy }T&=\frac{1}{2} m\left(a^{2} \dot{\theta}^{2}+a^{2} \sin ^{2} \theta \dot{\phi}^{2}\right), \text{potential energy }V=-m g a \cos \theta\\
		L=T-V&=\frac{1}{2} m\left(a^{2} \dot{\theta}^{2}+a^{2} \sin ^{2} \theta \dot{\phi}^{2}\right)+m g a \cos \theta\\
		\text{(b) }\frac{\partial L}{\partial \phi}=0\text{ so }\phi\text{ is cyclic coordinate }\frac{\partial L}{\partial \dot{\phi}}&=p_{\phi}=m a^{2} \sin ^{2} \theta \dot{\phi}\text{ is constant of motion.}\\
		\text{(c) }\frac{d}{d t}\left(\frac{\partial L}{\partial \dot{\theta}}\right)-\left(\frac{\partial L}{\partial \theta}\right)&=0\\
		\left(\frac{\partial L}{\partial \dot{\theta}}\right)=m a^{2} \dot{\theta} \Rightarrow \frac{d}{d t}\left(\frac{\partial L}{\partial \dot{\theta}}\right)&=m a^{2} \ddot{\theta} \Rightarrow\left(\frac{\partial L}{\partial \theta}\right)=m a^{2} \sin \theta \cos \theta \dot{\phi}^{2}-m g a \sin \theta\\
		\text{So, }\frac{d}{d t}\left(\frac{\partial L}{\partial \dot{\theta}}\right)-\left(\frac{\partial L}{\partial \theta}\right)&=0 \Rightarrow m a^{2} \ddot{\theta}-m a^{2} \sin \theta \cos \theta \dot{\phi}^{2}+m g a \sin \theta=0
		\end{align*}
	\end{answer}
	\item A particle of mass $m$ is constrained to move under gravity in inner surface of cone of half angle $\alpha$ as shown in figure.\\
		(a) Write down Lagrangian in suitable coordinate system\\
		(b) Identify cyclic coordinate and discuss conservation of momentum\\
		(c) Write down Lagrange's equation of motion\\
		\begin{figure}[H]
			\centering
			\includegraphics[height=3.5cm,width=5cm]{VP-03}
		\end{figure}
	\begin{answer}$\left. \right. $\\
		\begin{figure}[H]
			\centering
			\includegraphics[height=4cm,width=5cm]{VP-04}
		\end{figure}
		\begin{align*}
		\intertext{ A particle of mass $m$ is constrained to move in inner surface of cone of half angle $\alpha$ and $x, y, z$ are the coordinates of particle at any time $t$.}
		\text{(a) Number of particle $N=1$ so kinetic energy of the system is } T&=\frac{1}{2} m\left(\dot{x}^{2}+\dot{y}^{2}+\dot{z}^{2}\right)
		\intertext{	Assuming that the origin is at the vertex of the cone, the potential energy is given by $V=m g z$}
		\intertext{ Equation of Constraint is -}
		\text{	Particle is constraint to move on surface of cone where }&\tan \alpha=\frac{\sqrt{x^{2}+y^{2}}}{z},\text{ so }K=1.\\
		\text{So, degree of freedom (DOF) }&=3 N-K=3 \times 1-1=2
		\intertext{So, there is a need of two generalized Coordinates to write the Lagrangian.}
		\intertext{The two independent motion is (i) Linear motion along the slope and (ii) angular motion about $z$ axis. So, spherical symmetry is suitable coordinate.}
		\text{Put the value of }x=r \sin \theta \cos \phi, y&=r \sin \theta \sin \phi, z=r \cos \theta\\
		\text{Which gives }T=\frac{1}{2} m\left(\dot{r}^{2}+r^{2} \dot{\theta}^{2}+r^{2} \sin ^{2} \theta \dot{\phi}^{2}\right)\text{ and }V&=m g r \cos \theta\\
		\text{As }\tan \alpha=\frac{\sqrt{x^{2}+y^{2}}}{z} \Rightarrow \tan \alpha=\tan \theta \Rightarrow \theta&=\alpha \Rightarrow \dot{\theta}=0
		\intertext{If Lagrangian of the system is given by $L=\frac{1}{2} m\left(\dot{r}^{2}+r^{2} \sin ^{2} \alpha \dot{\phi}^{2}\right)-m g r \cos \alpha$ there is two generalized coordinates i.e., $r, \phi$}
		\text{(b) }L=\frac{1}{2} m\left(\dot{r}^{2}+r^{2} \sin ^{2} \alpha \dot{\phi}^{2}\right)-m g r \cos \alpha&
		\intertext{One can see Lagrangian is not explicitly dependent on $\phi$, which gives $\frac{\partial L}{\partial \phi}=0$. So, $\phi$ is cyclic coordinate.}
		\intertext{$\left(\frac{\partial L}{\partial \dot{\phi}}\right)=p_{\phi} \Rightarrow m r^{2} \sin ^{2} \alpha \dot{\phi}=c$ and $p_{\phi}=c$ so angular momentum of the system is constant during the motion.}
		\intertext{(c) The Lagrangian equation of motion in $r$ variable is given by $\frac{d}{d t}\left(\frac{\partial L}{\partial \dot{r}}\right)-\frac{\partial L}{\partial r}=0$, which gives
			$\left(\frac{\partial L}{\partial \dot{r}}\right)=m \dot{r},\left(\frac{\partial L}{\partial r}\right)=m r \sin ^{2} \alpha \dot{\phi}^{2}-m g \cos \alpha$}
		\intertext{$\frac{d}{d t}\left(\frac{\partial L}{\partial \dot{r}}\right)-\frac{\partial L}{\partial r}=0 \Rightarrow m \ddot{r}-m r \sin ^{2} \alpha \dot{\phi}^{2}+m g \cos \alpha=0$, which is equivalent to Newton's law of motion}
		\end{align*}
	\end{answer}
		\item The system is shown in the figure. The particle $m_{2}$ moves on a vertical axis and the whole system rotates about this axis with a constant angular velocity $\omega$.\\
		(a) Write down Lagrangian of the system in spherical polar co-ordinate.\\
		(b) Discuss conservation of momentum\\
		(c) Write down equation of motion.\\
		\begin{figure}[H]
			\centering
			\includegraphics[height=4.5cm,width=4.3cm]{VP-05}
		\end{figure}
	\begin{answer}$\left. \right. $\\
		\begin{figure}[H]
			\centering
			\includegraphics[height=4.5cm,width=6.5cm]{VP-06}
		\end{figure}
		\begin{align*}
		\intertext{	(a) Number of Particle,$ N=3$}
		\intertext{The kinetic energy is given by}
		T&=\frac{1}{2} m_{1}\left(\dot{x}_{1}^{2}+\dot{y}_{1}^{2}+\dot{z}_{1}^{2}\right)+\frac{1}{2} m_{2}\left(\dot{x}_{2}^{2}+\dot{y}_{2}^{2}+\dot{z}_{2}^{2}\right)+\frac{1}{2} m_{3}\left(\dot{x}_{3}^{2}+\dot{y}_{3}^{2}+\dot{z}_{3}^{2}\right)
		\intertext{Potential energy assuming $O$ as origin}
		V&=-m_{1} g z_{1}-m_{1} g z_{2}-m_{2} g z_{3}\\
		x_{1}&=r_{1} \sin \theta_{1} \cos \phi_{1}, y_{1}=r_{1} \sin \theta_{1} \sin \phi_{1}, z_{1}=r_{1} \cos \theta_{1}\\
		x_{2}&=r_{2} \sin \theta_{2} \cos \phi_{2}, y_{2}=r_{2} \sin \theta_{2} \sin \phi_{2}, z_{2}=r_{2} \cos \theta_{2}\\
		x_{3}&=r_{3} \sin \theta_{3} \cos \phi_{3}, y_{3}=r_{3} \sin \theta_{3} \sin \phi_{3}, z_{3}=r_{3} \cos \theta_{3}
		\intertext{The kinetic energy in spherical coordinate is given by}
		T&=\frac{1}{2} m_{1}\left(\dot{r}_{1}^{2}+r_{1}^{2} \dot{\theta}_{1}^{2}+r_{1}^{2} \sin ^{2} \theta_{1}^{2} \dot{\phi}_{1}^{2}\right)+\frac{1}{2} m_{2}\left(\dot{r}_{2}^{2}+r_{2}^{2} \dot{\theta}_{2}^{2}+r_{2}^{2} \sin ^{2} \theta_{2}^{2} \dot{\phi}_{2}^{2}\right)\\
		&+\frac{1}{2} m_{3}\left(\dot{r}_{3}^{2}+r_{3}^{2} \dot{\theta}_{3}^{2}+r_{3}^{2} \sin ^{2} \theta_{3}^{2} \dot{\phi}_{3}^{2}\right)\\
		\text{And potential }&\text{energy is given by }V=-m_{1} g r_{1} \cos \theta_{1}-m_{2} g r_{2} \cos \theta_{2}-m_{3} g r_{3} \cos \theta_{3}
		\intertext{Equations of Constraint are -}
		\end{align*}
		(1) Length of mass $m_{1}$ from origin $O$ is fixed, $r_{1}=l$\\
		(2) Length of another mass $m_{2}$ from origin $O$ is fixed, $r_{2}=l$\\
		(3) Both masses $m_{1}$ and $m_{2}$ make same angle with $z$ - axis, $\theta_{1}=\theta_{2}=\theta$\\
		(4) Both masses $m_{1}$ make angle in the $x-y$ plane with $x$ axis as $\phi_{1}=\phi_{2}+c$,\\
		(5) Particle $m_{2}$ makes angle zero with $z$ - axis i.e., $\theta_{3}=0$\\
		(6) Particle $m_{2}$ makes angle zero in $x-y$ plane with $x$ axis i.e., $\phi_{3}=0$,\\
		(7) Particle $m_{2}$ have distance $r_{3}=2 l \cos \theta$ from origin $O$.\\
		So, $K=7$. So degree of freedom (DOF) $=3 N-K=3.3-7=2$
		There is two independent motion (i) linear motion of $m_{2}$ on $z$ axis and (ii) both mass $m_{1}$ rotating about $z$ axis.\\
		So there is a need of two generalized coordinates.\\
		Using equation of constraint one will transform Lagrangian as $L=T-V$
		\begin{align*}
		L&=m_{1} l^{2}\left(\dot{\theta}^{2}+\dot{\phi}^{2} \sin ^{2} \theta\right)+2 m_{2} l^{2} \dot{\theta}^{2} \sin ^{2} \theta+2\left(m_{1}+m_{2}\right) g l \cos \theta\\
		\text{	Put the value of }\dot{\phi}&=\omega,\text{ then Lagrangian is given by}\\
		L&=m_{1} l^{2}\left(\dot{\theta}^{2}+\omega^{2} \sin ^{2} \theta\right)+2 m_{2} l^{2} \dot{\theta}^{2} \sin ^{2} \theta+2\left(m_{1}+m_{2}\right) g l \cos \theta\\
		\text{(b) }\left(\frac{\partial L}{\partial \phi}\right)&=0\text{ so $\phi$ is cyclic coordinate hence conjugate momentum}\\ \left(\frac{\partial L}{\partial \dot{\phi}}\right)&=\left(\frac{\partial L}{\partial \omega}\right)=p_{\phi}=2 m_{1} l^{2} \omega \sin ^{2} \theta
		\intertext{is constant during the motion. So, $z$ - component of angular momentum is conserved.}
		\text{(c) }&\frac{d}{d t}\left(\frac{\partial L}{\partial \dot{\theta}}\right)-\frac{\partial L}{\partial \theta}=0\\
		&\left(\frac{\partial L}{\partial \dot{\theta}}\right)=\left(2 m_{1} l^{2} \dot{\theta}+4 m_{2} l^{2} \dot{\theta} \sin ^{2} \theta\right) \\
		&\frac{d}{d t}\left(\frac{\partial L}{\partial \dot{\theta}}\right)=\left(2 m_{1} l^{2}+4 m_{2} l^{2} \sin ^{2} \theta\right) \ddot{\theta}+4 m_{2} l^{2} \sin 2 \theta \dot{\theta}^{2} \\
		&\left(\frac{\partial L}{\partial \theta}\right)=m_{1} l^{2} \omega^{2} 2 \sin \theta \cos \theta+2 m_{2} l^{2} \dot{\theta}^{2} 2 \sin \theta \cos \theta-2\left(m_{1}+m_{2}\right) g l \sin \theta
		\intertext{The equation of motion is given as -}
		\frac{d}{d t}\left(\frac{\partial L}{\partial \dot{\theta}}\right)&-\frac{\partial L}{\partial \theta}=0\\
		\left(2 m_{1} l^{2}+4 m_{2} l^{2} \sin ^{2} \theta\right) \ddot{\theta}&+2 m_{2} l^{2} \sin 2 \theta \dot{\theta}^{2}-m_{1} l^{2} \omega^{2} \sin 2 \theta+2\left(m_{1}+m_{2}\right) g l \sin \theta=0
		\end{align*}
	\end{answer}
	\item
		If kinetic energy and potential energy is given by $T=\frac{1}{2} m\left(\dot{x}^{2}+\dot{y}^{2}\right)$ and $V=-m g y$ respectively.
		\begin{tasks}(1)
			\task[\textbf{a.}] Write down Lagrangian of the system.
			\task[\textbf{b.}] Identify generalized coordinate and generalized velocity.
			\task[\textbf{c.}] Identify cyclic coordinate and discuss conservation of momentum.
			\task[\textbf{d.}]  Discuss equation of motion.
		\end{tasks}
	\begin{answer}
		\begin{align*}
		\intertext{(a)} L=T-V \Rightarrow \frac{1}{2} m\left(\dot{x}^{2}+\dot{y}^{2}\right)&-(-m g y) \Rightarrow \frac{1}{2} m\left(\dot{x}^{2}+\dot{y}^{2}\right)+m g y\\
		\intertext{(b)} \text{ Generalized coordinate }q_{1}&=x, q_{2}=y\text{ generalized velocity} \dot{q}_{1}=\dot{x}, \dot{q}_{2}=\dot{y}
		\intertext{(c)} \left(\frac{\partial L}{\partial q_{1}}\right)&=\left(\frac{\partial L}{\partial x}\right)=0, \text{so it is a cyclic coordinate}\\ \text{Then} \left(\frac{\partial L}{\partial \dot{x}}\right)&=p_{x}=m \dot{x}, \intertext{Identified as linear momentum in $x$ direction is constant of motion.}
		\left(\frac{\partial L}{\partial y}\right)&=m g \neq 0,\text{ so it is not cyclic coordinate}
		\intertext{(d)} \frac{d}{d t}\left(\frac{\partial L}{\partial \dot{x}}\right)-\left(\frac{\partial L}{\partial x}\right)&=0 \\ \frac{d}{d t} m \dot{x}-0&=0 \Rightarrow m \dot{x}=c, \intertext{which is exactly explained in section (b)}
		\frac{d}{d t}\left(\frac{\partial L}{\partial \dot{y}}\right)-\left(\frac{\partial L}{\partial y}\right)&=0 \\ \frac{d}{d t} m \dot{y}-m g&=0 \Rightarrow m \ddot{y}-m g=0
		\end{align*}
	\end{answer}
	\item
		Atwood's machine \\
		\begin{figure}[H]
			\centering
			\includegraphics[height=4.7cm,width=4cm]{EP-04}
		\end{figure}
	\begin{answer}
		\begin{align*}
		\intertext{A conservative system with holonomic, scleronomous constraint  (the pulley is assumed frictionless and massless). There is only one independent coordinate $x$, the position of the other weight being determined by the constraint that the length of the rope between them is $l$. The potential energy is}
		V&=-M_{1} g x-M_{2} g(l-x)
		\intertext{while the kinetic energy is}
		T&=\frac{1}{2}\left(M_{1}+M_{2}\right) \dot{x}^{2}
		\intertext{Lagrangian has the form}
		L&=T-V=\frac{1}{2}\left(M_{1}+M_{2}\right) \dot{x}^{2}+M_{1} g x+M_{2} g(l-x)
		\intertext{equation of motion}
		\frac{\partial L}{\partial  x}&=\left(M_{1}-M_{2}\right) g\\
		\frac{\partial  L}{\partial  x}&=\left(M_{1}+M_{2}\right) \dot{x}
		\intertext{$\mathrm{so}$}
		\left(M_{1}+M_{2}\right) \ddot{x}&=\left(M_{1}-M_{2}\right) g,\\
		\ddot{x}&=\frac{M_{1}-M_{2}}{M_{1}+M_{2}} g,\\
		\end{align*}
	\end{answer}

\end{enumerate}

Classical Mechanics  -CSIR/chapter/Variational Principles and Lagrange's Equations.tex


\documentclass[11pt]{article}

\input{preamble}

%\usepackage{tikz}
%\usepackage{tikz-qtree}
\usepackage{qtree}
\usepackage{xfrac}
\usepackage{graphicx}

\title{Week 5: Support Vector Machines\\ and Tensor Backpropagation}

\begin{document}
\author{\url{https://mlvu.github.io}}

\maketitle

\section{Support Vector Machines}

\subsection{Tensor loss}

The basic optimization objective for Support Vector Machines is
\begin{align*}
\text{minimize}\;\;&\frac{1}{2}||\oc{\bw}|| + \rc{C}\sum_i \gc{p_i}\\
\text{such that}\;\;&y_i(\oc{\bw}^T\x_i + \bc{b}) \geq 1 - \gc{p_i}\\
\text{and} \;\;&\gc{p_i} \geq 0
\end{align*}

\qu What does $i$ index? How many terms does the sum have, and how many constraints are there?

\ans{It iterates over the data. There are as many terms as there are instances in the data. There are two constraints per instance.}{}

What is the value of $y_i$ in this expression? What is its function?

\ans{$y_i$ is 1 for positive examples, and -1 for negative examples. It allows us to rewrite the constraints

\begin{align*}
\oc{\bw}^T\x_i + \bc{b} \geq 1 &\;\text{if $\x_i$ is positive} \\
\oc{\bw}^T\x_i + \bc{b} \leq -1 &\;\text{if $\x_i$ is negative} \\
\end{align*}

to a single constraint.
}{}

\qu There are two common ways to rewrite this expression before implementing it. What are they (in general terms) and what are their benefits?

\ans{The first option is to rewrite everything in terms of $\bw$ and $b$ in order to get rid of the constraints. This is useful when we want to use the SVM as the last layer in a neural network. Without constraints, we are free to use basic backpropagation.

The second option is to use Lagrange multipliers to get rid of $\oc{\bw}$ and rewrite everything in terms of the multipliers. This expresses the solution purely in terms of the support vectors.

The main benefit is that the whole algorithm can be written in terms of the dot products of pairs of instances. This means we do not need to see the actual feature vectors to compute the support vectors, only the dot products. This allows us to apply the \emph{kernel trick}.
}{}

\subsection{Lagrange multipliers}

Lagrange multipliers are a useful trick to know. In the lectures we only had time to describe the trick itself, and the rules for applying it. That should be enough to do the exam questions and the questions below, but if you want some more intuition for why Lagrange multipliers work the way they do please read  \href{https://www.khanacademy.org/math/multivariable-calculus/applications-of-multivariable-derivatives/constrained-optimization/a/lagrange-multipliers-single-constraint}{this article from the required reading}.

We have the following optimization problem:
\begin{align*}
\text{minize} &\;f(\bc{a}, \bc{b}) = \bc{a}^2 + 2\bc{b}^2 \\
\text{such that} &\;\bc{a}^2 = - \bc{b}^2 + 1
\end{align*}

\qu The first step is to rewrite the constraint so that the right side is equal to zero. Do so.
\ans{$\bc{a}^2 + \bc{b}^2 - 1 = 0$}{}
What does the constraint say about the allowed inputs (what shape do the allowed inputs make in the $(\bc{a}, \bc{b})$-plane)?
\ans{The solutions are constrained to a circle, centered on the origin, with radius 1 (the so-called bi-unit circle).}{}

We now define a function $L(\bc{a}, \bc{b}, \rc{\alpha}) = f(\bc{a}, \bc{b}) + \rc{\alpha} G$, where $G$ is the left hand side of the constraint equal to zero (how much any given $a$ and $b$ violate the constraint).\footnotemark

\qu Write out $L(\bc{a}, \bc{b}, \rc{\alpha})$ for our problem.
\ans{$L(\bc{a}, \bc{b}, \rc{\alpha}) = \bc{a}^2 + 2\bc{b}^2 + \alpha(\bc{a}^2 + \bc{b}^2 - 1) $}{}

\footnotetext{For plain Lagrange multipliers, where the constraints are all equalities, we can either add or subtract the term containing the constraint. For inequality constrains, it depends on whether we are maximizing or minimizing.}

We take the derivative of $L$ with respect to each of its \emph{three} parameters, and set these equal to zero.

\qu Fill in the blanks
\ans{
\begin{align*}
\frac{\kp L}{\kp \bc{a}} &= \frac{\kp(\bc{a}^2 + 2\bc{b}^2 + \rc{\alpha} \bc{a}^2 + \rc{\alpha} \bc{b}^2 - \rc{\alpha})}{\kp \bc{a}} \\
&= 2\bc{a} + 2\rc{\alpha} \bc{a} \\
 	\frac{\kp L}{\kp \bc{b}} &= 4\bc{b} + 2 \rc{\alpha} \bc{b} \\
 	\frac{\kp L}{\kp \rc{\alpha}} &= \bc{a}^2 + \bc{b}^2 - 1
\end{align*}
}{}

\begin{align}
a (\ans{2+2\rc{\alpha}}{\ldots})&= 0\\
b (\ans{4 + 2\rc{\alpha}}{\ldots})&= 0 \\
a^2 + b^2 &= 1 \label{line:constraint}
\end{align}

\noindent Note that the last line recovers the original constraint. We now have three equations with three unknowns, so we can solve for $a$ and $b$. From the shape of the function (it's symmetric in both the $a$ and $b$ axes), we should expect at least two solutions.

We can get these from the above equations by noting that if $\bc{a}$ and $\bc{b}$ are both nonzero, we can derive a contradiction. Thus either $\bc{a}$ or $\bc{b}$ must be zero.

\qu Give the solutions for both cases (remember that $x^2 = 1$ has \emph{two} solutions).

\ans{
From line (\ref{line:constraint}), above we see that if $a =0$, then $b^2=1$ and vice versa. This gives us
\begin{align*}
a = 0&, b = 1\\
a = 0&, b = -1\\
a = 1&, b = 0\\
a = -1&, b = 0\\
\end{align*}
as extrema. Filling these in, we find that the last two lines minimize the function. (The first two are maxima, as can be seen by clicking the Wolfram Alpha link below.
}{}

Happily, \href{https://goo.gl/Uaz5mg}{Wolfram Alpha} agrees with us (and provides some informative plots).

\subsection{The kernel trick}

The feature space of $\oc{k}$ is a projection of point $\ba$ to point $\ba'$ such that
\[
\oc{k}(\ba, \bb) = {\ba'}^T\bb' \p
\]

We have a dataset with two features Let $\bm{a} = \ma{a_1\\ a_2}$ and $\bm{b} = \ma{b_1\\ b_2}$. We define the \emph{kernel}
\[
\oc{k_1}(\bm{a}, \bm{b}) = (\bm{a}^T\bm{b})^2 \p
\]

\qu Show that the feature space defined by$\oc{k_!}$ is
\[
\ma{{x_1}^2\\\sqrt{2}x_1x_2\\{x_2}^2}\p
\]
Hint: start by writing out the definition as a scalar function. See if you can re-arrange this back into a dot procut of two other vectors.
\ans{

Starting from the definition of $k$:
\begin{align*}
k(\ba, \bb) &= \left( \ma{a_1\\ a_2}^T\ma{b_1\\ b_2} \right)^2\\
 &= \left ( a_1 b_1 + a_2 b_2 \right)^2 \\
 &= a_1 b_1 a_1 b_1 + 2 a_2 b_2 a_1 b_1 + a_2 b_2 a_2 b_2 \\
 &= a_ 1a_1\cdot b_1 b_1 + 2 \cdot a_1 a_2 \cdot b_1 b_2 + a_2a_2\cdot b_2b_2\\
 &= \ma{a_1a_1\\ \sqrt{2}a_1a_2\\a_2a_2} ^T \ma{b_1b_1\\ \sqrt{2}b_1b_2\\b_2b_2}
\end{align*}
}{}

\qu What is the feature space for the kernel
\[
\oc{k_2}(\ba, \bb) = \left(\ba^T\bb +1\right)^2\;\;\;\text{?}
\]
\ans{
\begin{align*}
k(\ba, \bb) &= \left( \ma{a_1\\ a_2}^T\ma{b_1\\ b2} + 1\right)^2\\
 &= (a_1 b_1 + a_2 b_2)^2 +  2( a_1 b_1 + a_2 b_2) + 1 \\
 &= a_1b_1a_1b_1 + 2a_1b_1a_2b_2 + a_2b_2a_2b_2 + 2a_1b_1 + 2a_2b_2 + 1\\
 &= \ma{1 \\ \sqrt{2}a_1 \\ \sqrt{2}a_2 \\ {a_1}^2\\\sqrt{2}a_1a_2 \\{a_2}^2} ^T\ma{1 \\ \sqrt{2}b_1 \\ \sqrt{2}b_2 \\ {b_1}^2\\\sqrt{2}b_1b_2 \\{b_2}^2}
\end{align*}
}{}

\section{Backpropagation Revisited}

\subsection{The multivariate chain rule}

If we want to do backpropagation for a computation graph where an output variable depend in an input variable through multiple intermediate values (the graph contains a diamond), we require the \emph{multivariate chain rule}. If you don't quite remember how it goes, re-read slides 24--29 fo the deep learning lecture before trying these questions.

We will use the backpropagation algorithm to find the derivative of the function
\[
f(x) = \sin(x^2)\cos(x^3)
\]
with respect to $x$.

\qu First, we break the function up into modules. Fill in the blanks.

\begin{align*}
f &= \rc{a}\gc{b} \\
\rc{a} &= \ans{\sin(c)}{\ldots}\\
\gc{b} &= \ans{\cos(d)}{\ldots} \\
\oc{c} &= x^2 \\
\bc{d} &= x^3 \\
\end{align*}

\qu Draw the computation graph with nodes $x, \oc{c}, \bc{s}, \rc{a}, \gc{b}, f$

\ans{

\hspace{5em}\includegraphics[width=0.4\linewidth]{compgraph}

}{}

\qu Work out the \emph{local} derivatives. Which is the correct expression for the gradient $\kp f / \kp x$?

\ans{
\begin{align*}
\frac{\kp f}{\kp x} &= \frac{\kp f}{\kp \rc{a}}\frac{\kp \rc{a}}{\kp x} + \frac{\kp f}{\kp \gc{b}}\frac{\kp \gc{b}}{\kp x} & \text{multivariate chain rule}\\
&= \frac{\kp f}{\kp \rc{a}} \frac{\kp \rc{a}}{\kp \oc{c}} \frac{\kp \oc{c}}{\kp x} + \frac{\kp f}{\kp \gc{b}}\frac{\kp \gc{b}}{\kp x} & \text{chain rule for $\oc{c}$: $\frac{\rc{a}}{x} = \frac{\kp \rc{a}}{\kp \oc{c}} \frac{\kp \oc{c}}{\kp x}$}\\
&= \frac{\kp f}{\kp \rc{a}} \frac{\kp \rc{a}}{\kp \oc{c}} \frac{\kp \oc{c}}{\kp x} + \frac{\kp f}{\kp \gc{b}}\frac{\kp \gc{b}}{\kp \bc{d}}\frac{\bc{d}}{x} & \text{chain rule for $\bc{d}$}\\
 &= \gc{b} \cdot \cos(\oc{c}) \cdot 2x - \rc{a}\cdot \sin(\bc{d})\cdot 3x^2& \text{work out all local derivatives}
\end{align*}}{}

\noindent This is a common exam question so make sure to practice if you're not sure. You can easily create new questions for yourself by coming up with any function $f(x)$ with a diamond shape in the computation (just make sure that $x$ is used twice).

\subsection{Tensor Backpropagation}

In scalar backpropagtion, we need to work out only the local derivatives. Once we have these, we can multiply them in any order to get the global derivative. If we want to apply backpropagation to tensors, things are not so easy.

\qu Why not? What is it about the local derivatives in tensor backpropagation that makes it difficult to apply in this way?

\ans{In tensor backpropagation, the local derivatives are often, for instance, the derivative of a vector function over matrix parameters, like the local derivative $\kp \gc{\bm k}/\kp \oc{\bm W}$ shown in the slides. In this case, the collection of all scalar derivatives (every output with every input) is best represented as a 3 tensor. This would take way too much memory, and there's then no natural way to multiply this local derivative with the others to compute the global derivative.}{}

The solution is not to compute the local derivatives explicitly, but only to compute the gradients on the inputs over the loss, given the gradients of the outputs over the loss.

We will practice this for a simple feedforward layer (without activation). Consider a module $f$ that computes:\footnotemark
\[
\y = f(\x, \oc{\W}, \bc{\bb}) = \oc{\W}\x + \bc{\bb}
\]

\footnotetext{Normally, we would consider $\x$ the input, and $\oc{\W}$ and $\bc{\bb}$ the parameters, but for our AD engine, the distinction does not matter: everything going in to the computation is an input, and we may need the gradient over all three.}

\noindent Assume that this module is part of a much larger network. Its output $\y$ is fed as input to another module, which produces a new output that is fed to another module and so on. At the end, a single scalar loss $L$ is produced.

\hspace{5em}\includegraphics[width=0.7\linewidth]{networkloss}

\noindent Ultimately, we want to work out the derivative of this loss,  with respect to our inputs:
\[
\frac{\kp L}{\kp \x}\;\;\; \frac{\kp L}{\kp \oc{\W}}\;\;\; \frac{\kp L}{\kp \bc{\bb}}
\]

\noindent We'll start with the bias term $\bc{\bb}$. Since matrix/vector calculus can get complicated, and doesn't translate to tensors of higher rank, we'll develop everything in terms of scalar calculus.  Once we've taken the derivatives we'll transform everything to matrix operations to make the implementation efficient.

We're interested in $\frac{\kp L}{\y}\frac{\kp \y}{\kp \bc{b}_j}$, but we need matrix/vector calculus to do that. Instead we will consider $L(f(\bc{\bb}))$ as a \emph{scalar} computation, which has multiple intermediate values $y_1, y_2 \ldots, y_k$. By the multivariate chain rule, we can just take the derivative over each path (through each value $y_i$), and sum them:
\[
\frac{\kp L}{\kp \bc{b}_j} = \sum_i \frac{\kp L}{\kp y_i} \frac{\kp y_i}{\kp \bc{b}_j}
\]

\noindent We will assume that $\kp L/\kp y_i$ is given to us by the AD engine as a vector $\bm d$ with $d_i = \kp L/\kp y_i$. All we need to work out is a simple matrix operation that computes $\kp L/\kp \bc{b}_j$ for all $j$ and lays the results out in the same shape as $\bc{\bb}$. We'll start by working out the scalar derivative.

\qu Fill in the gaps
\begin{align*}
\frac{\kp L}{\kp \bc{b}_j} &= \sum_i \frac{\kp L}{\kp y_i} \frac{\kp y_i}{\kp \bc{b}_j} = \sum_i d_i \ans{\frac{\kp y_i}{\kp \bc{b}_j}}{\ldots}\\
&= \sum_i d_i \frac{\kp [\ans{\oc{\W}\x + \bc{\bb}}{\ldots}]_i}{\kp \bc{b_j}} = \sum_i d_i \frac{\kp [\oc{\W}\x]_i + \bc{b}_i}{\kp \bc{b_j}}\\
&= \sum_i d_i\frac{\kp \bc{b}_i}{\kp \bc{b}_j} = \ans{d_j}{\ldots}
\end{align*}

\qu What should the \texttt{backward()} function for $f$ return as the gradient of $\bc{\bb}$?

\ans{We should return a vector $\bc{\bb}'$ with the same shape as $\bc{\bb}$, where $\bc{b}'_j = \kp L / \kp \bc{b}_j$. Since $d_j$ is the gradient for $\bc{b}_j$, we can just return the vector $\bm d$.}{}

We'll do the same for the gradient over $\x$.

\qu Work out the scalar derivative $\kp L / \kp x_j$ using the multivariate chain rule (again taking $\kp L / \kp y_i$ as given).
\ans{
\begin{align*}
\frac{\kp L}{\kp x_j} &= \sum_i 	\frac{\kp L}{\kp y_i}\frac{\kp y_i}{\kp x_j} = \sum_i d_i \frac{\kp y_i}{\kp x_j} \\
&= \sum_i d_i \frac{\kp \oc{\W}\x}{\kp x_j} = \sum_i d_i \frac{\kp \oc{\W}_{i\cdot}\times\x + \kc{\bb} }{\kp x_j} \\
&=  \sum_i d_i \frac{\kp \sum_k \oc{\W}_{ik}x_k}{\kp x_j} = \sum_i d_i \frac{\kp \oc{\W}_{ij}x_j}{\kp x_j}\\
&=  \sum_i d_i \oc{\W}_{ij}
\end{align*}

}{}

\qu What should the \texttt{backward()} function for $f$ return as the gradient of $\x$?

\ans{We are looking for a function that returns a vector $\x'$ where $x'_j = \sum_i d_i \oc{\W}_{ij}$. In other words, $x'_j$ should be the dot product between $\bm d$ and and the $j$-th column of $\oc{\W}$. We get this if we  compute $\x' = {\bm d}^T\oc{\W}$.

\includegraphics[width=0.7\linewidth]{matbp}

}{}

\section{Bonus: Expectation Maximization}

\emph{This is not an exam question, but it's helpful to do if you want to understand the EM algorithm.}

Assume we have a Gaussian Mixture Model in one dimension with two components: $\rc{N(0, 1)}$ and $\gc{N(1,1)}$. The weights $\rc{w_1}$ and $\gc{w_2}$ of the components are equal.

{
\centering
\includegraphics[width=0.7\linewidth]{hw-components-unscaled} \\
\includegraphics[width=0.7\linewidth]{hw-stackplot}
}

\qu Compute the probability density of the point 0, under the Gaussian Mixture.
\ans{
\begin{align*}
p(0) &= \rc{\frac{1}{2} N(0, 1)} + \gc{\frac{1}{2} N(1,1)}\\
 &= \rc{\frac{1}{2\sqrt{2\pi}} \exp(0)} + \gc{\frac{1}{2\sqrt{2\pi}} \exp\left(-\frac{1}{2}\right)} \\
 &= \rc{\frac{1}{2}\frac{1}{\sqrt{2 \pi}}} + \gc{\frac{1}{2}\frac{1}{\sqrt{2 \pi e}}} \approx 0.32
\end{align*}
}{}

\qu Under the EM algorithm, what responsibility is assigned to each component for the point 0?

\ans{
To compute the responsibility, we compute the probability of the component $z$ given the point $x$: $p(z\mid x)$, and normalize over all components.
We get for component 1:
\[
\frac{\rc{\frac{1}{2\sqrt{2\pi}} \exp(0)}}{\rc{\frac{1}{2\sqrt{2\pi}} \exp(0)} + \gc{\frac{1}{2\sqrt{2\pi}} \exp\left(-\frac{1}{2}\right)}} = \frac{\rc{1}}{\rc{1} + \gc{\frac{1}{\sqrt{e}}}} \approx 0.62
\]
and for component 2:
\[
\frac{\gc{\frac{1}{2\sqrt{2\pi}} \exp\left(-\frac{1}{2}\right)}}{\rc{\frac{1}{2\sqrt{2\pi}} \exp(0)} + \gc{\frac{1}{2\sqrt{2\pi}} \exp\left(-\frac{1}{2}\right)}} = \frac{\gc{\frac{1}{\sqrt{e}}}}{\rc{1} + \gc{\frac{1}{\sqrt{e}}}} \approx 0.37
\]

Note: Using Bayes' rule, this translates to $p(z\mid x) = \frac{p(x\mid z)p(z)}{p(x)}$. The denominator is the sum computed in the previous exercise, and the responsibilities are the proportions of each term to the total.\footnotemark

This is no accident, it is essentially what Bayes' rule tells us: to compute $p(z\mid x)$, we find $p(x)$ by marginalizing $Z$ out of $p(X=x, Z)$. This gives us a big sum, with one term for each $z$. The proportion of this term to the total is $p(z\mid x)$.

}{}

\end{document}


\begin{tabularx}{\linewidth}{lXXX}
	\toprule
	               & \textbf{Worst case run time} & \textbf{Average case run time} & \textbf{In place or not?}      \\ \midrule
	Merge Sort     & \nlgn                        & \nlgn                          & No. $ \Theta(n) $.             \\
	Insertion Sort & \nn                          & \nn                            & Yes. $ \Theta(1) $ additional. \\
	Bubble Sort    & \nn                          & \nn                            & Yes.                           \\
	Selection Sort & \nn                          & \nn                            & Yes.                           \\
	Quick Sort     & \nn                          & \nlgn                          & Yes.                           \\
	Heap Sort      & \nlgn                        & \nlgn                          & Yes.                           \\ \bottomrule
\end{tabularx}

\subsection{Behavior of sorting algorithms}

\begin{description}
	\item[Merge sort] (CLRS p.29-34) Divide-and-conquer. Recursively split list into two sublists of equal size, then merge the sorted sublists by comparing the front elements and taking the smallest until both lists are empty.
	\item[Insertion sort] (CLRS p.17-18) Runs through the list, and places the $j$th element in the correct place out of the sublist $A[1..j-1]$, the already sorted elements.
	\item[Bubble sort] (CLRS p.40, Problem 2-2) Sorts by repeatedly looping through the list, swapping out of order elements. The bigger elements slowly "bubble" upwards.
	\item[Selection sort] (CLRS p.29, Exercise 2.2-2) Runs through the list and takes the smallest unsorted element and places it in front of the array.
	\item[Quick sort] (CLRS Chapter 7, p.170 onwards) Takes some pivot element $x$ from a place in the list, and places all elements smaller than $x$ to the left of $x$, and all elements bigger to the right. Then it recursively quicksorts the sublists of elements smaller than $x$ and bigger than $x$.
	\item[Heap sort] (CLRS Chapter 6, p.151 onwards) Constructs a max heap out of the array. Then it repeatedly exchanges the maximum element with the last element in the heap, then heapifies again.
	\subitem \textsc{Max-Heapify}(i): Assuming that the trees at \textsc{Left}(i) and \textsc{Right}(i) are heaps, make the tree rooted at $i$ a heap.
\end{description}

Algorithms and Data Structures - Reference/sorting.tex