\documentclass[Economics.tex]{subfiles}
\begin{document}
\chapter{Macroeconomic Policies}
In general, macroeconomic policies can be divided into demand management policies, and supply-side policies.

Demand management policies in general consist of monetary policies and fiscal policies. Monetary policies involve the manipulation of monetary variables like exchange rate, interest rate and the supply of money to influence \AD{}. Fiscal policies involve the use of government spending and taxation to influence \AD{}.
\section{Exchange rate policy}
Decreasing exchange rate increases net exports, as devaluing the currency reduces the cost of the country's exports in foreign currency while increasing the cost of imports in the country's currency, so assuming \(\PEDx{} > 0\) and \(\PEDm{} > 1\), the quantity demanded of the country's exports in foreign currency rises so export revenue rises, and the quantity demanded of imports in the country falls more than proportionately so import expenditure falls, which reinforce an increase in net export revenue, increasing \AD{}. The reverse argument applies.

The condition that \(\PEDx{} > 0\) and \(\PEDm{} > 1\) can be extended and mathematically proven to simply having the sum of the two be greater than one. This is the Marshall-Lerner condition, which states that for a devaluation of a country's currency to increase net exports, \(\PEDx{} + \PEDm{} > 1\).
\subsection{Merits of exchange rate policy}
For countries with relatively large external demand, like Singapore, exchange rate policy has the greatest effect on \AD{}, since net exports will make up the greatest proportion of \AD{}. Net exports takes up about 75\% of Singapore's \AD{}, for example.
\subsection{Problems of exchange rate policy}
\subsubsection{Imported inflation}
For countries that import a large proportion of the raw materials used in production of goods, a devaluation of the currency may lead to imported inflation, with prices of goods in domestic currency increasing, because the price of imported raw materials in domestic currency increases, leading to an increased unit cost of production and thus a fall in supply of goods which involve imported raw materials. The price of exports in foreign currency thus may not fall as much, reducing the increase in net exports.

If the increase in net exports leads to demand-pull inflation, causing price levels to rise, then the price of export goods in domestic currency may increase, offsetting the fall in price of exports in foreign currency due to the devaluation, similarly reducing the increase in net exports.
\subsubsection{J-curve effect}
A devaluation, in the short run, causes net exports and trade balance to fall due to the J-curve effect. The main reason for this is time lags, as producers and consumers take a while to adjust their purchases to the changed prices caused by the change in exchange rate. There may also be contractual agreements or planned advance orders, preventing the quantity of exports or imports from changing until the contracts lapse.  This causes export revenue to remain constant, assuming quantity of exports is constant, while import expenditure rises as a constant quantity of imports is combined with increasing prices of imports in domestic currency, so net exports falls. This is in line with the Marshall-Lerner condition, as quantities not changing implies the demands are both price inelastic.
\section{Interest rate policy}
Lowering interest rate increases consumption and investment as interest rate is the cost of borrowing, which when lowered will make consumers more willing and able to borrow to buy expensive items like cars, increasing consumption, and firms invest more as investments with lower expected returns will appear more profitable since the cost of borrowing to undertake those investments have fallen, increasing investment. Thus \AD{} increases. The reverse argument applies.

A reduced interest rate will also cause an outflow of hot money as investors seek higher returns from countries with higher interest rates, so the supply of the country's currency increases while the demand falls and so the currency depreciates. Prices of the country's exports in foreign currency falls while prices of imports in the country's currency rises, and assuming the Marshall-Lerner condition holds, net exports increases, also increasing \AD{}. This is a small point, however.
\subsection{Merits of interest rate policy}
Since investment increases, capital accumulation increases and this will benefit potential growth in the long run as \AS{} will increase in future with increased capital accumulation.
\subsection{Problems with interest rate policy}
If the economy is in recession, consumer and business confidence may be low, and increasing interest rate may not increase consumption or investment by much as consumers want to save more in case they e.g.\ become unemployed, and firms are pessimistic about investments. This reduces the effectiveness of interest rate policy.

Some firms may not need to borrow in order to invest as they may have their own reserves. Changing interest rate will not really affect how much such firms invest, which diminishes the effectiveness of interest rate policy.

Singapore does not use interest rate policy due to its choice of controlling its exchange rate and having free capital flow. If Singapore were to lower interest rates, for example, hot money would flow out of Singapore, foiling the policy.
\section{Fiscal policy}
Fiscal policies involve the use of government spending and taxation to influence \AD{}.

Reducing personal and corporate income taxes will increase the disposable income of households and the post-tax profits of firms, increasing their purchasing power and expected yields from investments respectively. For households, they increase consumption of normal goods and so consumption increases. For firms, the marginal efficiency of investment curve shifts right and at every interest rate, investment increases. Thus \AD{} increases.

Increasing government spending directly increases \AD{}, it being a component of \AD{}.
\subsection{Merits of fiscal policy}
A fall in personal income taxes will increase the opportunity cost of leisure, which disincentivises leisure and encourages work. People who are currently not seeking jobs may decide to do so, which will boost \AS{} in future.

Since investment increases, capital accumulation increases and this will benefit potential growth in the long run as \AS{} will increase in future with increased capital accumulation.
\subsection{Problems with fiscal policy}
Increasing government spending may lead to the crowding-out effect if government spending is financed by borrowing, as the government will be competing with private firms for loans, which increases the demand for loans and thus the interest rate, reducing investment and diminishing the increase in \AD{} from increased government spending.

The government may also risk running into a budget deficit if it increases government spending. To finance these debts, the government may have to borrow, and then taxes may rise in future in order to repay loans, which will counter an increase in \AD{} in the present. Too huge a government debt may weaken investor confidence which reduces foreign investment in the country, leading to capital flight, which will destabilise the exchange rate, and a reduced credit rating, which will make it more expensive for the country to borrow.

Fiscal policy may also involve time lags, depending on whether legislation is required to change in government spending or taxes, and how fast the changes cause an effect on consumption and investment.

If the economy is in recession, consumer and business confidence may be low, and direct taxes may not increase consumption or investment by much as consumers want to save more in case they e.g.\ become unemployed, and firms are pessimistic about investments. This reduces the effectiveness of this form of fiscal policy.

In Singapore, fiscal policy has limited effect due to its small fiscal multiplier, which in turn is due to having a high marginal propensity to import and marginal propensity to save. The effect of tax cuts is also likely to be small, given that domestic demand is a relatively small component of \AD{}, and that our tax base is small -- only one-third of the working population. Singapore's tax system is also not very counter-cyclical as taxes are based on the previous year's income: if the current year is a recession year while the previous is a boom year, individuals and firms pay tax in the current year based on higher incomes earned the previous year, when ideally the system should leave them with more money due to the recession.
\section{Other policies}
To solve a current account deficit, a government may use export subsidies and tax rebates for export industries to boost the country's exports by making their unit cost of production lower, in turn reducing export prices and thus increasing export revenue, assuming \(\PEDx{} > 1\), boosting the current account and thus the balance of payments.

The government can also use protectionism to do so -- the above policy is actually a form of protectionism anyway.

Of course, this comes with all the demerits of protectionism.
\section{Supply-side policy}
Supply-side policies are a basket of policies used to influence \AS{}. They are generally categorised into market-oriented policies and interventionist policies. Market-oriented policies rely on market forces and competition to achieve greater efficiency, while interventionist policies rely on market intervention and the correction of supposed market failures.
\subsection{Market-oriented policies}
Cutting direct tax rates, while primarily a fiscal policy, affects \AS{} as well. As explained earlier, cutting corporate income taxes results in an increase in investments, increasing capital accumulation, innovation and the development of new technology, leading to an increase in productive capacity and thus \AS{}. Lower personal income taxes increases the opportunity cost of leisure, increasing the incentive to work for longer or more efficiently and also enticing those previously not in the labour market to work. Supply in individual markets increases and so \AS{} increases.

\paragraph{Problems} However, reducing personal income taxes may also encourage people to work fewer hours since they can get the same amount of disposable income while working less. Reducing corporate income taxes may result in firms paying higher dividends to shareholders instead of investing.

Cutting unemployment benefits will also increase the opportunity cost of leisure, which incentivises work, eventually leading to an increase in \AS{}.

\paragraph{Problems} However, this may increase income inequality especially for those who are structurally unemployed, and if there is low unemployment, this will not help much at all.

The government can also encourage competition by introducing pro-competition policies like antitrust laws, removing barriers to entry to regulated markets, privatisation, and reducing trade barriers. The increase in competition encourages firms to become more efficient, producing a greater output from a given amount of resources. Unit cost of production falls, so supply curves shift right, and \AS{} increases.

\paragraph{Problems} In practice, implementing such laws alienates the business community, and it also reduces supernormal profits, making it more difficult for firms to innovate and do research and development as they have less resources to do so.

The government can reform trade unions through laws that restrict the extent to which unions can push wages above equilibrium and enforce restrictive practives, increasing employment, labour market flexibility, and efficiency. The reduced labour costs in turn increase firms' profits, enabling more investment, helping \AS{} increase in future.

\paragraph{Problems} In practice, implementing such laws will alienate workers. Flexible labour markets will also increase income inequality as workers are forced to accept jobs at lower wages. It also leads to lower job security as trade unions have less power with regards to retrenchment and the like, which may cause greater stress in the workplace and thus lower efficiency, diminishing the increase in \AS{}.
\subsection{Interventionist policies}
The government can provide subsides for education and training, which will reduce the price of doing so. Employers and workers will be more willing and able to send employees to and attend, respectively, training. If the training helps to improve workers' productivity, each worker can produce more output per man hour, resulting in a lower unit cost of production, increasing supply and thus \AS{}.

\paragraph{Merits} This also increases \AD{} through an increase in government spending.

\paragraph{Problems} However, it requires time to take effect, and it may not be very effective at all, depending on how receptive workers are. It also requires government spending, and so has all the associated issues.

Subsidising research and development will make firms more willing and able to conduct it as the price of doing so decreases. If the research results in new methods of production that are more efficient, the unit cost of production falls; firms may also be able to produce more from the same amount of resources, so supply and \AS{} increases.

\paragraph{Merits} This also increases \AD{} through an increase in government spending.

\paragraph{Problems} Research and development may not result in any better method of production -- it is uncertain. If this happens, funds used for the research and development by firms would have been wasted since an opportunity cost has been incurred and they could have been used for other purposes like upgrading equipment or giving employees bonuses, but instead they were channeled into research and development which ultimately did not result in an increase in profits or other benefit for the firm.

To combat search unemployment, the government can provide information through mass media, job agencies and job fairs, to help unemployed workers be matched to suitable jobs more quickly.

\paragraph{Problems} The success of such a policy depends really on how keen the unemployed are about getting unemployed. If they do not really want a job, they may not seek for jobs as actively, and so stay unemployed for longer.
\subsection{Merits of supply-side policy}
Supply-side policy can act on both \AD{} and \AS{}, bringing about non-inflationary growth; the increase in \AD{} helps to drive the increase in national income, while the increase in \AS{} prevents inflation, allowing the actual growth to be sustained.

If prices are lowered due to supply-side policy, it may also improve the price competitiveness of exports, which will improve the country's current account and thus balance of payments.
\subsection{Problems with supply-side policy}
Sup\-ply-side policies often involve high costs, which will incur a high opportunity cost, as resources that could be channeled elsewhere are now channeled to these policies, and if the benefits of the policy are less than the benefits that the resources could have brought if used elsewhere, then there is a misallocation of resources.

If the costs of supply-side policy are funded through taxes, the government may increase taxes, which may have disincentive effects on work as the opportunity cost of leisure is decreased. Workers now have less incentive to work, which may cause labour productivity to fall, offsetting the benefits from supply-side policy.

If the costs are otherwise funded through government borrowing, and taxes may rise in future in order to repay loans, which will lead to disincentive effects as above. Too huge a government debt may weaken investor confidence which reduces foreign investment in the country, leading to capital flight, which will destabilise the exchange rate, and a reduced credit rating, which will make it more expensive for the country to borrow.

Supply-side policies are also uncertain, as mentioned earlier.

On their own, supply-side policies will not cause actual growth if the economy is not near or at full employment. An increase in \AD{} is needed for the economy to have actual growth.
\section{Macroeconomic conflict}
Macroeconomic goals can sometimes conflict with each other. Most often, inflation conflicts with everything else as combatting inflation usually involves cooling the economy and reducing pressures while the other goals involve boosting the economy.
\end{document}


\documentclass{article}
%\usepackage{fullpage}
%\usepackage{nopageno}
\usepackage[margin=1.5in]{geometry}
\usepackage{tikz}
\usetikzlibrary{shapes.geometric, calc}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage[normalem]{ulem}
\usepackage{fancyhdr}
\usepackage{cancel}
\usepackage{enumerate}
%\renewcommand\headheight{12pt}
\pagestyle{fancy}
\lhead{May 2, 2014}
\rhead{Jon Allen}
\allowdisplaybreaks

\newcommand{\abs}[1]{\left\lvert #1 \right\rvert}

\begin{document}
Part 2 (3 points): Due in class Friday, May 2.

Chapter 8: \#33 (just the first part where you calculate R7 and R8), and the problems below.

Schroder exercise 1: Draw the dissection of the 11-gon that corresponds to the bracketing ( a1 ( ( a2 a3 a4 a5 ) a6 ) (a7 (a8 a9) a10 ) ).

Schroder exercise 2: Draw the Schroder paths from (0,0) to (6,0). Verify that exactly half have no horizontal steps on the x-axis.
\section*{Chapter 8}
\begin{enumerate}
\setcounter{enumi}{31}
  \item
  Use the recurrence relation (8.31) to compute the small Schr\"{o}der numbers $s_8$ and $s_9$.

  We are given the $s_1$ through $s_7$ on page 310. They are $1,1,3,11,45,197,903,\dots$
  \begin{align*}
    0&=(n+2)s_{n+2}-3(2n+1)s_{n+1}+(n-1)s_n\\
    s_{n+2}&=\frac{3(2n+1)}{n+2}s_{n+1}+\frac{1-n}{n+2}s_n\\
    s_{n}&=\frac{3(2(n-2)+1)}{(n-2)+2}s_{n-1}+\frac{1-(n-2)}{(n-2)+2}s_{n-2}\\
    s_{n}&=\frac{3(2n-3)}{n}s_{n-1}+\frac{3-n}{n}s_{n-2}\\
    s_8&=\frac{3\cdot13}{8}\cdot903-\frac{5}{8}\cdot197\\
    &=4279\\
    s_9&=\frac{3\cdot15}{9}\cdot4279-\frac{6}{9}\cdot903\\
    &=20793
  \end{align*}

  \item
  (just the first part where you calculate R7 and R8)

  Use the recurrence relation (8.32) to compute the large Schr\"{o}der numbers $R_7$ and $R_8$. Verify that $R_7=2s_8$ and $R_8=2s_9$, as stated in Corollary 8.5.8.
\begin{align*}
  R_n&=R_{n-1}+\sum\limits_{k=0}^{n-1}{R_kR_{n-1-k}},\quad(n\ge1)\\
  R_0&=1\\
  R_1&=1+1=2\\
  R_2&=2+2+2=6\\
  R_3&=6+6+4+6=22\\
  R_4&=22+22+12+12+22=90\\
  R_5&=90+90+44+36+44+90=270+88+36=394\\
  R_6&=394+394+180+132+132+180+394=1806\\
  R_7&=1806+1806+788+540+484+540+788+1806=8558\\
  R_8&=8558+8558+3612+2364+1980+1980+2364+3612+8558=41586\\
\end{align*}
And since I accidentally did \#32 anyhow, I could as well do the second part. $41586/2=20793$ and $8558/2=4279$
\end{enumerate}
\section*{Schr\"{o}der Exercises}
\begin{enumerate}
  \item
  Draw the dissection of the 11-gon that corresponds to the bracketing $( a_1 ( ( a_2 a_3 a_4 a_5 ) a_6 ) (a_7 (a_8 a_9) a_{10} ) )$.

  \begin{tikzpicture}
    \node (pol) [draw, thick, black,rotate=90,minimum size=6cm,regular polygon, regular polygon sides=11] at (0,0) {};

    \foreach \n [count=\nu from 1, remember=\n as \lastn, evaluate={\nu+\lastn}] in {1,2,...,10}
    \node[anchor=\n*(360/11)]at(pol.side \n){$a_{\nu}$};
    \draw (pol.corner 8) -- node[anchor=north]{$(a_8 a_9)$}(pol.corner 10);
    \draw (pol.corner 7) -- node[anchor=south]{$(a_7(a_8 a_9)a_{10})$}(pol.corner 11);
    \draw (pol.corner 2) -- node[anchor=north]{$(a_2a_3a_4a_5)$}(pol.corner 6);
    \draw (pol.corner 2) -- node[anchor=east]{$((a_2a_3a_4a_5)a_6)$}(pol.corner 7);
    \node[anchor=360]at(pol.side 11){$(a_1((a_2a_3a_4a_5)a_6)(a_7(a_8 a_9)a_{10}))$};
  \end{tikzpicture}

  \item
  Draw the Schroder paths from $(0,0)$ to $(6,0)$. Verify that exactly half have no horizontal steps on the $x$-axis.

  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,0) node (p1) {} (1,0) node (p2) {}
  (1.5,0) node (p3) {} (2,0) node (p4) {} (2.5,0) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,.5) node (p1) {} (1,0) node (p2) {}
  (1.5,0) node (p3) {} (2,0) node (p4) {} (2.5,0) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,0) node (p1) {} (1,0) node (p2) {}
  (1.5,.5) node (p3) {} (2,0) node (p4) {} (2.5,0) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,0) node (p1) {} (1,0) node (p2) {}
  (1.5,0) node (p3) {} (2,0) node (p4) {} (2.5,.5) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,.5) node (p1) {} (1,0) node (p2) {}
  (1.5,.5) node (p3) {} (2,0) node (p4) {} (2.5,0) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,.5) node (p1) {} (1,0) node (p2) {}
  (1.5,0) node (p3) {} (2,0) node (p4) {} (2.5,.5) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,0) node (p1) {} (1,0) node (p2) {}
  (1.5,.5) node (p3) {} (2,0) node (p4) {} (2.5,.5) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,.5) node (p1) {} (1,1) node (p2) {}
  (1.5,.5) node (p3) {} (2,0) node (p4) {} (2.5,0) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,0) node (p1) {} (1,0) node (p2) {}
  (1.5,.5) node (p3) {} (2,1) node (p4) {} (2.5,.5) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,.5) node (p1) {} (1,1) node (p2) {}
  (1.5,.5) node (p3) {} (2,0) node (p4) {} (2.5,.5) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,.5) node (p1) {} (1,0) node (p2) {}
  (1.5,.5) node (p3) {} (2,1) node (p4) {} (2.5,.5) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,.5) node (p1) {} (1,.5) node (p2) {}
  (1.5,.5) node (p3) {} (2,0) node (p4) {} (2.5,0) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,.5) node (p1) {} (1,.5) node (p2) {}
  (1.5,.5) node (p3) {} (2,0) node (p4) {} (2.5,.5) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,0) node (p1) {} (1,0) node (p2) {}
  (1.5,.5) node (p3) {} (2,.5) node (p4) {} (2.5,.5) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,.5) node (p1) {} (1,0) node (p2) {}
  (1.5,.5) node (p3) {} (2,.5) node (p4) {} (2.5,.5) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,.5) node (p1) {} (1,1) node (p2) {}
  (1.5,1.5) node (p3) {} (2,1) node (p4) {} (2.5,.5) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,.5) node (p1) {} (1,1) node (p2) {}
  (1.5,1) node (p3) {} (2,1) node (p4) {} (2.5,.5) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,.5) node (p1) {} (1,0) node (p2) {}
  (1.5,.5) node (p3) {} (2,0) node (p4) {} (2.5,.5) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,.5) node (p1) {} (1,1) node (p2) {}
  (1.5,.5) node (p3) {} (2,1) node (p4) {} (2.5,.5) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,.5) node (p1) {} (1,.5) node (p2) {}
  (1.5,.5) node (p3) {} (2,.5) node (p4) {} (2.5,.5) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,.5) node (p1) {} (1,1) node (p2) {}
  (1.5,.5) node (p3) {} (2,.5) node (p4) {} (2.5,.5) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}
  \begin{tikzpicture}[every node/.style={draw,shape=circle,fill=black,inner sep=0pt,minimum size=3pt}]
  \path (0,0) node (p0) {} (.5,.5) node (p1) {} (1,.5) node (p2) {}
  (1.5,.5) node (p3) {} (2,1) node (p4) {} (2.5,.5) node (p5) {} (3,0) node (p6) { };
  \draw (p0) -- (p1) -- (p2) -- (p3) -- (p4) -- (p5) -- (p6);
  \draw (0,0) -- (0,1.5) -- (3,1.5) -- (3,0);
  \end{tikzpicture}

  \#1,2,3,4,5,6,6,8,9,12,14$\to$11 out of 22 have horizontal steps on $x$-axis
\end{enumerate}
\end{document}


\chapter{Interaction between an Atom and a Laser Field}
\label{AppendixLMI}

We consider an electromagnetic field with the vector and electrostatic potential
\begin{align}
  \vec{A}(\vec{r}, t)
  &= \frac{E_{0}}{\omega} \vec{e}_z \sin\left( ky - \omega t \right)\,, \\
  \Phi(\vec{r}, t) &= 0\,.
\end{align}
This corresponds to a cosine shape electromagnetic field propagating along the
$y$-axis, with the electric field oscillating in $z$-direction and the magnetic
field oscillating in $y$-direction,
\begin{align}
  \vec{E}(\vec{r}, t) &= - \frac{\partial}{\partial t} \vec{A}(\vec{r}, t)
                         - \vec{\nabla} \Phi(\vec{r}, t)
                       = E_0 \vec{e}_z \cos(ky - \omega t)\,, \\
  \vec{B}(\vec{r}, t) &= \vec{\nabla} \times \vec{A}(\vec{r}, t)
                       = \frac{\partial}{\partial y} A_z \vec{e}_x
                       = B_0 \vec{e}_x \cos(ky - \omega t)\,,
\end{align}
with $B_0 = \frac{E_0}{\omega} k = \frac{E_0}{c}$, where $k=\frac{\omega}{c}$,
$c$ is the speed of light and $\omega$ is the laser frequency.

The Hamiltonian for an atom's valence electron at position $\vec{r}$, with
electron mass $m$, electron-charge $q$, and $\vec{r}$ and $\vec{p}$ now being
operators, reads
\begin{equation}
\begin{split}
\Op{H}
  & = \frac{1}{2m} \left[
      \vecOp{p} - q \vecOp{A}(\vecOp{r}, t)
    \right]^2
    + \Op{V}(\vecOp{r})
    - \frac{q}{m} \vecOp{S} \cdot \vecOp{B}(\vecOp{r}, t)
    + \vec{\nabla} \Op{\Phi}(\vecOp{r}, t)
 \\ &
  = \Op{H}_0 - \frac{q}{m} \vecOp{p} \cdot \vecOp{A}
             - \frac{q}{m} \vecOp{S} \cdot \vecOp{B}
             + \frac{q}{2m} \left[ \vecOp{A}(\vecOp{r}, t)\right]^2\,,
  \label{eq:LMI_ham}
\end{split}
\end{equation}
with the electron's drift Hamiltonian
\begin{equation}
  \Op{H}_0 = \frac{\vecOp{p}^{\,2}}{2m} + \Op{V}(\vecOp{r})\,,
\end{equation}
and the spin operator $\vecOp{S}$ coupling to the magnetic field.
The origin of the coordinate system is in the atoms nucleus.
Since $\Norm{\vec{A}^{\,2}} \ll \Norm{\vec{A}}$ for realistic laser field
amplitudes, we set the last term to zero.

The spatial dependence of the vector potential, $ky = \frac{2\pi y}{\lambda}$,
where $y$ is on the order of an atomic radius $a_0$ and $\lambda$ is the
wavelength of the laser is extremely small. We can therefore Taylor-expand the
vector potential as
\begin{equation}
\begin{split}
  \vec{A}(\vec{r}, t)
  & = \frac{E_0}{w} \vec{e}_z \sin(ky - \omega t)
  \\
  & = \frac{E_0}{2 \ii \omega} \vec{e}_z \left(
        \ee^{\ii k y} \ee^{-\ii \omega t} - \ee^{-\ii k y} \ee^{\ii \omega t}
      \right)
  \\
  & \approx
      \frac{E_0}{2 \ii \omega} \vec{e}_z \left(
        (1 + \ii k y) \ee^{-\ii \omega t} - (1 - \ii k y) \ee^{\ii \omega t}
      \right)
  \\
  & = \frac{E_0}{w} \vec{e}_z \sin(\omega t) + B_0 y \vec{e}_z \cos(\omega t)\,.
\end{split}
\label{eq:LMI_taylor}
\end{equation}

Also, since an electron that is localized with a Bohr radius $a_0$
\index{Bohr radius}
must have a minimum momentum $\vecOp{p}$ such that $\frac{\hbar}{p} \le a_0$, and
$\vecOp{S}$ is on the order of $\hbar$, we can show
$\Norm{\vecOp{p} \cdot \vecOp{A}} \gg \Norm{\vecOp{S}\cdot \vecOp{B}}$,
\begin{equation}
  \frac{\Norm{\vecOp{S} \cdot \vecOp{B}}}{\Norm{\vecOp{p} \cdot \vecOp{A}} }
  \approx \frac{\hbar k E_0/\omega}{p E_0 / \omega}
  = \frac{\hbar k}{p}
  < \frac{a_0}{\lambda} \ll 1\,.
\end{equation}
Therefore, we are justified in approximating
\begin{equation}
  \vec{B}(\vec{r}, t) \approx B_0 \vec{e}_x \cos{\omega t}\,.
\end{equation}

Inserting this and Eq.~\eqref{eq:LMI_taylor} into Eq.~\eqref{eq:LMI_ham} yields
\begin{equation}
  \Op{H}
  \approx
    \Op{H}_0
    - \frac{q}{m} \frac{E_0}{\omega} \Op{p}_z \sin(\omega t)
    - \frac{q B_0}{m} B_0 \Op{p}_z \Op{y} \cos(\omega t)
    - \frac{q}{2m} \Op{S}_x B_0 \cos(\omega t)\,.
\end{equation}
Furthermore,
\begin{equation}
  \Op{p}_z \Op{y}
   = \frac{1}{2} \left(\Op{p}_z \Op{y} - \Op{z} \Op{p}_y\right)
     +\frac{1}{2} \left(\Op{p}_z \Op{y} - \Op{z} \Op{p}_y\right)
  = \frac{1}{2} \Op{L}_x +\frac{1}{2} \left(\Op{p}_z \Op{y}
     - \Op{z} \Op{p}_y\right)\,,
\end{equation}
resulting in
\begin{equation}
\begin{split}
  \Op{H}
 &
  \approx
    \Op{H}_0
    - \frac{q}{m} \frac{E_0}{\omega} \Op{p}_z \sin(\omega t)
    - \frac{q}{2 m c} E_0 \cos(\omega t)
      \left[ \Op{p}_z \Op{y} - \Op{z} \Op{p}_y\right]
 + \\ & \quad
    - \frac{q}{2m} \left(\Op{L}_x + \Op{S}_x\right) B_0 \cos(\omega t)\,.
\end{split}
\end{equation}

The three interaction terms are interpreted as follows:
\begin{itemize}[noitemsep]
  \item
  \begin{equation}
  \Op{H}_{ED} = \frac{q}{m} \frac{E_0}{w} \Op{p}_z \sin(\omega t)
  \end{equation}
  is the \emph{electric dipole} interaction in momentum space.
  \index{dipole moment!electric}
  It can be rewritten to its more familiar form in coordinate space
  \begin{equation}
  \Op{H}_{ED} = q \Op{z} \, E_0 \cos(\omega t) = \Op{\mu} E_0 \cos(\omega t)\,,
  \end{equation}
  where $\Op{\mu}$ has been introduced as the dipole operator.
  \index{dipole operator}
  \item
  \begin{equation}
    \Op{H}_{EQ} = - \frac{q}{2 m c} E_0 \cos(\omega t)
                    \left[ \Op{p}_z \Op{y} - \Op{z} \Op{p}_y\right]
  \end{equation}
  describes the \emph{electric quadrupole} interaction.
  \index{quadrupole moment!electric}
  \begin{item}
  \begin{equation}
    \Op{H}_{MD} = - \frac{q}{2m} \left(\Op{L}_x
                  + \Op{S}_x\right) B_0 \cos(\omega t)
  \end{equation}
  describes the \emph{magnetic dipole} interaction.
  \index{dipole moment!magnetic}
  \end{item}
\end{itemize}
Both the electric quadrupole and the magnetic dipole are negligible compared
to the electric dipole. Therefore, in the dipole-approximation the total
Hamiltonian becomes
\begin{equation}
  \Op{H} \approx \Op{H}_0 + \Op{\mu} E(t)\,.
  \label{eq:LMI_dipole_ham}
\end{equation}
The dipole approximation results from the assumption that the wavelength of the
laser is much larger than the width of the atom, and thus that the spatial
dependence of the field can be dropped, allowing to define the $z$-component of
the electric field as
\begin{equation}
  E(t)  = E_0 \cos(\omega t)\,.
\end{equation}

When Eq.~\eqref{eq:LMI_dipole_ham} is written in the energy representation given
by the eigenstates of $\Op{H}_0$, the selection rules for the dipole transitions
are obtained. That is, for certain quantum numbers the corresponding matrix
element of $\Op{\mu}$ will vanish. For a Hydrogen atom with eigenstates
$\Ket{nlm}$, the dipole is zero unless $\Delta l = 1$ and $\Delta m = 0, 1$.
\index{selection rules}

\enlargethispage{\baselineskip}
For example, we may consider
a Hamiltonian for a sub-system consisting of three levels $\Ket{0}$, $\Ket{1}$,
and $\Ket{2}$, with energies $E_0$, $E_1$, $E_2$. The dipole transition $\Ket{0}
\rightarrow \Ket{1}$ and $\Ket{2} \rightarrow \Ket{3}$ is allowed with
a resulting dipole moment of $\mu_{01} = \Braket{0|\Op{\mu}|1}$ and
$\mu_{12}=\Braket{1|\Op{\mu}|2}$, respectively, but
$\Ket{1} \rightarrow \Ket{3}$ is forbidden. This Hamiltonian
would we written in the energy representation as
\begin{equation}
  \Op{H} = \begin{pmatrix}
    E_0           & \mu_{01} E(t) &   0           \\
    \mu_{01} E(t) & E_1           & \mu_{12} E(t) \\
    0             & \mu_{12} E(t) & E_2           \\
  \end{pmatrix}\,,
\end{equation}
the form used for the Hamiltonians e.g.\ in chapters~\ref{chap:robust}
and~\ref{chap:3states}.


\label{sec:experiments}

\subsection{Toy Problem -- Annulus}
\label{sec:annulus}
We first demonstrate our approach on a toy problem.
The true generative model of the observed data is a constant speed circular orbit around the origin in the $x$-$y$ plane, such that $\mathbf{x}_t = \left\lbrace x_t, y_t, \dot{x}_t, \dot{y}_t \right\rbrace \in  \mathbb{R}^4$.
To analyze this data we use a misspecified model that only simulates linear forward motion.
To overcome the model mismatch and fit the observed data, we add Gaussian noise to position and velocity.
We impose a failure constraint limiting the change in the distance of the point from the origin to a fixed threshold.
This condition mirrors our observation that states in brittle simulators have large allowable perturbations in particular directions, but very narrow permissible perturbations in other directions.
The true radius is unknown and so we must amortize over possible radii.

\input{figures/figure_3}

The results of this experiment are shown in Figure~\ref{fig:ring}.
The interior of the black dashed lines in Figure~\ref{fig:ring:space} indicates the permissible $\dot{x}$-$\dot{y}$ perturbation, for the given position and zero velocity, where we have centered each distribution on the current position for ease of visual inspection.
Red contours indicate the original density $p(\mathbf{z}_t | \mathbf{x}_{t-1})$, and blue contours indicate the learned density $q_{\phi}(\mathbf{z}_t | \mathbf{x}_{t-1})$.
The fraction of the probability mass outside the black dashed region is the expected rejection rate.
Figure~\ref{fig:ring:ar} shows the rejection rate drops from approximately $75\%$ under the original model to approximately $4\%$ using a trained $q_{\phi}$.

\input{figures/figure_4}

We then use the learned $q_{\phi}$ as the perturbation proposal in an SMC sweep, where we condition on noisy observations of the $x$-$y$ coordinates.
As we focus on the sample efficiency of the sweep, we fix the number of calls to the simulator in Algorithm \ref{alg:rs} to a single call, instead of proposing and rejecting until acceptance.
Failed particles are then not resampled (with certainty) during the resampling.
This means that each iteration of the SMC makes a fixed number of calls to the simulator, and hence we can compare algorithms under a fixed sample budget.
Figure \ref{fig:ring:smc:var} shows that we recover lower variance evidence approximations for a fixed sample budget by using $q_{\phi}$ instead of $p$.
A paired t-test evaluating the difference in variance returns a p-value of less than $0.0001$, indicating a strong statistical difference between the performance under $p$ and $q_{\phi}$, confirming that using $q_{\phi}$ increases the fidelity of inference for a fixed sample budget.

\subsection{Bouncing Balls}
\label{sec:experiments:bb}
Our second example uses a simulator of balls bouncing elastically, as shown in Figure \ref{fig:balls:trajectory}.
We model the position and velocity of each ball, such that the dimensionality of the state vector, $\mathbf{x}_t$, is four times the number of balls.
We add a small amount of Gaussian noise at each iteration to the position and velocity of each ball.
This perturbation induces the possibility that two balls overlap, or, a ball intersects with the wall, representing an invalid physical configuration and results in simulator failure.
We note that here, we are conditioning on the state of \emph{all} balls simultaneously, and proposing the perturbation to the state \emph{jointly}.

Figure \ref{fig:balls:space} shows the distribution over position perturbation of a single ball, conditioned on the other ball being stationary.
Blue contours show the estimated distribution over accepted perturbations learned by autoregressive flow.
Figure \ref{fig:balls:rr} shows the rejection rate under $p$ and $q_{\phi}$ as a function of the position of the first ball, with the second ball fixed in the position shown, showing that rejection has been all but eliminated.
We again see a reduction in the variance of the evidence approximation computed by a particle filter when using $q_{\phi}$ instead of $p$ (figure in the supplementary materials).

\subsection{MuJoCo}
\label{sec:experiments:tosser}
We now apply our method to the popular robotics simulator MuJoCo~\citep{todorov2012mujoco}, specifically using the built-in example ``tosser,'' where a capsule is ``tossed'' by an actuator into a bucket, shown in Figure \ref{fig:tosser:im}.
Tosser displays ``choatic'' aspects, as minor changes in the position of the object results in large changes in the trajectories achieved by the simulator.

MuJoCo allows some overlap between the objects to simulate contact dynamics.
This is an example of model misspecification borne out of the requirements of reasonably writing a simulator.
We therefore place a hard limit on the amount objects are allowed to overlap.
This is an example of a user-specified constraint that requires the simulator to be run to evaluate.
We add Gaussian distributed noise to the position and velocity of the capsule.

\input{figures/figure_5.tex}

Figure \ref{fig:tosser} shows the results of this experiment.
The capsule is mostly in free space resulting in an average rejection rate under $p$ of $10\%$.
Figure \ref{fig:tosser:ar} shows that the autoregressive flow learns a proposal with a lower rejection rate, reaching $3\%$ rejection.
However these rejections are concentrated in the critical regions of state-space, where chaotic behavior occurs, and so this reduction yields an large reduction in the variance of the evidence approximation, as shown in Figure \ref{fig:tosser:smc:var}.

\input{figures/figure_6.tex}

We conclude this example by evaluating our method on hypothesis testing using pseudo-marginal evidence estimates.
The results for this are shown in Figure \ref{fig:tosser_hyp}.
We test $5$ different hypothesis of the mass of the capsule.
Using $p$ results in higher variance evidence approximations than when $q_{\phi}$ is used.
Additionally, under $p$ the wrong model is selected ($2$ instead of $3$), although with low significance ($p=0.125$), while using $q_{\phi}$ selects the correct hypothesis with $p=0.0127$.
For this experiment we note that $q_{\phi}$ was trained on a single value of mass, and that this ``training mass'' was different to the ``testing mass.''
We believe this contributes to the increased variance in hypothesis $1$, which is very light compared to the training mass.
Training a $q_{\phi}$ with a further level of amortization over different mass values would further increase the fidelity of the model selection.
This is intimately linked with the larger project of jointly learning the model, and so we defer investigation to future works.

\subsection{Neuroscience Simulator}
\label{sec:sub:wormsim}
We conclude by applying our algorithm to a simulator for the widely studied \emph{Caenorhabditis elegans} roundworm.
WormSim, presented by \citet{boyle2012gait}, is a simulator of the locomotion of the worm, using a $510$ dimensional state representation.
We apply perturbations to a $98$ dimensional subspace defining the physical position of the worm, while conditioning on the full $510$ dimensional state vector.
The expected rate of failure increases sharply as a function of the scale of the perturbation applied, as shown in Figure \ref{fig:wormsim:bot_rate}, as the integrator used in WormSim is unable to integrate highly perturbed states.

The rejection rate during training is shown in Figure \ref{fig:wormsim:bot_nf_rate}.
We are able to learn an autoregressive flow with lower rejection rates, reaching approximately $53\%$ rejection, when $p$ has approximately $75\%$ rejection.
Although the rejection rate is higher than ultimately desired, we include this example as a demonstration of how rejections occur in simulators through integrator failure.
We believe larger flows with regularized parameters can reduce the rejection rate further.


\section{Eager Typechecking}

There are two variations on eager typechecking.
They both implement pure Canonical LF, in the sense that
terms remain canonical throughout the typechecking process.
(We will admit non-canonical terms during lazy typechecking.)

The difference between the variations lies in the treatment
of substitutions. The first variation
considers substitutions as either a $\Shift^n$ or a $M\cdot\sigma$.
The second adds a ``lazy'' composition $\sigma_1\Comp\sigma_2$.
We write the rules for both versions together, as the majority
are identical.  Where there are extra rules needed for the
extra case of a substitution, they will be labelled as such.

%-------------------------------------------------------------------------------
% Terms
%-------------------------------------------------------------------------------

\subsection{Terms}

$$
\begin{array}{llll}
\mathbf{Levels} & L & ::= & \Type \Spb \Kind \\
\mathbf{Expressions} & U,V & ::= & L \Spb \PiTyp{U_1}{U_2} \Spb \lambda U \Spb H\cdot S \\
\mathbf{Heads} & H & ::= & c \Spb i\\
\mathbf{Spines} & S & ::= & \Nil \Spb U;S\\
\mathbf{Eager\ Substitutions} & \sigma & ::= & M\cdot\sigma \Spb \Shift^n \\
\mathbf{Lazy\ Substitutions} & \sigma & ::= & M\cdot\sigma \Spb \Shift^n \Spb \sigma_1 \Comp \sigma_2\\
\end{array}
$$

%-------------------------------------------------------------------------------
% Typecheck
%-------------------------------------------------------------------------------

\subsection{Typechecking}

\bigskip
\framebox{$\CheckTy{U}{V}$}
\bigskip

$$
\begin{array}{cc}
\infer{\CheckTy{\Type}{\Kind}}{} &
\infer{\CheckTy{\PiTyp{A}{U}}{V}}{\CheckTy{A}{\Type} & \CheckTy[\Gamma,A]{U}{V}}\\\\
\infer{\CheckTy{c\cdot S}{V}}{\Sigma(c) = U & \Focus{S}{U}{V'} & \Equiv{V'}{V}} &
\infer{\CheckTy{\Lam{M}}{\PiTyp{A_1}{A_2}}}{\CheckTy[\Gamma,A_1]{M}{A_2}} \\\\
\infer{\CheckTy{i\cdot S}{A_2}}{\Gamma(i)=A_1 & \Focus{S}{A_1}{A_2'} & \Equiv{A_2'}{A_2}}
\end{array}
$$

\bigskip

\begin{Note}\label{context:shift}
Note that you must shift the type you extract from $\Gamma$, as the
free variables (indices) should point to the slots before $i$.  Moving
the type $A$ from the context to the consequent must adjust the pointers.
We thus define $\Gamma(i) = A$ as the $i$th element of $\Gamma$
under $\Shift^i$.
\end{Note}

\bigskip
\framebox{$\Focus{S}{U}{V}$}
\bigskip

$$
\begin{array}{lr}
\infer{\Focus{\Nil}{\Type}{\Type}}{} &
\infer{\Focus{\Nil}{P}{P}}{} \\\\
\infer{\Focus{(M;S)}{\PiTyp{A}{U}}{V}}{\CheckTy{M}{A} & \Focus{S}{U[M\cdot\IdSub]}{V}}
\end{array}
$$

%-------------------------------------------------------------------------------
% Substitutions
%-------------------------------------------------------------------------------

\subsection{Substitutions}

(The notation $\Shift$ means $\Shift^1$, and $\IdSub$ means $\Shift^0$.)

We \emph{apply} subsitutions to terms.

\bigskip
\framebox{$U\Msub = U'$}

\begin{align*}
\Type\Msub &= \Type \\
(\PiTyp{A}{U})\Msub &= \PiTyp{(A\Msub)}{(U\Ssub)}\\
(c\cdot S)\Msub &= c\cdot (S\Msub) \\
(\Lam{M})\Msub &= \Lam{(M\Ssub)}\\
(i\cdot S)\Msub &= \begin{cases}
                     j\cdot S\Msub \mbox{\ if $i\Msub = j$} \\
                     M \App S\Msub\mbox{\ if $i\Msub = M$}
                   \end{cases}
\end{align*}

\framebox{$S\Msub = S'$}

\begin{align*}
\Nil\Msub &= \Nil\\
(M;S)\Msub &= M\Msub;S\Msub
\end{align*}

\framebox{$i\Msub = M$}
\bigskip

This judgment is the first place we distinguish between
the different notions of substitution.  The rule (*)
holds only for the second variant.

\begin{align*}
1[M\cdot\sigma] &= M\\
n+1[M\cdot\sigma] &= n[\sigma]\\
i[\Shift^n] &= i+n\\
i[\sigma_1\Comp\sigma_2] &= (i[\sigma_1])[\sigma_2]\tag{*}
\end{align*}

We still need the notion of beta reduction when a
head gets instantiated with a lambda.  We show
only the possible cases.

\bigskip
\framebox{$M \App S = M'$}

\begin{align*}
(H\cdot S)\App\Nil &= H\cdot S\\
\Lam{M}\App(M';S) &= M[M'\cdot\IdSub]\App S
\end{align*}

In the first case of eager typechecking, composition doesn't
have a syntactic existence.  Thus we need to carry out all
compositions eagerly.  The rules for composing substitutions are:

\bigskip
\framebox{$\sigma\Comp\sigma' = \sigma''$}

$$
\begin{array}{llll}
(M\cdot \sigma) & \Comp \sigma' &= &M[\sigma']\cdot (\sigma\Comp\sigma') \\
\Shift^n & \Comp \Shift^m &= &\Shift^{n+m}\\
\Shift^0 & \Comp \sigma &= &\sigma\\
\Shift^{n+1}&\Comp (M\cdot\sigma) &= &\Shift^n\Comp\sigma
\end{array}
$$

%-------------------------------------------------------------------------------
% Equivalence
%-------------------------------------------------------------------------------

\subsection{Equivalence}

If we only allowed constant declarations in a signature then checking equivalence
of terms would be a simple matter of checking syntactic equality.
With definitions of the form $c : A = M$, we must account
for the fact that a focusing phase might return a type $A$ to
check against a type $A'$ that are not syntactically equal, but
if one expanded all the definitions and normalized the resulting
terms than they would be identical.  We thus need a judgment for the
equivalence of types $A$ and terms $M$.  (Since we are not allowing
type level definitions, we do not need to check for equivalent kinds.)

We use the judgment $c\StepsTo M$  to mean
that the constant $c$ has definition $M$.

\bigskip
\framebox{$\Equiv{U}{U'}$}
\bigskip

$$
\begin{array}{lcr}\
\infer{\Equiv{U}{U'}}{\Equiv{U'}{U}} &
\infer{\Equiv{\PiTyp{U_1}{U_2}}{\PiTyp{U_1'}{U_2'}}}{\Equiv{U_1}{U_1'} & \Equiv{U_2}{U_2'}} &
\infer{\Equiv{c\cdot S}{c\cdot S'}}{\Equiv{S}{S'}} \\\\
\infer{\Equiv{\Lam{M}}{\Lam{M'}}}{\Equiv{M}{M'}} &
\infer{\Equiv{i\cdot S}{i\cdot S'}}{\Equiv{S}{S'}} &
\infer{\Equiv{c\cdot S}{M}}{c\StepsTo M' & \Equiv{M'@S}{M}}
\end{array}
$$

\bigskip
\framebox{$\Equiv{S}{S'}$}
\bigskip

$$
\begin{array}{lcr}
\infer{\Equiv{\Nil}{\Nil}}{} &
\infer{\Equiv{M;S}{M';S'}}{\Equiv{M}{M'} & \Equiv{S}{S'}}
\end{array}
$$

\subsection{A Note on Implementing Equivalence Checking}

  For various reasons, we need equivalence checking to
be as fast as possible.  Equivalence checking is complicated
by notational definitions.  If checking $A=B$ fails, we might need
to expand definitions in one or both terms.  One could simply
expand all definitions to yield a sound algorithm, but this would
be horribly slow.

  The solution given by Twelf, suggested by
Pfenning and Reed, is to store two extra bits of information
with each constant.  The first is the \emph{height} of a constant.
This merely records the definition depth of a constant.  Constants that
do not refer to other constants have height 0.  A constant $c$ that refers
to others has height $1 + \max\Set{\mbox{height of constants occuring in } c}$.
Note that only constants with the same height can be equal.
The second bit of data is the \emph{root}, it{i.e.} the head of the term that would be obtained by
expanding all definitions.  Note that any two equal terms must have the
same root after full expansion.

These two bits of data yield a natural algorithm for determining
equivalence of terms.  If, when checking equality of $A=B$,
we find that two constants must be equal for $A,B$ to
be equivalent, but are not syntactically equal,
we check to see if the roots are the same.
If not, we fail.  Otherwise, we check the heights.  If the heights
differ, we expand the constant with the greater height until
they are equal, and check again.  Once the levels
are equal, while the constants are still distinct, we
expand both definitions, level by level, until all constants
are expanded.  In the worst case, this will take as much time
as expanding all the definitions.  In the usual case, however,
where the constants differ\footnote{indeed, unification fails
around \%80 of the time}, the clash will be found long before
the terms are fully expanded.

This leads to an additional rule

$$
\begin{array}{cccc}
\infer{\Equiv{c\cdot S}{c'\cdot S'}}{\Root(c)=\Root(c') & \Card{c} \geq \Card{c'} & c\StepsTo M' & \Equiv{M'@S}{c'\cdot S'}}
\end{array}
$$

where $\Card{c}$ is the height of $c$, and $\Root(c)$ is the root.
This holds in both the eager and lazy cases.


\documentclass[10pt]{article}
\usepackage[T1]{fontenc}

% Document Details
\newcommand{\CLASS}{AMATH 563}
\newcommand{\assigmentnum}{Multiple Resolution Analysis}

\usepackage[margin = 1in, left=0.75in,right=0.75in]{geometry}
\input{../../import/title.tex} % Title Styling
\input{../../import/styling.tex} % General Styling
\input{../../import/code.tex} % Code Display Setup
\input{../../import/math.tex} % Math shortcuts

\usepackage{dblfloatfix}    % To enable figures at the bottom of page

% Problem
\newenvironment{problem}[1]{\vspace{2em}{\large\sffamily\textbf{#1}}\itshape\par}{}

\usepackage{nameref}
\newcommand{\vln}{\rotatebox{90}{--}}

\begin{document}

\twocolumn[{%
\begin{@twocolumnfalse}
\maketitle
\vspace{2em}
\begin{abstract}
We introduce Multiple Resolution Analysis and the Discrete Wavelet Transform. We then discuss how these might be applied to a data set of global ocean temperatures over the past few decades in order to isolate El Ni\~no. We find that the Oceanic Ni\~no Index can be recovered using just a single spatial point. Finally, we train a neural net to predict the next time the ONI will reach a certain value.
\end{abstract}

\vspace{4em}
%\tableofcontents
%\vspace{3em}
\pagebreak\end{@twocolumnfalse}
}]

\section{Introduction and Overview}
Multiple Resolution Analysis (MRA) provides a way to view data on multiple scales in both the position and frequency domains simultaneously.

The El Ni\~no Southern Oscillator (ENSO) is an ``irregularly periodic varaition in winds and sea surface temperatures over the tropical easter Pacific Ocean''. The warm phase is referred to as El Ni\~no and the cold phase is referred to as La Ni\~na. When the ENSO is at its extremes, there is often extreme weather throughout part of the world.

We apply MRA to a ocean surface temperature data set in the hopes of extracting information useful in predicting El Ni\~no.

\section{Theoretical Background}

\subsection{Multiple Resolution Analysis (MRA)}
Broadly, Multiple Resolution Analysis provides a way to obtain information at multiple scales (as the name suggests). We illustrate this with an example.

Suppose we have an audio recording of a piano. The recording tells us the amplitude of the sounds waves at a given time. What if we want to know which notes were played? We could take the Fourier transform of our recording and get information about the frequencies present. However, unless the notes were played through the entire recording, this would not be very useful.

Multiple resolution analysis lets us get information about what notes were played when. Low frequencies, which have periods comparable to the entire recording can be extracted, but without timing information, while high frequencies can be pinpointed in time. This example illustrates the basic principle of MRA.

\begin{figure*}[h]\centering
\foreach \i in {0,...,7}{
\begin{subfigure}{.24\textwidth}\centering
\includegraphics[width=\textwidth]{img/haar/basis_\i.pdf}
\end{subfigure}
}
\caption{Haar basis with \( N=8 \)}
\label{haar}
\end{figure*}

\subsection{Discrete Wavelet Transform}
One way to get time frequency information is to split the data into many pieces and take the Fourier transform of each piece. This would tell us information about which frequencies are present, as well as when these frequencies are present. Such a decomposition is often referred to as a spectogram or Short Time Discrete Fourier Transform (STDFT).

In the discrete case it is easy to see that such a transform is just a change of basis. Figure~\ref{haar} shows a possible basis for \( \RR^8 \). This basis gives us information about 4 different frequencies, as well as information about where these frequencies occur in time.

From this figure it is obvious that these functions are orthogonal, and could be appropriately normalized. It is also clear that such a basis can be generalized to any \( 2^k \)-dimensional vectorspace over \( \RR \). This basis has some advantages over a STDFT, namely that it is able to pick up low frequency modes which have a period longer than the windows in a STDFT.

This approach can be generalized to to a a class of transforms commonly referred to as the Discrete Wavelet Transform. While we show only a class of ``wavelets'' calleed ``Haar Wavelets'', many other shapes can be used. Note that the same ``wavelet'' seen in the first non-constant mode is just shifted and scaled to produce an orthonormal basis. Therefore it should not be surprising that there are many possible wavelets which are used in practice.

Like the Discrete Fourier Transform, there are efficient algorithms for computing the DWT for many wavelets.

Such a transform can be applied to vector-spaces where the vectors are multi-dimensional arrays (for instance a photo).
More specifically, suppose our data lives in \( \RR^{k_1\times k_2\times\cdots\times k_d} \).

Let \( \mB_i = \{b_{i1},b_{i2},\ldots, b_{ik_i}\} \) be a wavelet basis for \( \RR^{k_i} \). Then,
\begin{align}
    \mB = \mB_1\otimes \mB_2 \otimes \cdots \otimes \mB_d
\end{align}
is a basis for \( \RR^{k_1\times k_2\times\cdots\times k_d} \), where \( \otimes \) denotes the tensor product. In the 2-dimensional case this is simply the collection of all possible outer products of the elements of the bases for each dimension.

Such a change of basis would give information about the position and frequency/wave number of the data in each dimension. For instance, as discussed later, information about the location of a ocean temperate event in both space (longitude and latitude) and time along with information about frequency (in both spatial dimensions and in time) can be gathered.

Such a change of basis is not practical to compute for an arbitrary wavelet basis using naive algorithms since the dimension of the vector-space would generally be quite large. In particular, computing the standard inner product of two vectors of size \( N \) requires \( \mO(2N) \) operations. Even if these vectors can be accessed in constant time, a change of basis would require \( N \) inner products to project the data onto each of the \( N \) new basis vectors. Therefore it will require something like \( \mO(N^2) \) operations to compute the change of basis. With vectors with millions of components this is not practical.

There are almost certainly better algorithms for certain bases to help deal with this problem.

\subsection{SVD}
We can also perform MRA using the SVD.

Given any vector of the size of a single snapshot of our data we can project our data onto this vector and see how this ``mode'' varies in time. That means, if we are able to isolate a vector which represents El Ni\~no we would be able to see hot his varies in time.

\subsection{Filtering}
Filtering is a common technique in signal processing where the frequency content of a signal is altered. A low pass filter maintains the low frequencies of a signal and attenuates the high frequencies of a signal. This can be useful for de-noising a signal or removing a high frequency periodic piece of a signal.

\subsection{Oceanic Ni\~no Index}

The Oceanic Ni\~no Index (ONI) is a 3 month running mean of ERSST.v5 SST anomalies in the Ni\~no 3.4 region (5\( ^\circ \)N-5\( ^\circ \)S, 120\( ^\circ \)-170\( ^\circ \)W) \cite{oni}. This gives a measure of El Ni\~no events.

\begin{figure*}[bh]\centering
\foreach \j in {0,...,7}{
\begin{subfigure}{.24\textwidth}\centering
\includegraphics[width=\textwidth]{img/svd/1455/u_\j.pdf}
\end{subfigure}
}
\caption{First 8 modes of SVD of full data (all with same color scale)}
\label{full_svd_u}
\end{figure*}

\begin{figure*}[bh]\centering
\foreach \j in {0,...,7}{
\begin{subfigure}{.24\textwidth}\centering
\includegraphics[width=\textwidth]{img/svd/1455/vh_\j.pdf}
\end{subfigure}
}
\caption{Weights of first 8 modes of SVD against time}
\label{full_svd_vh}
\end{figure*}

\begin{figure*}[t]\centering
\begin{subfigure}{.48\textwidth}\centering
\foreach \j in {0,...,3}{
\includegraphics[width=.48\textwidth]{img/svd/smallMN/u_\j.pdf}
}
\end{subfigure}
\begin{subfigure}{.48\textwidth}\centering
\foreach \j in {0,...,3}{
\includegraphics[width=.48\textwidth]{img/svd/smallMN/vh_\j.pdf}
}
\end{subfigure}
\caption{First 4 modes of SVD of data localized in space (all with same color scale) and corresponding weights in time}
\label{smallMN_svd}
\end{figure*}

\begin{figure*}[t]\centering
\begin{subfigure}{.48\textwidth}\centering
\foreach \j in {0,...,3}{
\includegraphics[width=.48\textwidth]{img/svd/shortT/u_\j.pdf}
}
\end{subfigure}
\begin{subfigure}{.48\textwidth}\centering
\foreach \j in {0,...,3}{
\includegraphics[width=.48\textwidth]{img/svd/shortT/vh_\j.pdf}
}
\end{subfigure}
\caption{First 4 modes of SVD of data localized in time (all with same color scale) and corresponding weights in time}
\label{shortT_svd}
\end{figure*}
\section{Algorithm Implementation and Development}
We create some basic functions to help with out analysis of our data set.\footnote{I would have preferred to just put this section in with the computational results section, but I wasn't sure how strict the formatting rules Nathan sent out are.}

\subsection{Discrete Wavelet Transform}
We implement a function to generate a wavelet basis of arbitrary size given a mother wavelet. Using these we can construct a basis for our data. However, since the dataset we are working with has over 94 million points, it is not practical at all to use our naive implementation to compute the coefficients in the wavelet basis.

Our function assumes that there are \( N = 2^k \) points in the data set for \( k\in\NN \). A wavelet with period equal to the number of points is then generated. The points are then split, and two wavelets, each with period equal to \( N/2 \) but starting positions at 0 and \( N/2 \). This process is repeated. We note that we do define our function recursively, but rather using an appropriately indexed loop.

This function could be used to generate bases for a multi dimensional DWT, however then all of the basis functions would have to be stored.

\subsection{Singular Value Decomposition}
We write a function to take the SVD of a slice of our data. In particular, the array is reshaped so that the spatia axes lie on a single axis prior to taking the SVD.

\subsection{Filtering}
We use Scipy's signal module for filtering.

\subsection{Prediction}
Being able to predict El Ni\~no events would be of great use as it would allow farmers and others whose livelihoods depend on global climate events to prepare for upcoming changes. In past assignments we have explored methods of predicting time-series data including sparse regression and neural nets.

\section{Computational Results}
We are given ocean surface temperature data of size \( (M,N,T) = (180,360,1455) \). Each time point is taken one week apart starting on December 31, 1989 and ending on November 12, 2017.

\subsection{Discrete Wavelet Transform}
We attempt to project a smaller portion of our data. This was inefficient and we did not pursue this approach. Our code is usable in the sense that it will provide the desired information, however it is still a quadratic time algorithm which seemed to be running too slowly to justify spending more time on this approach.

\subsection{Singular Value Decomposition}
The SVD of the entire data set is taken. The 8 most dominant modes are shown in Figure~\ref{full_svd_u}. The corresponding weights of these modes in time are shown in Figure~\ref{full_svd_vh}. It is clear that the 5th mode more or less tracks the ONI.

We now perform some rudimentary MRA by observing a spatial slice of the data over the full time range as well as a temporal slice of the data over the whole earth. Neither give particularly interesting results. More specifically, in space we isolated a region of the Pacific Ocean off the coast of South America near the Ni\~no 3.4 region and in time we isolated the year 1997 which was a particularly strong El Ni\~no event.

Figures~\ref{smallMN_svd} shows the modes and how they vary in time when we localize in space. We see that the coefficients of many of these modes track the ONI well.
Figures~\ref{shortT_svd} shows the modes and how they vary in time when we localize in time. We see that the more important modes of the SVD look more like what we expect the El Ni\~no to look like.

While these results were initially encouraging, as discussed later, it turns out that even a single point from this region is enough to track the ONI and so this is not so interesting.

\subsection{Direct Filtering}
The region of the 7th mode of the SVD in the Pacific ocean off the coast of South America looks like what we might expect the El Ni\~no to look like. We isolate the desired portion of this mode and zero the rest of the data. A small Gaussian blur is then applied to clear the harsh boundaries of the clipping.

The dataset is then projected onto a normalized version of this mode. We look at the coefficients of this mode in time and find that we are more or less able to recover the The Oceanic Ni\~no Index (ONI) (with a low pass filter).

With this as a motivation we try an even simpler vector taking just a single point \( (90,245) \) corresponding to the position 0\(^\circ\)S, 145\(^\circ\)W in the center of the Ni\~no 3.4 region. Again a low pass filter is applied and the ONI is recovered. This is shown in Figure~\ref{elnino}.

\begin{figure}[h]\centering
%\begin{subfigure}{.45\textwidth}\centering
%    \includegraphics[width=\textwidth]{img/el_nino_mode.pdf}
%\end{subfigure}\hfill
\begin{subfigure}{.45\textwidth}\centering
    \includegraphics[width=\textwidth]{img/el_nino_filtered.pdf}
    \caption{temperature at (90,245) (dark grey), filtered temperature (black) ONI data (light grey fill)}
\end{subfigure}\hfill
\begin{subfigure}{.45\textwidth}\centering
    \includegraphics[width=\textwidth]{img/el_nino_wavelet_transform.pdf}
    \caption{CWT of temperature at (90,245) (ricker wavelet)}
\end{subfigure}
\caption{Analysis of temperature at (90,245)}
\label{elnino}
\end{figure}

\subsection{Prediction}
We train a neural net to predict the next time the ONI will be above a fixed threshold. To build the target data we first detect when the ONI reaches the specified threshold. These points are marked, and then the time to reach the next one of these points is found for each time in the given data set. We then train a neural network with the input being the current global ocean temperatures and the output being the next time until the ONI reaches the specified threshold using 5\% of the data for cross validation.

As before, since we do not have much time to train a network, and since we do not understand how to pick the proper network architecture, our results are not notable. That said everything we did was designed to be easily scaleable so that it could be adjusted in the future. This seems like an example of where a convolutional neural net might be useful.

\section{Summary and Conclusions}

The Discrete Wavelet Transform seems to provide a clean way of detecting and analyzing multi-scale phenomena without and sort of supervision. Given more time we would like to write an efficient algorithm to compute this transform. At the very least rather than projecting using inner products, slicing could be use for some wavelets. This would be more efficient in terms of floating point operations, and given fast memory access might prove feasible on our computing resources. Alternatively, and perhaps more practically , the algorithm could be implemented for parallel computation. Projecting onto a basis amounts to computing many independent inner products, which is the perfect type of problem for a GPU or large cluster. Parallelization would allow the fast computation of the projection onto any basis.

The Oceanic Ni\~no Index seems to be almost completely recoverable from a single point. This suggests that the definition of the index could be tightened. However, perhaps taking recordings over a larger area gives more precision.

Finally, we train a neural net to predict the next time the ONI will reach a certain threshold. Due to the limited time this was not very successful. However, our code is easily scaleable if we were to have more time to invest in the project.

\bibliographystyle{plain}
\bibliography{hw3}

\onecolumn
\section{Appendix A}
\label{AppendixA}
\lstinputlisting[linerange=\#<start:generate_wavelets>-\#<end:generate_wavelets>]{wavelettest.py}
\lstinputlisting[linerange=\#<start:svd_slice>-\#<end:svd_slice>]{hw3.py}

%\pagebreak
\section{Appendix B}
\lstinputlisting[linerange=\#<start:wavelet_decomp>-\#<end:wavelet_decomp>]{wavelettest.py}
\lstinputlisting[]{hw3.py}

\end{document}


% Created 2021-07-06 Tue 09:03
% Intended LaTeX compiler: pdflatex
\documentclass[presentation,aspectratio=169]{beamer}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{grffile}
\usepackage{longtable}
\usepackage{wrapfig}
\usepackage{rotating}
\usepackage[normalem]{ulem}
\usepackage{amsmath}
\usepackage{textcomp}
\usepackage{amssymb}
\usepackage{capt-of}
\usepackage{hyperref}
\usepackage{khpreamble}
\usepackage{amssymb}
\DeclareMathOperator{\shift}{q}
\DeclareMathOperator{\diff}{p}
\usetheme{default}
\author{Kjartan Halvorsen}
\date{2021-07-06}
\title{Root locus}
\hypersetup{
 pdfauthor={Kjartan Halvorsen},
 pdftitle={Root locus},
 pdfkeywords={},
 pdfsubject={},
 pdfcreator={Emacs 26.3 (Org mode 9.4.6)},
 pdflang={English}}
\begin{document}

\maketitle

\section{Pole placement}
\label{sec:org9c42cbf}
\begin{frame}[label={sec:orgf8465c7}]{Pole-placement and time-response}
\alert{Pair the pole-placement with the correct time-response (continuous time)!}
\begin{columns}
\begin{column}{0.4\columnwidth}
\begin{center}
\includegraphics[width=\linewidth]{../../figures/pzmap-apollo}
\end{center}
\end{column}
\begin{column}{0.6\columnwidth}
\begin{center}
\includegraphics[width=\linewidth]{../../figures/step-response-apollo}
\end{center}
\end{column}
\end{columns}
\end{frame}

\begin{frame}[label={sec:org6dc2ec2}]{Mapping of poles from continuous time to discrete time}
\begin{center}
\begin{tabular}{ll}
Continuous time & Discrete time\\
\hline
\(Y(s) \triangleq \laplace{y(t)}\) & \(Y(z) \triangleq \ztrf{y(kh)}\)\\
\(Y(s) = G(s)U(s) = \frac{b}{s+a}U(s)\) & \(Y(z) = H(z)U(z) = \frac{\beta}{z+\alpha}U(z)\)\\
Pole of the system: \(s+a=0 \; \Rightarrow \; s = -a\) & Pole of the system: \(z+\alpha = 0 \; \Rightarrow \; z = -\alpha\)\\
\includegraphics[width=0.22\linewidth]{../../figures/cont-stable} & \includegraphics[width=0.22\linewidth]{../../figures/discrete-stable}\\
\hline
\end{tabular}
\end{center}

The \alert{s-domain} of continuous-time systems is related to the \alert{z-domain} of discrete-time systems through  \[z = \mathrm{e}^{sh}\]
\end{frame}

\begin{frame}[label={sec:orge1375d9}]{Pole-placement and time-response}
Pair the pole-placement with the correct time-response (discrete time)!
\begin{columns}
\begin{column}{0.4\columnwidth}
\begin{center}
\includegraphics[width=\linewidth]{../../figures/pzmap-discrete-apollo}
\end{center}
\end{column}
\begin{column}{0.6\columnwidth}
\begin{center}
\includegraphics[width=\linewidth]{../../figures/step-reponse-discrete-apollo}
\end{center}
\end{column}
\end{columns}
\end{frame}

\section{Root locus}
\label{sec:orgd424d23}

\begin{frame}[label={sec:org7677301}]{Root locus: A brief review}

\begin{center}
  \begin{tikzpicture}[node distance=22mm, block/.style={rectangle, draw, minimum width=15mm}, sumnode/.style={circle, draw, inner sep=2pt}]

    \node[coordinate] (input) {};
    \node[sumnode, right of=input, node distance=16mm] (sum) {\tiny $\Sigma$};
    \node[block, right of=sum, node distance=20mm] (plant)  {$H(z)=\frac{B(z)}{A(z)}$};
    \node[block, below of=plant, node distance=12mm] (controller)  {$F(z)=K\frac{D(z)}{C(z)}$};
    \node[coordinate, right of=plant, node distance=30mm] (output) {};

    \draw[->] (input) -- node[above, pos=0.3] {} (sum);
    \draw[->] (sum) -- node[above] {} (plant);
    \draw[->] (plant) -- node[coordinate] (measure) {} node[above, near end] {} (output);
    \draw[->] (measure) |- (controller);
    \draw[->] (controller) -| (sum);
  \end{tikzpicture}
\end{center}

\begin{itemize}
\item The loop pulse-transfer function (loop gain) becomes \(L(z) = H(z)F(z) = K\frac{\overbrace{B(z)D(z)}^{Q(z)}}{\underbrace{A(z)C(z)}_{P(z)}} = K \frac{Q(z)}{P(z)}\).
\item The roots of \(Q(z)\) are called the \alert{open loop zeros}.
\item The roots of \(P(z)\) are called the \alert{open loop poles}.
\item The characteristic equation for the closed-loop system is \[ 1 + K\frac{Q(z)}{P(z)} = 0 \quad \Leftrightarrow \quad P(z) + KQ(z) = 0\]
\end{itemize}
\end{frame}

\begin{frame}[label={sec:org4d6f28e}]{Root locus: Definition}
Let
\[\begin{cases} P(z)&=z^n+a_1z^{n-1}+\dots+a_n = (z-p_1)(z-p_2)\cdots(z-p_n)\\
Q(z)&=z^m+b_1 z^{m-1}+\dots+b_m=(s-q_1)(z-q_2)\cdots(z-q_m) \end{cases},\ \ \ n\ge m \]

The root locus shows how the \alert{solution} to the characteristic equation
\begin{equation}
\label{eq:P(z)+KQ(z)=0}
P(z)+K\cdot Q(z)=0,\ \ \ 0\le K<\infty
\end{equation}
depend on the parameter \(K\). The root locus consists of the set of all points in the complex plane that are solutions to \eqref{eq:P(z)+KQ(z)=0} for some non-negative value of \(K\).
\end{frame}

\begin{frame}[label={sec:orgaed6fe4}]{Root locus: Characteristics}
\begin{description}
\item[{Start points}] The \(n\) roots of \(P(z)\), marked by crosses
\item[{End points}] The \(m\) roots of \(Q(z)\), marked  by circles
\item[{Asymptotes}] Number equal to the \emph{pole excess} \(n-m\)
\item[{Real axis}] Some segments of the real axis belong to the root locus
\end{description}
\end{frame}

\begin{frame}[label={sec:org83ded44}]{Root locus: Direction of the asymptotes}
The characteristic equation \(P(z)+K Q(z)=0\) can be written \(\frac{P(z)}{Q(z)} = -K\) and for large \(z\) it can be approximated as
\[ \frac{z^n}{z^m} = -K \quad \Leftrightarrow \quad z^{n-m} = -K.\]

Taking the argument of both sides of the equation gives
\((n-m)\arg z = \pi + k2\pi, \; k \in  \mathbb{Z}\)
So, the \alert{directions} of the asymptotes are given by the expression
\[ \theta_k = \arg z = \frac{(2k+1)\pi}{n-m}, \; k \in \mathbb{Z} \]
\end{frame}

\begin{frame}[label={sec:org7e82366}]{Root locus: The asymptotes' intersection with the real axis}
\[ z_{ip} = \frac{ \sum_{i=0}^n p_i - \sum_{i=0}^m q_i}{n-m}, \]
where \(\{p_i\}\) are the starting points (open-loop poles) and \(\{q_i\}\) are the end points (open-loop zeros).
\end{frame}

\begin{frame}[label={sec:org8888ac4}]{Root locus exerise: Pair the pulse-trf fcn and root locus}
\begin{columns}
\begin{column}{0.35\columnwidth}
 \small
\begin{align*}
  G_1(z) &= K\frac{(z+2.9)(z+0.2)}{(z-1)^2(z-0.3)}\\[3mm]
  G_2(z) &= K\frac{(z-0.5)(z+0.4)}{(z-1)(z-0.3)(z-0.1)}\\[3mm]
  G_3(z) &= K\frac{(z-0.5)(z+0.8)}{(z-1)^2(z-0.3)}\\[3mm]
  G_4(z) &= K \frac{z-0.6}{(z-1)(z-0.3)}
\end{align*}
\end{column}

\begin{column}{0.65\columnwidth}
\begin{center}
\includegraphics[width=1.04\linewidth]{../../matlab/rlocus_2x2-crop}
\end{center}
\end{column}
\end{columns}
\end{frame}
\end{document}


\documentclass{memoir}
\usepackage{notestemplate}

%\logo{~/School-Work/Auxiliary-Files/resources/png/logo.png}
%\institute{Rice University}
%\faculty{Faculty of Whatever Sciences}
%\department{Department of Mathematics}
%\title{Class Notes}
%\subtitle{Based on MATH xxx}
%\author{\textit{Author}\\Gabriel \textsc{Gress}}
%\supervisor{Linus \textsc{Torvalds}}
%\context{Well, I was bored...}
%\date{\today}

\begin{document}

% \maketitle

% Notes taken on 05/20/21

\section{More Direct Methods}
\label{sec:more_direct_methods}

Let \(u_0 \in C^{1}(\overline{\Omega })\) with \(\Omega \) some open bounded set. We want to minimize the functional:
\begin{align*}
	\inf_{u} \left\{\frac{1}{2} \int_\Omega \left| \nabla u \right|^2 \mid u =u_0 \text{ on }\partial\Omega  \right\}
\end{align*}
Let \(\left\{ u_j \right\}_j\) be a minimizing sequence for \(\inf_{\mathcal{A}} \mathcal{F}\) that satisfies the boundary conditions, and satisfying
\begin{align*}
	\lim_{j \to \infty} \frac{1}{2}\int_\Omega \left| \nabla u_j \right|^2 = \inf_{\mathcal{A}} \mathcal{F}.
\end{align*}
Because the limit supremum of the sequence is finite, we know that there exists a subsequence \(\left\{ \tilde{u}_j \right\}_j\) and \(T \in L^2(\Omega , \R^{n})\) such that
\begin{align*}
	\int_\Omega T \varphi  = \lim_{j \to \infty} \int_\Omega \varphi \nabla \tilde{u}_j \quad \forall \varphi  \in L^2(\Omega )
\end{align*}
Does this subsequence also have a limit of \(u\)? And is this \(T\) simply \(\nabla u\)?

\begin{thm}[Trace Inequality for Minimizers]
	If \(u \in C^{1}(\overline{\Omega })\) and \(\Omega \) is bounded with smooth boundary, then
	\begin{align*}
		\int_\Omega u^2 \leq C(n, \textrm{diam}\Omega ) \left[ \int_{\partial \Omega }u^2 + \int_\Omega \left| \nabla  u\right|^2 \right] .
	\end{align*}
\end{thm}
This quantifies the restrictions of the boundary conditions \(\begin{cases}
	u=0 & \partial\Omega \\
	\nabla u=0 & \Omega
\end{cases} \implies u \cong 0\) on \(\Omega \).

\begin{proof}[Proof of trace inequality]
	Choose \(X(x) = x\) so that \(\textrm{div}(X) = n\). We will pick \(x_0 \in \R^{n}\) later so that the following is satisfied:
	\begin{align*}
		n \int_\Omega u^2 = \int_\Omega u^2 \textrm{div}(x-x_0)\\
		= \int_\Omega \textrm{div}\left( (x-x_0)u^2 \right) - \int_\omega 2u \nabla u \cdot (x-x_0)\\
		\leq  \int_{\partial \Omega }u^2(x-x_0)\cdot \nu_\Omega + \int_\Omega 2 \left| u \right| \left| \nabla u \right| (x-x_0)
	\end{align*}
	Using some standard algebraic inequalities (i.e. \(2ab \leq \varepsilon a^2 + \frac{1}{\varepsilon}b^2\) so that \((\sqrt{\varepsilon} a = \frac{b}{\sqrt{\varepsilon} })^2\geq 0\)), we can put bounds on both terms individually to get
	\begin{align*}
		\int_{\partial \Omega } u^2(x-x_0) \cdot \nu_\Omega + \int_\Omega 2 \left| u \right| \left| \nabla u \right| (x-x_0)\leq 2 \textrm{diam} (\Omega ) \left[ \int_{\partial \Omega }u^2 + \int_\Omega 2\left| u \right| \left| \nabla u \right|  \right]\\
		\leq 2 \textrm{diam}(\Omega ) \int_{\partial \Omega } u^2+ 2 \textrm{diam}(\Omega ) \left[ \varepsilon \int_{\Omega }u^2 + \frac{1}{\varepsilon} \int_{\Omega } \left| \nabla u \right|^2 \right]\\
		\implies \left( n-2\varepsilon\textrm{diam}(\Omega ) \right) \int_{\Omega } u^2 \leq 2 \textrm{diam}(\Omega )\left[ \int_{\partial \Omega }u^2 + \frac{1}{\varepsilon} \int_\Omega  \left| \nabla u \right|^2 \right]
	\end{align*}
	Now we simply choose \(\varepsilon = \dfrac{n}{4 \textrm{diam}(\Omega )}\) and we have the statement
	\begin{align*}
		\int_\Omega u^2 \leq \dfrac{4 \textrm{diam}(\Omega)}{n} \left[ \int_{\partial \Omega }u^2 + \dfrac{4 \textrm{diam}(\Omega )}{n} \int_\Omega  \left| \nabla u \right|^2 \right]
	\end{align*}
\end{proof}

What is the relation between \(T\) and \(u\)?

\begin{defn}
	Let \(u \in L^{1}_{\textrm{loc}}(\Omega )\), and \(T \in L^{1}_{\textrm{loc}}(\Omega , \R^{n})\). We say that \(T\) is the unique vector field called the \textbf{weak} or \textbf{distributional gradient} of \(u\) if the following is satisfied:
	\begin{align*}
		\int_\Omega u \nabla \varphi = - \int_\Omega \varphi T \quad \forall \varphi  \in C^{\infty}_c (\Omega )
	\end{align*}
\end{defn}

Earlier, we asked if the limit of the subsequence was equivalent to the \(u\) earlier. Now we claim that the weak limit \(T\) of \(\left\{ \nabla u_j \right\}_j\) is the weak gradient of
 \begin{align*}
	u = \lim_{n \to \infty}^{w} \left\{ u_j \right\}_j .
\end{align*}
This follows because
\begin{align*}
	\int_\omega u \nabla \varphi = \lim_{j \to \infty} \int_\Omega u_j \nabla \varphi  = - \lim_{j \to \infty} \int_\Omega \varphi \nabla u_j\\
	= - \int_\Omega \varphi T
\end{align*}
In summary, by weak compactness in \(L^2\) and by Trace inequality, we have
\begin{align*}
	\begin{cases}
		u_j \to u & L^2(\Omega )\quad u \in W^{1,2}(\Omega )\\
		\nabla u_j \to \nabla u & L^2(\Omega ,\R^{n})
	\end{cases}
\end{align*}
We need to define a space where these weak objects can live:
\begin{defn}[\(L^2\) Sobolev Space]
	We define the \textbf{Sobolev Space in \(L^2\)} to be
	\begin{align*}
		W^{1,2}(\Omega ) := \left\{u \in L^2(\Omega ) \mid \exists  \text{ weak gradient }\nabla u \in L^2(\Omega ,\R^{n}) \right\}
	\end{align*}
\end{defn}

Now we ask a new question: is it true that
\begin{align*}
	\int_\Omega \left| \nabla u \right|^2 \leq \liminf_{j \to \infty} \int_\Omega  \left| \nabla u_j \right|^2
\end{align*}

Yes. Pick any \(T \in L^2(\Omega ,\R^{n})\). Then
\begin{align*}
	\int_\Omega (\nabla u) \cdot T = \lim_{j \to \infty} \int_\Omega (\nabla u_j) \cdot T\\
	\leq \lim_{j \to \infty} \left( \int_\Omega \left| \nabla u_j \right|^2 \right)^{ \sfrac{1}{2}} \left( \int_{\Omega } \left| T \right|^2 \right)^{ \sfrac{1}{2}}
\end{align*}
Now choose
\begin{align*}
	T = \frac{\nabla u}{\|\nabla u\|_{L^2}}
\end{align*}
to get the inequality we desire.\\

Another question. Is \(u \in \mathcal{A}\), where
\begin{align*}
	\mathcal{A} = \left\{ \mid  \right\}
\end{align*}

\begin{exmp}
	Consider
	\begin{align*}
		u_\alpha (x) = \dfrac{1}{\left| x \right|^{\alpha }}\\
		\nabla u_\alpha (x) = \dfrac{-\alpha }{\left| x \right|^{\alpha +1}} \hat{x}
	\end{align*}
	where \(\hat{x} = \dfrac{x}{\left| x \right| }\). Then
	\begin{align*}
		\int_{B_\varepsilon(0)} \left| \nabla u_\alpha  \right|^2 \approx \int_{0}^{\varepsilon} \dfrac{\rho^{n-1}}{\left( \rho^{\alpha +1} \right)^2} \,d \rho < \infty \\
		\iff n-1 -2(\alpha +1) > -1 \iff\alpha < \frac{n}{2}-1
	\end{align*}
\end{exmp}

	Now we ask a seemingly silly question: is \(\nabla u_{\alpha }\) the weak gradient of \(u_\alpha \)? Yes, but we have to be careful-- if \(\nabla u = 0\) almost everywhere in \(\R^2\), say, \(u\) is a step function, then the weak gradient doesn't exist. One can show that
	\begin{align*}
		\int \varphi \nabla u_{\alpha } = - \int u_\alpha \nabla \varphi \quad \forall \varphi \in C^{\infty}_c(B_1(0))
	\end{align*}
via some standards calculations from distribution theory. Because this holds, we have
\begin{align*}
	u^{(n)} = \sum_{k=1}^{n} \dfrac{2^{-k}}{\left| x-x_k \right|\alpha }
\end{align*}
where \(\left\{ x_k\right\}_{k=1}^{\infty} \) is countably dense in \(B_1(0)\). We have that \(u^{(n)} \in W^{1,2}(\Omega )\) if \(\alpha < \frac{n}{2}-1\).In fact, this is a Cauchy sequence in \(W^{1,2}(\Omega )\) (check this), and so
\begin{align*}
	\exists u \in W^{1,2}(\Omega ) \text{ such that{t} } u = \lim_{N \to \infty} u^{(N)}= \sum_{k=1}^{\infty} \dfrac{2^{-k}}{\left| x-x_k \right|^{\alpha }}
\end{align*}

\begin{exmp}
	Let \(u \not\in L^{\omega }_{\textrm{loc}}(B_{1}(0))\). For all \(B_\varepsilon(x) \subset B_1(0)\), we have
	\begin{align*}
		\textrm{essential sup}_{B_\varepsilon} \left| u \right|  = \infty
	\end{align*}
\end{exmp}

\begin{rmrk}
What is the essential supremum? Let \(f:\Omega \to \R\). Consider the case when \(\tilde{f}:\Omega \to \R\) by
\begin{align*}
	\tilde{f}(x) = \begin{cases}
		f(x) & \Omega \setminus Q^{n}\\
		j & x = x_j \in Q^{n}\cap \Omega
	\end{cases}
\end{align*}
So that \(f = \tilde{f}\) almost everywhere in \(\Omega \) and \(\sup_{j} \tilde{f}= \infty\). A traditional supremum is unbounded and so won't work nicely here. Instead, notice that
\begin{align*}
	\left\{x \in \Omega  \mid f(x) > t \right\} = \left\{f > t \right\} \\
	\mathcal{L}^{n}( \left\{ f > t \right\} ) = \mathcal{L}^{n}\left( \left\{ \tilde{f} > t \right\}  \right)
\end{align*}
With this in mind, we define
\begin{align*}
	\textrm{essential sup}_{\Omega } f := \inf_{t} \left\{ t\mid \mathcal{L}^{n}( \left\{f >t \right\} ) = 0 \right\} .
\end{align*}
\end{rmrk}

\begin{exmp}
	\(u=u_0\) on \(\partial \Omega \)?\\

	\(u_j \to u\) in \(L^2(\Omega )\) but we do \textit{not} have convergence of boundary data. To see this, look at the diagram in \ref{fig:convergence-of-boundary-data}.
\end{exmp}

\begin{figure}[ht]
    \centering
     \def\svgwidth{1\linewidth}
     \input{./figures/convergence-of-boundary-data.pdf_tex}
    \caption{Convergence of Boundary Data}
    \label{fig:convergence-of-boundary-data}
\end{figure}

The moral is that we need more properties to hold: if \(u_j = u_0 \partial \Omega \), \(\sup_{j} \int \left| \nabla u_j \right|^2 < \infty\) and
\begin{align*}
	\begin{cases}
		u_j \to u\\
		\nabla u_j \to \nabla u
	\end{cases}
\end{align*}
then we can say that \(u = u_0\) on \(\partial \Omega \) in the distributional sense.\\

To summarize the work we've done, if we are working with problems of the form
\begin{align*}
	\inf_{u} \left\{\frac{1}{2}\int_\Omega \left| \nabla u \right|^2 \mid u = u_0 \text{ on }\partial \Omega  \text{ in distribution} \right\}
\end{align*}
where \(u \in W^{1,2}(\Omega )\), then the direct method gives you a minimizer. For the restricted problem with \(u \in C^{1}(\overline{\Omega })\) :
\begin{align*}
	\inf_{u} \left\{\frac{1}{2}\int_\Omega \left| \nabla u \right|^2 \mid u = u_0 \text{ on }\partial\Omega  \text{ in classical sense} \right\} \\
	= \frac{1}{2}\int_\Omega  \left| \nabla u \right|^2
\end{align*}
we have the existence of a minimizer \(u \in W^{1,2}(\Omega )\).\\

Some classical results give us some insight:
\begin{thm}[Serrin '61]
	If \(\left\{ u_j \right\}_j \subset L^{1}_{\textrm{loc}}(\Omega )\) and \(\lim_{j \to \infty} u_j = u\) in \(L^{1}_{\textrm{loc}}(\Omega )\) and \(u_j,u\) have weak gradient in \(L^{1}(\Omega ,\R^{n})\), then for all \(f:\R^{n}\to [0,\infty)\) convex, we have
	\begin{align*}
		\int_\Omega f(\nabla u) \leq \liminf_{j \to \infty} \int_\Omega f(\nabla u_j).
	\end{align*}
\end{thm}
We have a converse too. If \(f:\R^{n}\to [0,\infty)\) is continuous and the above result holds, then for all \(u_j,j\) as above, \(f\) is convex.

\end{document}

Topics in Partial Differential Equations/Calculus of Variations/Notes/source/Lecture4.tex