\chapter{Analytical mechanics}

% https://halshs.archives-ouvertes.fr/halshs-00116768/file/chap5.pdf
\emph{The origins of analytical mechanics in 18th century}, Marco Panza
\cite{panza2003origins}

About the name:
Why is it ``analytical mechanics'' and not ``analytic mechanics''?

\emph{Analytical mechanics} is mathematical analysis applied to mechanics.
Calculus is a part of mathematical analysis.
Calculus is about limit, derivative, and integral.
Several \emph{formulations} of mechanics are
Newtonian, Lagrangian, Hamiltonian, Routhian,
one due to Appell,
and one due to Udwadia\textendash{}Kalaba.

% https://en.wikipedia.org/wiki/Analytical_mechanics
% https://en.wikipedia.org/wiki/Mathematical_analysis
% https://en.wikipedia.org/wiki/Gauss%27s_principle_of_least_constraint

\section{Principle of economy}

\section{Variational principles}

\emph{D'Alembert's principle of virtual work}?
Virtual work?
Virtual displacement?
% https://en.wikipedia.org/wiki/D%27Alembert%27s_principle
% https://en.wikipedia.org/wiki/Virtual_displacement
% http://fy.chalmers.se/~tfemc/mekanikkompendium.pdf
% https://en.wikipedia.org/wiki/Hamilton%27s_principle
% https://en.wikipedia.org/wiki/Principle_of_least_action
% https://en.wikipedia.org/wiki/Routhian_mechanics
% https://en.wikipedia.org/wiki/Appell%27s_equation_of_motion
% https://en.m.wikipedia.org/wiki/Udwadia–Kalaba_equation

\section{Lagrangian mechanics}

Introduction to analytical mechanics \cite[p.~43]{varvoglis2014history}

% https://en.wikipedia.org/wiki/Lagrangian_mechanics#From_Newtonian_to_Lagrangian_mechanics

% https://archive.org/details/springer_10.1007-978-94-015-8903-1
% https://en.m.wikipedia.org/wiki/Mécanique_analytique
Lagrange's \emph{M\'ecanique analytique} was published in 1788 \cite{lagrange1997analytical}.

Lagrangian mechanics only works for conservative forces?

The \emph{Lagrangian} of a mechanical system is?

% https://en.m.wikipedia.org/wiki/Lagrangian_mechanics

Let \( e_k \in \Real^n \) be the \emph{\(k\)th basis vector of \( \Real^n \)};
it is \( e_k = [\delta_{ik}]_{i=1}^n \);
the \(k\)th component of \( e_k \) is one; every other component of \( e_k \) is zero;
\( (e_k)_k = 1 \) and \( (e_k)_i = 0 \) if \( i \neq k \).

For example, assume \( \Real^3 \), and let there be a particle \( M \)
whose mass is \( m \)
and whose position is \( (0,0,0) \).
The \emph{gravitational field} of \( M \) at \( x \) is \( g(x) = - G m x / |x|^3 \).
Such \( g \) is a vector field.
The \emph{gravitational potential} of \( M \) is the \( \phi \) such that \( \nabla \phi = g \).
Such \( \phi \) is a scalar field.
The \emph{potential energy of \( N \) due to \( M \)} is \( K_{MN} = m_N \cdot \phi(x_N) \).

Every particle in a system translates to two things in the \emph{phase space}: a position and a momentum.
We can describe a system without explicit reference to time.
We describe each particle by a set of \emph{position-momentum pairs}.
This set is the \emph{phase space} of the system.

Why bother using phase space if systems of equations work just fine?

We can \emph{describe} the trajectory of a particle using a function
whose input is a real number representing relative time
and output is a three-dimensional real vector representing relative position.
This is straightforward to imagine.

However, we can also describe the same thing by a set of ordered pairs
\( \{ (t,x) ~|~ \text{the object is at \(x\) at time \(t\)} \} \).

Newton's law of gravity describes the force that a
\emph{point mass} exerts on another point mass.
It still applies to planets even if when we assume that a planet is a point mass.
\[
    F_{ab} = \frac{G m_a m_b}{|r_{ab}|^2} \hat{r}_{ab}
\]

% https://en.wikipedia.org/wiki/Gauss%27s_law_for_gravity

Newton's second law:
\(F = dp\) where \(F(t)\) is force acting on the point mass at time \(t\)
and \(p(t)\) is the momentum of the point mass at time \(t\).

The \emph{degree of freedom} of a system is the minimum number of parameters required to describe that system.

A system of \emph{equations of motion} has the form:
\begin{align*}
    x_1(t) &= \ldots
    \\
    & \vdots
    \\
    x_n(t) &= \ldots
\end{align*}

Imagine that in front of you there is a \emph{pendulum} hanging on a thread attached to the roof.
To model that system, we could pick the XYZ coordinate system
where, from your point of view,
the positive X axis is rightward, the positive Y axis is forward, and the positive Z axis is upward.
Thus, at all times, the force acting on the pendulum is \( F = (0,0,-mg) \).
This should be straightforward to imagine.
But you have to determine the tension of the thread that constrains the pendulum's motion.

But we can pick another coordinate system where a point is described by \( (\theta) \).
Let \(\theta\) be the angle from the vertical axis to the thread:
\( \theta = 0 \) means that the thread is vertical,
and positive \( \theta \) means that the pendulum is to your right.
Let \( K \) be the line length.
\(h = K - (1 - \cos \theta) K = K \cos \theta\).
\(P = m g h\).
\(K = \frac{1}{2} m v^2\).
Generalized coordinates: \((h,\theta)\) instead of \((x,y)\).
Translation rules: \(x = K \sin \theta\) and \(y = K \cos \theta\).
The point \(x,y=0,0\) is the lowest point of the pendulum.

\section{Example: Two rigid bodies}

Assume constant mass.
\begin{align*}
    m_1 \cdot (d^2 x_1)(t) &= G m_1 m_2 \cdot (x_2(t) - x_1(t)) / \norm{x_1(t) - x_2(t)}^3
    \\
    m_2 \cdot (d^2 x_2)(t) &= G m_1 m_2 \cdot (x_1(t) - x_2(t)) / \norm{x_1(t) - x_2(t)}^3
\end{align*}
Matrix form:
\begin{align*}
    \bmat{
        m_1 \cdot (d^2 x_1)(t)
        \\
        m_2 \cdot (d^2 x_2)(t)
    }
    &=
    \frac{G m_1 m_2}{\norm{x_1(t) - x_2(t)}^3}
    \bmat{
        x_2(t) - x_1(t)
        \\
        x_1(t) - x_2(t)
    }
\end{align*}

\paragraph{Example}
Uniform gravitation field.
Phase space coordinate \(h\) where \(h\) is height.
\(P(h) = m g h\).
\(K(h) = m (v(h))^2 / 2\).
Conservation of energy: \(P + K = E\).
\(d_h E = 0 = m g + \frac{1}{2} m \cdot (d_h v)(h) \cdot 2 v(h)\).
\(0 = g + (d_h v)(h) \cdot v(h)\).
\(- g = d_h v \cdot v\).

\section{Canonical coordinates}

\section{Poisson bracket}

The \emph{Poisson bracket} is ...

\section{Hamiltonian mechanics}

With physical laws, we can predict the state of physical systems.

A
\index{configuration space}%
\emph{configuration space} is a vector space where each vector is a generalized coordinate tuple.

Example of Hamiltonian mechanics:
In two-body problem,
the state space is ...,
the configuration space is ...,

\section{Noether's theorem}

\subsection{Conservation of energy}

\subsection{Newton's third law of motion}


\documentclass[a4paper]{report}

\usepackage[margin=3.0cm]{geometry}
\usepackage{amsmath}
\usepackage[pdftex]{graphicx}
%\usepackage{graphics}
\usepackage{subfig}

\title{BLASFEO reference guide}
\author{Gianluca Frison}

\begin{document}

\maketitle
\tableofcontents

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\chapter{Introduction}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

BLASFEO - BLAS For Embedded Optimization.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\chapter{Matrix data type}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The fundamental data type in BLASFEO is a C struct defining a matrix, called {\tt strmat}.
Depending on the chosen linear algebra library, the struct is defined differently.

\section{{\tt strmat} definition}

\subsection{BLASFEO}

\begin{verbatim}
struct d_strmat
	{
	int bs;
	int m;
	int n;
	int pm;
	int cn;
	double *pA;
	double *dA;
	int use_dA;
	int memory_size;
	};
\end{verbatim}
where the struct members are
\begin{description}
\item[bs] height of the panel
\item[m] number of rows
\item[n] number of columns
\item[pm] number of rows of the matrix as allocated in memory, used for memory alignment
\item[cn] number of rows of the matrix as allocated in memory, used for memory alignment
\item[pA] pointer to a pm$\times$pn array of doubles, the first element is aligned to cache line size
\item[dA] pointer to a min(m,n) array of doubles, used e.g. to store the inverse of the diagonal of the matrix
\item[use\_dA] flag to tell if dA contains useful information
\item[memory\_size] size of the memory (in bytes) needed for pA and pD
\end{description}

\subsection{BLAS}

\begin{verbatim}
struct d_strmat
	{
	int m; // rows
	int n; // cols
	double *pA; // pointer to a m*n array of doubles
	int memory_size; // size of needed memory
	};
\end{verbatim}
\begin{description}
\item[m] number of rows
\item[n] number of columns
\item[pA] pointer to a m$\times$n array of doubles
\item[memory\_size] size of the memory (in bytes) needed for pA
\end{description}

\section{{\tt strmat} management}

\begin{verbatim}
void d_allocate_strmat(int m, int n, struct d_strmat *sA);
\end{verbatim}

\begin{verbatim}
void d_free_strmat(struct d_strmat *sA);
\end{verbatim}

\begin{verbatim}
int d_size_strmat(int m, int n);
\end{verbatim}

\begin{verbatim}
void d_create_strmat(int m, int n, struct d_strmat *sA, void *memory);
\end{verbatim}

\section{{\tt strmat} conversion}

\begin{verbatim}
void d_cvt_mat2strmat(int m, int n, double *A, int lda, struct d_strmat *sA,
     int ai, int aj);
\end{verbatim}

\begin{verbatim}
void d_cvt_tran_mat2strmat(int m, int n, double *A, int lda, struct d_strmat *sA,
     int ai, int aj);
\end{verbatim}

\begin{verbatim}
void d_cvt_strmat2mat(int m, int n, struct d_strmat *sA, int ai, int aj,
     double *A, int lda);
\end{verbatim}

\begin{verbatim}
void d_cvt_tran_strmat2mat(int m, int n, struct d_strmat *sA, int ai, int aj,
     double *A, int lda);
\end{verbatim}

\section{{\tt strmat} print}

\begin{verbatim}
void d_print_strmat(int m, int n, struct d_strmat *sA, int ai, int aj);
\end{verbatim}

\end{document}


\subsection{Fully-Connected Neural Network}

\subsubsection{Operation}

\begin{figure}[h]
	\centering
	\includegraphics[width=0.8\textwidth]{img/dense.pdf}
	\caption{Diagram of the combined, fully connected NN.}
	\label{FIG:nn}
\end{figure}

The fully-connected neural network is shown in Figure~\ref{FIG:nn}. It consists of two dense layer instances controlled by a state machine. The output of layer 1 is fed directly into the layer 2. The output of layer 2 is a 10-value vector which represent the confidence that the input image showed a specific numeral.

The Serializer module is connected to the final pooling layer. The $m=32$ output channels need to be converted into a stream of single values of length VECTOR\_WIDTH. For this, the pooling layer is stalled by keeping the ready signal low while a vector of $m$ values is serialized.

\subsubsection{Interface}
\begin{itemize}
	\item Input interface, a stream of values of length VECTOR\_WIDTH
	\item Output interface, a vector of 10 values of length VECTOR\_WIDTH
\end{itemize}
\subsubsection{Parameter}
\begin{itemize}
	\item VECTOR\_WIDTH: integer
	\item INPUT\_COUNT: integer
	\item OUTPUT\_COUNT: integer
\end{itemize}


This appendix presents more complete details and justification for the calibrated parameters in Table~\ref{table:calibration}.  We begin by calibrating market-level and preference parameters by standard methods, then specify additional parameters to characterize the idiosyncratic income shock distribution.

\subsection{Macroeconomic Calibration}
\label{sec:MacroCal}

We assume a coefficient of relative risk aversion of
$
\input /Volumes/Data/Papers/cAndCwithStickyE/cAndCwithStickyE-AEJM/Calibration/CRRA.txt
$. % $ marks is needed to avoid space between CRRA.txt and "," in the pdf
The quarterly depreciation rate $\delta$ is calibrated by assuming annual depreciation of 6 percent,
i.e.,
$
\DeprFac^{4}= \input /Volumes/Data/Papers/cAndCwithStickyE/cAndCwithStickyE-AEJM/Calibration/DeprFacAnn.txt
$.  Capital's share in aggregate output takes its usual value of
$
\kapShare= \input /Volumes/Data/Papers/cAndCwithStickyE/cAndCwithStickyE-AEJM/Calibration/CapShare.txt
$.

We set the variances of the quarterly transitory and permanent shocks at the approximate values respectively:
\begin{eqnarray*}
	\sigma^{2}_{\Theta} & = & \input /Volumes/Data/Papers/cAndCwithStickyE/cAndCwithStickyE-AEJM/Calibration/TranShkAggVar.txt ,
	\\ \sigma^{2}_{\Psi}  & =  & \input /Volumes/Data/Papers/cAndCwithStickyE/cAndCwithStickyE-AEJM/Calibration/PermShkAggVar.txt ,
\end{eqnarray*}
which allow the model to match high degree of persistence in aggregate labor income.\footnote{We measure labor income using U.S.\ NIPA data as wages and salaries plus transfers minus personal contributions for social insurance.} These values are consistent with papers such as \cite{jermannProduction}, \cite{bcfHabits}, and \cite{ckmCritique}, considered standard  in the RBC literature. These authors model the state of technology as either a highly persistent AR(1) process or a random walk; but the underlying calibrations come from the autocorrelation properties of measured aggregate dynamics, which are matched about as well by our specification of the income process.

To finish the calibration, we consider a simple perfect foresight model (PF-DSGE), with all aggregate and idiosyncratic shocks turned off.  We set the perfect foresight steady state aggregate capital-to-output ratio to \input /Volumes/Data/Papers/cAndCwithStickyE/cAndCwithStickyE-AEJM/Calibration/KYratioSS.txt on a quarterly basis (corresponding to the usual ratio of 3 for capital divided by annual income).  Along with the calibrated values of $\kapShare$ and $\delta$, this choice implies values for the other steady-state characteristics of the PF-DSGE model:
\begin{eqnarray*}
	{\KLev} & = & \input /Volumes/Data/Papers/cAndCwithStickyE/cAndCwithStickyE-AEJM/Calibration/KYratioSS.txt ^{1/(1-\kapShare)},
	\\   {\Wage} & = & (1-\kapShare) {\KLev}^{\kapShare},
	\\   {\Rprod} & = & \DeprFac+\kapShare {\KLev}^{\kapShare-1}
	.
\end{eqnarray*}
In the SOE model, we fix the interest factor $\Rprod$ and wage rate $\Wage$ to these PF-DSGE steady state values.

A perfect foresight representative agent would achieve this steady state if his discount factor satisfied ${\Rprod} \beta = 1$.  For the SOE model, however, we choose a much lower value of $\beta$ ($ \input /Volumes/Data/Papers/cAndCwithStickyE/cAndCwithStickyE-AEJM/Calibration/betaSOE.txt $), resulting in agents with wealth holdings around the median observed in the data;\footnote{The exact value of the median is depends in part on whether housing equity should be viewed as part of the precautionary buffer stock, the age range of the households being matched, the measure of permanent income, and many other extraneous issues.} the value of $\beta$ satisfying ${\Rprod} \beta = 1$ is used in the closed economy models presented in the online appendix, allowing those models to fit the \textit{mean}  observed wealth.

\subsection{Calibration of Idiosyncratic Shocks}

The annual-rate idiosyncratic transitory and permanent shocks are assumed to be:
\begin{eqnarray*}
	\sigma_{\theta}^{2} & = & \input /Volumes/Data/Papers/cAndCwithStickyE/cAndCwithStickyE-AEJM/Calibration/TranShkVarAnn.txt ,
	\\ \sigma_{\psi}^{2}             & = & \input /Volumes/Data/Papers/cAndCwithStickyE/cAndCwithStickyE-AEJM/Calibration/PermShkVarAnn.txt
	.
\end{eqnarray*}

Our calibration for the sizes of the idiosyncratic shocks are conservative relative to the literature;\footnote{See Table~1 in the ECB working paper version of \cite{cstKS} for a comprehensive overview of estimates of variances of idiosyncratic income shocks; Carroll, Christopher~D., Jiri Slacalek, and Kiichi Tokuoka (2014): ``Buffer-Stock Saving in a Krusell--Smith World,'' working paper 1633, European Central Bank, \url{https://www.ecb.europa.eu/pub/pdf/scpwps/ecbwp1633.pdf}.} using data from the {\it Panel Study of Income Dynamics}, for example, \cite{carroll&samwick:nature} estimate $\sigma_{\psi}^{2} = 0.0217$ and $\sigma_{\theta}^{2} = 0.0440$; Storesletten, Telmer, and Yaron~(\citeyear{sty:consumption}) estimate $\sigma_{\psi}^{2} \approx 0.017$, with varying estimates of the transitory component.  But recent work by \cite{lmpPermShocks} suggests that controlling for participation decisions reduces estimates of the permanent variance somewhat; and using very well-measured Danish administrative data, \cite{nv:risk} estimate $\sigma_{\psi}^{2} \approx 0.005$ and $\sigma_{\theta}^{2} \approx 0.015$, which presumably constitute lower bounds for plausible values for the truth in the U.S. (given the comparative generosity of the Danish welfare state).

We assume that the probability of unemployment is 5 percent per quarter.  This approximates the historical mean unemployment rate in the U.S., but model unemployment differs from real unemployment in (at least) two important ways.  First, the model does not incorporate unemployment insurance, so labor income of the unemployed is zero.  Second, model unemployment shocks last only one quarter, so their duration is shorter than the typical U.S.\ unemployment spell (about 6 months).  The idea of the calibration is that a single quarter of unemployment with zero benefits is roughly as bad as two quarters of unemployment with an unemployment insurance payment of half of permanent labor income (a reasonable approximation to the typical situation facing unemployed workers).  The model could be modified to permit a more realistic treatment of unemployment spells; this is a promising topic for future research, but would involve a considerable increase in model complexity because realism would require adding the individual's employment situation as a state variable.

The probability of mortality is set at $\PDies= \input {/Volumes/Data/Papers/cAndCwithStickyE/cAndCwithStickyE-AEJM/Calibration/DiePrb.txt}$, which implies an expected working life of 50 years; results are not sensitive to plausible alternative values of this parameter, so long as the life length is short enough to permit a stationary distribution of idiosyncratic permanent income.


% !TEX root =  paper.tex

\begin{figure*}[t!]
	\centering
	\includegraphics[width=0.45\linewidth]{./figures/variation_of_information-microns-train-600.png}
	\includegraphics[width=0.45\linewidth]{./figures/variation_of_information-microns-test-600.png}
	\includegraphics[width=0.45\linewidth]{./figures/variation_of_information-FlyEM-train-600.png}
	\includegraphics[width=0.45\linewidth]{./figures/variation_of_information-FlyEM-test-600.png}
	\caption{VI scores of our method (red) compared to the baseline segmentation (green) and an oracle (blue) that optimally partitions the graph based on ground truth. Lower scores are better. Our method improves the accuracy of the segmentation in all cases.}
	\label{fig:variation-of-information}
\end{figure*}

%\section{Results}

\subsection{Error Metric}
\label{sec:variation-of-information}

We evaluate the performance of the different methods using the split variation of information (VI)~\cite{meila2003comparing}.
Given a ground truth labeling $GT$ and our automatically reconstructed segmentation $SG$, over- and under-segmentation are quantified by the conditional entropies $H(GT | SG)$ and $H(SG | GT)$, respectively. Since we are measuring the entropies between two clusterings, lower VI scores are better.

\subsection{Variation of Information Results}

In Fig.~\ref{fig:variation-of-information}, we show the VI results of the pixel-based reconstructions of the Kasthuri and FlyEM data (Sec.~\ref{sec:neuroproof}) for varying thresholds of agglomeration (green). We use one of these segmentations (green circle) as out input dataset with an agglomeration threshold of 0.3 for all datasets. The results from our method are shown in red for varying the $\beta$ parameter. We show comparisons to an oracle (blue) that correctly partitions the graph from our method based on ground truth.

Our algorithm improves the accuracy of the reconstruction for every dataset, reducing the VI split score on average by 19.3\% on the three testing datasets.
Scores closer to the origin are better for this metric, and in every instance our results are below the green curve.
We see significant improvements on the Kasthuri datasets (VI split reduction of 27.7\% and 24.8\% on the training and testing datasets respectively) and more modest improvements on the FlyEM datasets (reduction of 15.2\% and 18.0\%). This is because the baseline segmentation algorithm for the isotropic FlyEM data (Sec.~\ref{sec:neuroproof}) performs much better, reducing the potential for improvements. Isotropic datasets are easier to segment using state-of-the-art region-based methods than anisotropic ones~\cite{plaza2014annotating}.

Fig.~\ref{fig:positive-results} shows successful merges on the Kasthuri Vol. 2 dataset. Several of these examples combine multiple consecutive segments that span the volume.
In the third example we correct the over-segmentation of a dendrite and attached spine-necks.
Fig.~\ref{fig:negative-results} shows typical failure cases of our method (red circles).
In two of these examples the algorithm correctly predicted several merges before a single error rendered the segment as wrong.
In the third example (blue circle) a merge error in the initial segmentation propagated to our output.
We now analyze how each major component of our method contributes to this final result.

\begin{figure}[t]
	\centering
	\includegraphics[width=0.85\linewidth]{./figures/VI-results/multicut-correct1.png}
	\includegraphics[width=0.85\linewidth]{./figures/VI-results/multicut-correct2.png}
	\includegraphics[width=0.85\linewidth]{./figures/VI-results/multicut-correct3.png}
	\includegraphics[width=0.85\linewidth]{./figures/VI-results/multicut-correct4.png}
	\includegraphics[width=0.85\linewidth]{./figures/VI-results/multicut-correct5.png}
	\caption{Segments of neurons that were correctly merged by our method.}
	\label{fig:positive-results}
\end{figure}

\begin{figure}[t]
	\centering
	\includegraphics[width=0.85\linewidth]{./figures/VI-results/multicut-incorrect1.png}
	\includegraphics[width=0.85\linewidth]{./figures/VI-results/multicut-incorrect2.png}
	\includegraphics[width=0.85\linewidth]{./figures/VI-results/multicut-incorrect3.png}
	\includegraphics[width=0.85\linewidth]{./figures/VI-results/multicut-incorrect4.png}
	\caption{Circles indicate areas of wrong merges by our method (red) or by the initial pixel-based segmentation (blue).}
	\label{fig:negative-results}
\end{figure}

\subsection{Graph Pruning Results}

Table \ref{table:skeletonization} shows the results of pruning the skeleton graph using the algorithm discussed in Sec.~\ref{sec:skeletonization}. This edge pruning is essential for the graph partitioning algorithm, which has a computational complexity dependence on the number of edges. The baseline algorithm considers all adjacent regions for merging. Our method removes a significant portion of these candidates while maintaining a large number of the true merge locations (e.g., 753 compared to 763). Our pruning heuristic removes at least $6\times$ the number of edges on all datasets, achieving a maximum removal rate of $20\times$.

\begin{table}
	\centering
	\small
	\begin{tabular}{c c c} \hline
		\textbf{Dataset} & \textbf{Baseline} & \textbf{After Pruning} \\ \hline
		Kasthuri Training & 763 / 21,242 & 753 / 3,459 \\
		Kasthuri Vol. 2 & 1,010 / 26,073 & 904 / 4,327 \\
		FlyEM Vol. 1 & 269 / 14,875 & 262 / 946 \\
		FlyEM Vol. 2 & 270 / 16,808 & 285 / 768 \\ \hline
		%		Kasthuri Vol. 1 & 763 / 21242 (3.47\%) & 753 / 3459 (17.88\%) \\
		%		Kasthuri Vol. 2 & 1010 / 26073 (3.73\%) & 904 / 4327 (17.28\%) \\
		%		FlyEM Vol. 1 & 269 / 14875 (1.78\%) & 262 / 946 (21.69\%) \\
		%		FlyEM Vol. 2 & 270 / 16808 (1.58\%) & 285 / 768 (27.07\%)\\ \hline
	\end{tabular}
	\caption{The results of our graph pruning approach compared to the baseline graph with all adjacent regions. We show the number of true merge locations (e.g., 763) compared to total number of edges in the graph (e.g., 21,242) for each case.}
	\label{table:skeletonization}
\end{table}

We generate edges in our graph by using information from the skeletons.
In particular, we do not enforce the constraint that edges in our graph correspond to adjacent segments.
Although neurons are continuous, the EM images often have noisy spots which cause an interruption in the input segmentation.
We still want to reconstruct these neurons despite the fact that the initial segmentation is non-continuous.
The second and fourth examples in Fig.~\ref{fig:positive-results} show correctly reconstructed neurons where two of the segments are non-adjacent.
This is a large benefit over enforcing segment adjacency.

There are some pairs of segments which we do not consider for merging because of our reliance on the skeletons.
Fig.~\ref{fig:skeleton-results} shows such a case.
The endpoints of both segments are circled.
In this example the small segment is carved from the larger segment in a location where there are no skeleton endpoints.

\begin{figure}[h!]
	\centering
	\includegraphics[width=0.85\linewidth]{./figures/merge_candidate1.png}
	\caption{A false negative example of our method due to graph pruning. The distance between the endpoints (circled) of the two segments is too far to be flagged as a merge candidate.}
	\label{fig:skeleton-results}
\end{figure}

\subsection{CNN Classification Results}

Fig.~\ref{fig:receiver-operating-characteristic} shows the receiver operating characteristic (ROC) curve of our CNN classifier for all test datasets.
%We train our CNN using one of the Kasthuri volumes and test using the other three datasets.
Since our CNN only takes as input a region of the label volume we can train on  anisotropic data and test on isotropic data.
This provides a major benefit given the time-intensive task of manually generating ground truth for each dataset at various resolutions.

As shown by the ROC curve, the test results on the Kasthuri data are better than the results for FlyEM.
We believe this is in part because of the differences in the datasets (i.e., isotropy and $xy$ resolution).
To test this hypothesis, we also evaluate the performance of the FlyEM datasets when the network trains on FlyEM Vol. 1 and infers on FlyEM Vol. 2.\footnote{Since the FlyEM datasets have significantly fewer examples, we initialize the network with the weights from the Kasthuri training and have an initial learning rate of $10^{-4}$.}
The blue dotted curve in the figure shows a slight performance increase in this case. However, the improvement is minor, which led us to use the CNN trained on the anisotropic data for the rest of our experiments.

\begin{figure}
	\centering
	\includegraphics[width=0.95\linewidth]{./figures/receiver-operating-characteristic.jpg}
	\caption{The receiver operating characteristic (ROC) curves of our classifier on three connectomics datasets. The classifier works best on previously unseen data of the Kasthuri volume. The dashed blue line indicates better performance on the FlyEM datasets with retraining compared to without (solid blue).}
	\label{fig:receiver-operating-characteristic}
\end{figure}

\subsection{Graph Optimization Results}

The graph optimization strategy using multicut increases our accuracy over using just the CNN.
Table \ref{table:multicut} shows the changes in precision, recall, and accuracy for all four datasets compared to the CNN.
The precision increases on each dataset, although the recall decreases on all but one of the datasets.
Since it is more difficult to correct merge errors than split errors, it is often desirable to sacrifice recall for precision.
Over the three testing datasets, applying a graph-based partitioning strategy reduced the number of merge errors by 36.1\%, 12.2\%, and 13.6\%, respectively.

\begin{table}[h]
	\centering
	\small
	\begin{tabular}{c c c c} \hline
		\textbf{Dataset} & $\Delta$ \textbf{Precision} & $\Delta$ \textbf{Recall} & $\Delta$ \textbf{Accuracy} \\ \hline
		Kasthuri Training & +3.61\% & -0.53\% & +0.60\% \\
		Kasthuri Vol. 2 & +7.59\% & -1.77\% & +1.38\% \\
		FlyEM Vol. 1 & +2.68\% & +0.76\% & +0.66\% \\
		FlyEM Vol. 2 & +2.22\% & -1.05\% & +0.29\% \\ \hline
	\end{tabular}
	\caption{Precision, recall, and accuracy changes between CNN only and CNN paired with graph-optimized reconstructions for the training and three test datasets. The combined method results in better precision and accuracy.}
	\label{table:multicut}
\end{table}


\documentclass[11pt,a4paper]{article}

\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage[left=2cm,right=2cm,top=2cm,bottom=2cm]{geometry}

\usepackage{csquotes}
\usepackage{graphicx}
\graphicspath{ {./figs} }
\usepackage{float}
\usepackage{wrapfig}

\usepackage{caption}
\usepackage{subcaption}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}

\usepackage{mathrsfs}
\usepackage{mathtools}

\usepackage{color}
\usepackage{epsfig}
\usepackage{array}
\usepackage{multicol}
\usepackage{tikz}
\usepackage{listings}
\usepackage{minted}
\usepackage{mdframed}

\setlength{\parindent}{0em}
\setlength{\parskip}{0.5em}
% \textwidth 6.5in
% \textheight 9.in
% \oddsidemargin 0in
% \headheight 0in

\newtheorem{theorem}{Theorem}[section]

\definecolor{codegreen}{rgb}{0,0.6,0}
\definecolor{codegray}{rgb}{0.5,0.5,0.5}
\definecolor{backcolour}{rgb}{0.95,0.95,0.95}

\usepackage[hidelinks]{hyperref}
\hypersetup{
    colorlinks=false, %set true if you want colored links
    linktoc=all,      %set to all if you want both sections & subsections linked
}
\usepackage[nameinlink]{cleveref}

\lstset{ %
  language=python,                % choose the language of the code
  basicstyle=\footnotesize,       % the size of the fonts that are used for the code
  numbers=left,                   % where to put the line-numbers
  numberstyle=\footnotesize,      % the size of the fonts that are used for the line-numbers
  stepnumber=1,                   % the step between two line-numbers. If it is 1 each line will be numbered
  numbersep=5pt,                  % how far the line-numbers are from the code
  backgroundcolor=\color{white},  % choose the background color. You must add \usepackage{color}
  showspaces=false,               % show spaces adding particular underscores
  showstringspaces=false,         % underline spaces within strings
  showtabs=false,                 % show tabs within strings adding particular underscores
  frame=single,                   % adds a frame around the code
  tabsize=2,                      % sets default tabsize to 2 spaces
  captionpos=b,                   % sets the caption-position to bottom
  breaklines=true,                % sets automatic line breaking
  breakatwhitespace=false,        % sets if automatic breaks should only happen at whitespace
  escapeinside={\%*}{*)}          % if you want to add a comment within your code
}

\usemintedstyle{vs}

\begin{document}

\usetikzlibrary{positioning}
\tikzset{every picture/.style={line width=0.75pt}}

\pagestyle{plain}

\begin{multicols}{2}
  \begin{flushleft}
    MAT360 \\
    Autumn 2021\\
    Prof. Jan Martin Nordbotten\\
    \underline{University of Bergen}
  \end{flushleft}
  \vfill\null
  \columnbreak

  \begin{flushright}
    \includegraphics[height=2cm]{assets/uib.logo.png}
  \end{flushright}
\end{multicols}

\begin{center}
\textbf{\large Exemplary FEM implementations and convergence analysis}\\
Paul Stryck\\
\end{center}
\rule{\linewidth}{0.1mm}

\begin{abstract}
    \noindent
    This report presents the numerical validation of a custom FEM implementation with polynomial basis
    functions of arbitrary degree on structured and unstructured grids.
    The Poisson equation will be considered on the unit square.
    Different force functions for which the analytical solution is known will be used.
    Thus, the errors can be computed explicitly, and given convergence rates are verified.
    To cover a broad spectrum of potential implementation errors, both pure Dirichlet and mixed Dirichlet +
    Neumann boundary conditions will be used.
\end{abstract}

\begin{multicols}{2}

\subsection*{Continuous Problem}
The PDE under consideration is the homogeneous Poisson equation:
\begin{equation} \label{eq:poisson}
  \begin{split}
    - \Delta u &= f \quad \text{on } \Omega\\
    u &= 0 \quad \text{on } \partial\Omega
  \end{split}
\end{equation}
Where $\Omega \subset \mathbb{R}^n$ is an open and bounded set and $f \in L^2(\Omega)$.
A classical solution $u \in C^2(\Omega) \cap C^1(\bar{\Omega})$, will at most
exist if $f$ is at least continuous and restrictive assumptions regarding $\Omega$ have to be made.
Thus, a less restrictive formulation of the Problem (\ref{eq:poisson}) based on
the variational formulation will be derived.\\

Multiplying \autoref{eq:poisson} with test functions $v \in C^\infty_0(\Omega)$ and
integrating over $\Omega$ yields:

\begin{equation} \label{eq:poisson_var}
    - \int_\Omega \Delta u v\,dV = \int_\Omega fv\,dV \\ \forall v \in C^\infty_0(\Omega)
\end{equation}

If $\Omega$ has a Lipschitz boundary, \autoref{eq:poisson_var} can be simplified
using Green's first identity.
\begin{multline}\label{eq:poisson_var_greens}
    \int_\Omega \nabla u \cdot \nabla v \,dV
  - \int_{\partial\Omega} v \frac{\partial u}{\partial n} \,dS
  = \int_\Omega fv\,dV \\ \forall v \in C^\infty_0(\Omega)
\end{multline}

Where the boundary integral vanishes since $v = 0$ on $\partial\Omega$.
Additionally, $C^\infty_0(\Omega)$ is dense in $H^1_0(\Omega)$.
So \autoref{eq:poisson_var_greens} can be further simplified to:
\begin{equation} \label{eq:poisson_weak}
  \underbrace{\int_\Omega \nabla u \cdot \nabla v \,dV}_{\eqqcolon a(u,v)}
  = \underbrace{\int_\Omega fv\,dV \quad \forall v \in
  H^1_0(\Omega)}_{\eqqcolon b(v) = \langle b, v \rangle_{H^{-1}_0, H^1_0}}
\end{equation}
A function $u \in H^1_0(\Omega)$ satisfying \autoref{eq:poisson_weak} is called
weak solution of \autoref{eq:poisson}.
Assuming $u \in C^2(\Omega) \cap C^1(\bar{\Omega})$ is a classical solution to
\autoref{eq:poisson}. This implies $u \in H^1_0(\Omega)$ and thus any classical
solution would also satisfy the weak formulation. Thus, the weak formulation
indeed broadens the set of admissible functions.

Existence and uniqueness of a solution to \autoref{eq:poisson_weak} is given by
reformulating \autoref{eq:poisson_weak} to $a(u,v) = \langle b,v \rangle_{H^{-1}_0, H^1_0}$
and application of Lax-Milgram. The proof is omitted here.

\subsection*{Boundary Conditions}
It shall be briefly investigated how boundary conditions can be incorporated.
For this, Dirichlet and mixed Dirichlet + Neumann boundary conditions will be
considered.\\
First, considering Dirichlet boundary conditions, the problem is given by:
\begin{equation} \label{eq:poisson_dirichlet}
  \begin{split}
    -\Delta u &= f  \text{ in } \Omega\\
    u &= g \text{ on } \partial\Omega
  \end{split}
\end{equation}
with $f \in L^2(\Omega)$ and $g \in L^2(\partial\Omega)$.\\
In addition to the already established Hilbert space $H^1_0$ define the set:
\begin{equation*}
    V_g \coloneqq \left\{ v \in H^1(\Omega)\, :\, v\vert_{\partial\Omega} = g \text{ a.e. on } \partial\Omega\right\}
\end{equation*}
Which is well-defined for all $g \in L^2(\partial\Omega)$ by the trace
theorem. $H^1_0$ is of course a Hilbert space, whereas $V_g$ is not since it is
not closed under addition.

By using test functions from $H^1_0(\Omega)$, the weak formulation stays the
same as in \autoref{eq:poisson_weak} however, with solutions $u$ sought in $V_g$.
To obtain such a solution: Define the bilinear form $a$ and linear
functional $b$ as before:
\begin{equation*}
  \begin{split}
    a(u,v) &= \int_\Omega \nabla u \cdot \nabla v \,dx\\
    b(v)   &= \int_\Omega fv\,dx
  \end{split}
\end{equation*}

Now choose an arbitrary $\hat{g}\in V_g$ and solve:
\begin{equation*}
  a(\hat{u},v) = b(v) - a(\hat{g},v) \quad \forall v \in H^1_0(\Omega)
\end{equation*}
Where the existence of a unique solution $\hat{u}$ is guaranteed by Lax-Milgram.

The weak solution to \autoref{eq:poisson_dirichlet} is obtained by:
$$u = \hat{u} + \hat{g} \in V_g$$

\subsubsection*{Neumann Boundary Conditions}
To incorporate Neumann boundary conditions the boundary $\partial\Omega$ needs
to be disjointly split into $\Gamma_0$ and $\Gamma_1$, such that
$\Gamma_0 \cup \Gamma_1 = \partial\Omega$ and $\Gamma_0 \cap \Gamma_1 = \emptyset$.
The Poisson equation with mixed Neumann and homogeneous Dirichlet boundary
conditions can then be stated as:
\begin{equation} \label{eq:poisson_neumann}
  \begin{split}
    -\Delta u &= f  \text{ in } \Omega \\
    u &= 0 \text{ on } \Gamma_0 \\
    \frac{\partial u}{\partial n} &= h \text{ on } \Gamma_1, \quad h\in L^2(\Gamma_1)
  \end{split}
\end{equation}

The weak formulation of \autoref{eq:poisson_neumann} is given by:
\begin{multline}
  \int_\Omega \nabla u \cdot \nabla v\,dx
  = \int_\Omega fv\,dx + \int_{\Gamma_1}hv\,dx \\
  \forall v \in \left\{ v \in H^1(\Omega)\, :\, v\vert_{\Gamma_0} = 0 \text{ a.e. on } \Gamma_0\right\}
\end{multline}

By application of Lax-Milgram, it can be verified that \autoref{eq:poisson_neumann}
admits a unique, weak solution if $\int_{\Gamma_0}1\,dx > 0$.
In the case of pure Neumann conditions Lax-Milgram cannot be applied, since the
bilinear form is no longer coercive and solutions are only unique up to a constant.
To extend this to the case of $u = g$ on $\Gamma_0$ for some $g \in L^2(\Gamma_0)$
the same idea used for pure Dirichlet boundary conditions can be used.

\subsection*{Discretizing the Problem}
To obtain numerical solutions to \autoref{eq:poisson_weak}, the problem has to be
discretized. As by the lecture, the Hilbert space $H^1(\Omega)$ can be
successively approximated by some sequence of finite dimensional subspaces
$(V_i)_{i\in \mathbb{N}}$.\\
For now, the $V_i$ will be the space of piecewise linear, continuous
polynomials defined on a set of nodes obtain by a triangulation of $\Omega$.
Under the assumption that the triangulation is conforming and no triangles
collapse, it has been shown in the lecture that $\lim\limits_{i\to\infty}V_i \to
H^1(\Omega)$.

By forcing all basis functions to attain a certain value on the boundary, an
approximation to the set $V_g$ can be obtained. By forcing them to 0 on the
boundary, the space $H^1_0(\Omega)$ can be obtained and
$\lim\limits_{i\to\infty}V_i \to H^1_0(\Omega)$

Thus, the continuous problem $a(u,v) = b(v) \quad \forall v \in H^1_0(\Omega)$
can be approximated by the finite dimensional problem $a(u_i,v_i) = b(v_i)$ for all
basis vectors $v_i \in V_i$ (and explicitly enforcing the 0 boundary condition).

Since there are only a finite number of basis functions, this results in a linear system
where the properties of the system heavily depend on the choice of basis of $V_i$.

In this work, isoparametric Lagrange elements are chosen as a basis,
which results in a sparse system.

\section*{Numerical Validation with 1\textsuperscript{st} Order Piecewise Polynomials}
Different convergence rates, shown in the lecture are given by:
\begin{equation}
  \begin{split}
    \left|u-u_h\right|_{H^1(\Omega)} &\lesssim h \lVert u\rVert^2_{H^2(\Omega)}\\
    \lVert u - u_h \rVert_{L^2(\Omega)} &\lesssim h\lVert u\rVert^2_{H^1(\Omega)}\\
    \lVert u - u_h \rVert_{L^2(\Omega)} &\lesssim h^2\lVert u\rVert^2_{H^2(\Omega)}
  \end{split}
\end{equation}
Where $h$ is depended on the grid and $u \in H^2(\Omega)$ or $u \in H^1(\Omega)$
is the analytical solution to $a(u,v) = b(v)$.
$u_h$ is the solution to the discretized problem.

\subsection*{Smooth Solutions $u \in H^2$}
Throughout this report, all numerical experiments concerned with convergence rely
on a gradually refined grid. Here, $n$ will always denote the number of vertices
per edge on the unit square. Resulting in $n^2$ vertices in total and thus $h \sim \frac{1}{n}$.
How such refinements for structured and unstructured grids works can be seen in \autoref{fig:grids}

\begin{figure}[H]
  \centering
  \begin{subfigure}{.5\linewidth}
    \centering
    \includegraphics[width=.9\linewidth]{structured_grids}
    \caption{Structured Grids}
  \end{subfigure}%
  \begin{subfigure}{.5\linewidth}
    \centering
    \includegraphics[width=.9\linewidth]{unstructured_grids}
    \caption{Unstructured Grids}
  \end{subfigure}
  \caption{Grid Refinements for $n=4$ and $n=6$}
  \label{fig:grids}
\end{figure}

Considering the following problem:
\begin{equation}
  \label{eq:smooth_poisson_prob}
  \begin{split}
    \Omega &= \left[0,1\right]^2\\
    -\Delta u &= f\\
    u &= 0 \quad \text{on } \partial \Omega\\
  \end{split}
\end{equation}

With given solution
\begin{multline*}
  u(x,y) = 2^{4a} x^a (1-x)^a y^a (1-y)^a \quad a \in \mathbb{N}\\
  u \in H^2(\Omega)\,\forall a > 0
\end{multline*}
and $f$ accordingly.
This solution has the useful property
$$\lVert u \rVert_{H^2(\Omega)} \sim a.$$
This makes it ideal to verify the above convergence rates.

\begin{figure}[H]
  \centering
  \begin{subfigure}{.5\linewidth}
    \centering
    \includegraphics[width=.9\linewidth]{contour_1}
    \caption{$a = 1$}
  \end{subfigure}%
  \begin{subfigure}{.5\linewidth}
    \centering
    \includegraphics[width=.9\linewidth]{contour_10}
    \caption{$a = 10$}
  \end{subfigure}
  \caption{Solution $u$ with varying parameter $a$}
  \label{fig:smooth_dirichlet_solution}
\end{figure}

As seen in \autoref{fig:smooth_dirichlet_solution}, $u$ is a smooth bump
function where the steepness increases with $a$.
We expect a convergence rate of order $C\cdot h$ for the $H^1$ semi-norm
and $C\cdot h^2$ for the $L^2$ norm since $u \in H^2(\Omega) \forall a > 0$.
C is expected to increase with $a$ as it should stand in some relation to $\lVert u \rVert_{H^2}$.
For both, structured (\ref{fig:smooth_dirichlet_errs_str}) and unstructured (\ref{fig:smooth_dirichlet_errs_unstr})
grids exactly this is observed.

\begin{figure}[H]
  \centering
  \begin{subfigure}{1\linewidth}
    \centering
    \includegraphics[width=1\linewidth]{errors_smooth_reg}
    \caption{Structured Grids}
    \label{fig:smooth_dirichlet_errs_str}
  \end{subfigure}

  \begin{subfigure}{1\linewidth}
    \centering
    \includegraphics[width=1\linewidth]{errors_smooth_irreg}
    \caption{Unstructured Grids}
    \label{fig:smooth_dirichlet_errs_unstr}
  \end{subfigure}
  \label{fig:smooth_dirichlet_errs}
  \caption{$L^2$ Norm and $H^1$ Semi-Norm Errors for Smooth Solutions on Structured and Unstructured Grids}
\end{figure}

Where the overall error of the solution on the unstructured grid is significantly worse.
This is due to the fact, that no special care has been taken to attain a good quality mesh.

\subsection*{Less Regular Forces}
Considering the following problem with mixed Dirichlet and Neumann boundary conditions.
\begin{equation}
  \label{eq:poisson_less_smooth_prob}
  \begin{split}
    \Omega &= \left[-1,1\right]^2\\
    -\Delta u &= \begin{cases}
      \frac{\pi^2}{4} \operatorname{cos}\left(\frac{\pi}{2} \cdot y\right) \quad &x \le 0\\
      \frac{\pi^2}{4} \operatorname{cos}\left(\frac{\pi}{2} \cdot y\right) - a\left(a-1\right)x^{a-2} \quad &x > 0
    \end{cases}\\
    u(x,y) &= 0 \quad \text{on } [-1,0] \times -1 \cup [-1,0] \times 1\\
    u(x,y) &= x^a \quad \text{on } (0,1] \times -1 \cup (0,1] \times 1\\
    \frac{\partial u}{\partial {\bf n}} &= 0 \quad \text{on } -1 \times (-1,1)\\
    \frac{\partial u}{\partial {\bf n}} &= a \quad \text{on } 1 \times (-1,1)
  \end{split}
\end{equation}
With known solution:
\begin{equation*}
  u(x,y) = \begin{cases}
    \operatorname{cos}\left(\frac{\pi}{2} \cdot y\right) \quad &x \le 0\\
    \operatorname{cos}\left(\frac{\pi}{2} \cdot y\right) + x^a \quad &x > 0
  \end{cases}
\end{equation*}

Where
$$u \in H^{a+0.5 - \epsilon}(\Omega) \quad \forall \epsilon > 0.$$
With increasing $a$, the function $\Delta u$ will develop a singularity along the $x = 0$ line.
\begin{equation}
  \begin{split}
    a \le 1: \quad &u \notin C^0(\Omega)\\
    1 < a \le 1.5: \quad &u \in H^1(\Omega), u \notin H^2(\Omega) \\
    1.5 < a: \quad &u\in H^2(\Omega)
  \end{split}
\end{equation}
$a > 1$ is needed, so the interpolation operator is well-defined, and our established
convergence estimates make sense.
The varying regularity of the boundary conditions does not pose a problem, since only
square integrability for both, the Neumann, and Dirichlet Boundary is needed.

\begin{figure}[H]
  \centering
  \begin{subfigure}{.5\linewidth}
    \centering
    \includegraphics[width=.9\linewidth]{contour_nonsmooth_1}
    \caption{$a = 1.1$}
  \end{subfigure}%
  \begin{subfigure}{.5\linewidth}
    \centering
    \includegraphics[width=.9\linewidth]{contour_nonsmooth_2}
    \caption{$a = 2$}
  \end{subfigure}
  \caption{Solution $u$ with varying parameter $a$}
  \label{fig:non_smooth_mixed_solution}
\end{figure}

In the $H^1$ semi-norm, linear convergence is expected for $1.5 < a$.
For $1 < a \le 1.5$ we have $u \notin H^2(\Omega)$ and linear convergence in the $L^2$ norm at best.
And for $1.5 < a$ quadratic convergence in the $L^2$ norm at best.
However, the theorem guaranteeing a convergence estimates in its precise form states:
\begin{multline}
    \exists n \in \mathbb{N}: \, \lVert u - u_h \rVert_{L^2} \in \mathcal{O}\left(h\right), \\ \forall h < \frac{1}{n}, u \in H^1(\Omega)
\end{multline}
\begin{multline}
    \exists n \in \mathbb{N}: \, \lVert u - u_h \rVert_{L^2} \in \mathcal{O}\left(h^2\right), \\ \forall h < \frac{1}{n}, u \in H^2(\Omega)
\end{multline}
This threshold is likely to increase with less regularity of the solution. So a more natural
expectation is for the convergence rate to increase with increasing $a$.

Exactly linear convergence is observed in \autoref{fig:err_nonsmooth_h1} for the $H^1$ semi-norm and $1.5 < a$.
For $a = 1.3$, it still appears to converge but at a rate of $\mathcal{O}\left(h^{0.6}\right)$.

In \autoref{fig:err_nonsmooth_l2} it can be observed how the convergence rate depends on the regularity of the solution.
For the highly irregular solution with $a = 1.3$, linear convergence is only given during the first steps.
A similar behaviour is observed for $1.5 < a < 2.2$ where quadratic convergence is expected, but only given
during the first steps.
Truly quadratic convergence can only be observed for $2.2 < a$.
The estimated convergence rates are not exactly observed for the borderline cases.
This could be a hint to the fact that our estimates should be improved, or the needed threshold from where on the
convergence sets in is not reached if the function is only just in the needed space.

\begin{figure}[H]
  \centering
  \begin{subfigure}{1\linewidth}
    \centering
    \includegraphics[width=.8\linewidth]{errors_nonsmooth_h1}
    \caption{Convergence in the $H^1$ Semi-Norm}
    \label{fig:err_nonsmooth_h1}
  \end{subfigure}

  \begin{subfigure}{1\linewidth}
    \centering
    \includegraphics[width=.8\linewidth]{errors_nonsmooth_l2}
    \caption{Convergence in the $L^2$ Norm}
    \label{fig:err_nonsmooth_l2}
  \end{subfigure}
  \caption{Convergence With Varying Regularity of the Solution}
\end{figure}

\subsection*{Higher Order Elements}
To increase the accuracy of the solution on a coarser mesh, a polynomial basis
of higher order can be chosen.
This may be achieved by considering higher order local basis functions defined on the reference element.
So far, linear basis functions had been considered. These local basis functions are visualized in \autoref{fig:p1_2d_mesh_basis}.
\begin{figure}[H]
  \centering
  \includegraphics[width=.9\linewidth]{p1_2d_mesh_basis}
  \caption{Polynomial Basis of Degree 1 on 2-Simplex}
  \label{fig:p1_2d_mesh_basis}
\end{figure}

To obtain a basis for the entire space on the mesh, those local basis functions are mapped to each simplex.
Global continuity is ensured by joining enough nodes on the boundaries of adjacent simplexes.
3 basis functions of the function space defined on the entire mesh are visualized in \autoref{fig:p1_2d_stitch_basis}.
\begin{figure}[H]
  \centering
  \includegraphics[width=.9\linewidth]{p1_2d_stitch_basis}
  \caption{3 Piecewise Linear Basis Functions of the Function Space defined on the entire Grid}
  \label{fig:p1_2d_stitch_basis}
\end{figure}

Generalizing this approach to higher order Polynomials requires the definition of higher order polynomial
basis functions on the reference element. For this
$$N_p = \frac{(p+d)!}{p! \cdot d!}$$
nodes have to be chosen on each element. Where $p$ is the degree of the polynomial basis and $d$ is the spatial dimension.

To ensure regularity along adjacent simplexes on the mesh,
$$\tilde{N}_{p} = \frac{(p+(d-1))!}{p!\cdot(d-1)!}$$
nodes have to lie on each boundary segment of the reference simplex.

Exemplary local basis functions for $p=2$ and $p=3$ are visualized in \autoref{fig:higher_order_basis}.
\begin{figure}[H]
  \centering
  \begin{subfigure}{.49\linewidth}
    \includegraphics[width=1\linewidth]{p2_basis}
    \caption{$p=2$ Local Basis Function}
  \end{subfigure}\hfill%
  \begin{subfigure}{.49\linewidth}
    \raggedright
    \includegraphics[width=1\linewidth]{p3_basis}
    \caption{$p=3$ Local Basis Function}
  \end{subfigure}
  \caption{Higher Order Local Basis Functions}
  \label{fig:higher_order_basis}
\end{figure}

To compare how higher order elements perform compared to their linear counterparts,
the same numerical experiments as for the piecewise linears will be repeated with higher order elements.

The following estimate for the convergence rates of $p$\textsuperscript{th} order
elements has been established during the lecture:
\begin{equation}
  \label{eq:conv_high_order}
  \lVert u - u_h \rVert_{L^2} \lesssim h^{k'} \lVert u \rVert_{H^{k'+1}}
\end{equation}
Where $k' = \operatorname{min}\left(p,m\right)$ and $m$ is the largest integer s.t. $u \in H^{m+1}$.

By using Problem (\ref{eq:smooth_poisson_prob}) and (\ref{eq:poisson_less_smooth_prob})
again, the parameter $a$ can be adjusted easily to verify the above convergence
rate for higher order elements.

\subsubsection*{Smooth Solutions}
Using the problem (\ref{eq:smooth_poisson_prob}) with $a = 5$ we have $u \in H^k(\Omega) \forall k \in \mathbb{N}$.
Thus, a convergence rate of $ \sim h^{p+1}$ for a polynomial basis of arbitrary degree $p$ in the $L^2$ norm is expected.
This is verified on a regular grid in \autoref{fig:err_smooth_highorder_l2}.
\begin{figure}[H]
  \centering
  \begin{subfigure}{1\linewidth}
    \centering
    \includegraphics[width=.8\linewidth]{errors_smooth_highorder_h1}
    \caption{Convergence in the $H^1$ Semi-Norm}
    \label{fig:err_smooth_highorder_h1}
  \end{subfigure}

  \begin{subfigure}{1\linewidth}
    \centering
    \includegraphics[width=.8\linewidth]{errors_smooth_highorder_l2}
    \caption{Convergence in the $L^2$ Norm}
    \label{fig:err_smooth_highorder_l2}
  \end{subfigure}
  \caption{Convergence With Smooth Solutions and Higher Order Elements}
\end{figure}

Although, no specific error bound for the $H^1$ semi-norm was given, the experiments in \autoref{fig:err_smooth_highorder_h1} suggest,
that under the same assumptions as in \autoref{eq:conv_high_order}, we see convergence in $H^1$ semi-norm
with one order less.

It can be observed, that basis functions of degree $8$ and $10$ do converge very fast.
However, they stop increasing in accuracy after just a few refinement steps which is probably due to other numerical errors
associated with bad conditioning of the Vandermonde matrix and its inverse used to evaluate the basis functions.

\subsubsection*{Varying Regularity}
By, again, considering Problem (\ref{eq:poisson_less_smooth_prob}) and altering $a$, the different
convergence rates for varying regularity can be studied for local bases of different degrees.
The convergence estimate (\ref{eq:conv_high_order}) tells that for less regular solutions,
the convergence rate is not expected to increase when using higher order polynomials.
This is observed in \autoref{fig:err_ridge_l2}.

As in the smooth case, convergence in the $H^1$ semi-norm is also given at one order less than in the $L^2$ norm.
Even though no such convergence rate had been theoretically established.
\begin{figure}[H]
  \centering
  \begin{subfigure}{1\linewidth}
    \centering
    \includegraphics[width=.8\linewidth]{errors_ridge_26_l2}
  \end{subfigure}

  \begin{subfigure}{1\linewidth}
    \centering
    \includegraphics[width=.8\linewidth]{errors_ridge_36_l2}
  \end{subfigure}

  \begin{subfigure}{1\linewidth}
    \centering
    \includegraphics[width=.8\linewidth]{errors_ridge_46_l2}
  \end{subfigure}

  \begin{subfigure}{1\linewidth}
    \centering
    \includegraphics[width=.8\linewidth]{errors_ridge_66_l2}
  \end{subfigure}

  \begin{subfigure}{1\linewidth}
    \centering
    \includegraphics[width=.8\linewidth]{errors_ridge_106_l2}
  \end{subfigure}

  \caption{Convergence With Increasing Smoothness and Higher Order Elements}
  \label{fig:err_ridge_l2}
\end{figure}

\begin{figure}[H]
  \centering
  \begin{subfigure}{1\linewidth}
    \centering
    \includegraphics[width=.8\linewidth]{errors_ridge_26_h1}
  \end{subfigure}

  \begin{subfigure}{1\linewidth}
    \centering
    \includegraphics[width=.8\linewidth]{errors_ridge_36_h1}
  \end{subfigure}

  \begin{subfigure}{1\linewidth}
    \centering
    \includegraphics[width=.8\linewidth]{errors_ridge_46_h1}
  \end{subfigure}

  \caption{$H^1$ Semi-Norm Convergence With Increasing Smoothness and Higher Order Elements}
  \label{fig:err_ridge_h1}
\end{figure}

\subsection*{Numerical Challenges}
\subsubsection*{Choice of Basis}
An important step, is the choice of basis functions on the reference element, and how to represent this basis.
In this report, Lagrange elements of varying degree have been used.
These are defined by a set of distinct nodes $X \coloneqq \left\{x_i \in \mathbb{R}^d, i = 1,\cdots, N_p\right\}$ on the reference element.
And the $N_p$ basis functions are defined as:
$$\Phi_i(x_j) = \delta_{i,j}.$$
Where certain restrictions apply on how many nodes must be placed on the boundary.

As the so defined basis does not allow for point evaluation at arbitrary points on the reference element,
those $\Phi_i$ must be explicitly computed.
One now has two choices to make: Where to place the nodes, and in which polynomial basis to represent the $\Phi_i$.
The most obvious and popular choice, which was also used in this work, is using equally spaced nodes and the monomial
basis for $\Phi_i$.

To obtain $\Phi_i$ in the monomial basis, one could compute the inverse of the Vandermonde matrix evaluated at all
nodes $x_i \in X$. Unfortunately, the condition number of the Vandermonde matrix in monomial basis increases rapidly
as seen in \autoref{fig:vandermonde_cond}. For higher order elements this introduces a significant error in the point evaluation
which then propagates to the integration process and thus the solution.
\begin{figure}[H]
  \centering
  \includegraphics[width=1\linewidth]{vandermonde_cond}
  \caption{Condition Number of Vandermonde Matrix for increasing Polynomial Degree}
  \label{fig:vandermonde_cond}
\end{figure}

\subsubsection*{Singularities}
For weak solutions of \autoref{eq:poisson_weak} to exist, it suffices for the rhs to be square integrable.
Thus, $f$ could potentially be infinite at any countably many points
and point-evaluation of $f$ is not well-defined. However, the integration schemes employed by this
implementation are based on quadrature rules, which rely on point-evaluation. This is remedied by
simply setting all infinite values to $0$, which represents $f$ by a different function from
the same equivalence class, thus not changing the equation. And the numerical experiments suggest, that
this will still lead to convergence to the right solution. As long as only a few points are affected.

Numerical evidence suggests, that elements spanning the singularity give better convergence as seen in
\autoref{fig:oscilation}. Where for odd $n$ vertices are placed on the singularity and for $n$ even they are placed right next to it.
\begin{figure}[H]
  \centering
  \includegraphics[width=1\linewidth]{oscilation}
  \caption{Oscillating Error depending on meshing of the singularity}
  \label{fig:oscilation}
\end{figure}

\subsubsection*{Submanifolds}
As outlined during the Wednesday lectures, the entire theory is also applicable to immersed manifolds.
As long as they are in some sense well-behaved.
\begin{wrapfigure}{r}{0.5\linewidth}
  \begin{center}
    \includegraphics[width=1\linewidth]{submanifold}
    \caption{Solution for Smooth Bump Equation on Submanifold}
  \end{center}
\end{wrapfigure}
The code has been adapted to allow the definition of finite element function spaces on submanifolds.
However, the standard reference elements are not well suited for this, because the curvature of the
submanifold is lost during the triangulation process.

An alternative could be to use non-linear mappings from the reference simplex to the mesh elements.
This complicates the integration process a lot, since the pullbacks to the reference element become
computationally harder to evaluate.

\subsection*{Conclusion}
The Code has been verified by a variety of different problems.
Using different boundary conditions, varying regularity of the force function
and differently structured grids.

The given convergence estimates are exactly met, when expecting solutions clearly falling
into the needed class of functions.
When using borderline cases, where the solution is only just about regular enough,
or just not regular enough any more, the convergence estimates are only exactly met
for a finite amount of steps before the convergence slightly deviates from the expected rate.
As this only happens for edge cases and given the roughness of the estimates, this is not worrying.

To further optimize the implementation, the integration of non-linear transforms of reference elements could be
considered for immersed submanifolds.
To optimize the implementation speed, the global node numbering algorithm should be reconsidered
to optimize the sparsity pattern of the resulting matrix and to optimize memory access.

But as a proof of concept and a tool to investigate convergence rates of higher order elements,
this implementation is well suited.

\end{multicols}
\end{document}


\documentclass[]{article}

\author{Braden Fineberg}
\usepackage[pdftex]{graphicx}
\usepackage{achicago}
\usepackage{fullpage}
\usepackage{ amssymb }
\usepackage{amsmath}
%\usepackage[top=1in, bottom=1in, left= 1 in, right=1in]{geometry}
\usepackage{setspace}
\usepackage[nopar]{lipsum} % for dummy text
\newcommand{\HRule}{\rule{\linewidth}{0.25mm}}

\usepackage{listings}
\usepackage{subfig}
\usepackage{color} %red, green, blue, yellow, cyan, magenta, black, white
\definecolor{mygreen}{RGB}{28,172,0} % color values Red, Green, Blue
\definecolor{mylilas}{RGB}{170,55,241}
\renewcommand\thesection{\Alph{section}}

\begin{document}

	\lstset{language=Matlab,%
		%basicstyle=\color{red},
		breaklines=true,%
		morekeywords={matlab2tikz},
		keywordstyle=\color{blue},%
		morekeywords=[2]{1}, keywordstyle=[2]{\color{black}},
		identifierstyle=\color{black},%
		stringstyle=\color{mylilas},
		commentstyle=\color{mygreen},%
		showstringspaces=false,%without this there will be a symbol in the places where there is a space
		numbers=left,%
		numberstyle={\tiny \color{black}},% size of the numbers
		numbersep=9pt, % this defines how far the numbers are from the text
		emph=[1]{for,end,break},emphstyle=[1]\color{red}, %some words to emphasise
		%emph=[2]{word1,word2}, emphstyle=[2]{style},
		basicstyle=\tiny,
	}

\input{../title.tex}

\section{Derivation of (3) and (4)}

Given the fact that the price of a stock can be viewed as a random walk, a form of Brownian Motion, we know that there is some likelihood that tomorrows close price will move up or down in a roughly normal distribution. This concept, combined with the Markovian nature of stock movements allows us to assume that the price is Brownian. Now given the fact that the price is Brownian, with mean $\mu h$ and variance $\sigma^2 h$, we know that the expected change in the stocks price tomorrow is $\mu h$. The equations of (3) and (4) compute the unbiased mean and variance of the Brownian motion over time by looking at independent (due to Markovian stock prices) days of stock change. They capitalize on the central limit theory to compute an expected mean and variance normalized by the number of samples and a scalar h.

\section{Determination of drift and volatility}
Drift and volatility are the difference of two logs. Using the following code, I calculated mu and sigma.

\begin{lstlisting}
drift = log(close_price(2:end)) - log(close_price(1:end-1));
N = length(drift);
h = 1/365; %stock prices close one day apart

mu = sum(drift)/(N*h);
mu

sigma = sum((drift-mu*h).^2)/((N-1)*h);
sigma
\end{lstlisting}

The calculated mean is 0.6275. The variance is 0.2174.

\section{Is Geometric Brownian motion a good model?}
Using the following code, I tested normality. I found the QQ-Plot to be most informative. This data can be considered normal, but it is not strongly normal.

\begin{lstlisting}
close all;
x = -.12:.01:.12;
n = histcounts(drift, x);

figure();
bar(x(1:end-1), n/N/.01);
xlim([-.12 .12]);
hold on;
norm = normpdf(x,mu*h,sqrt(sigma*h));
plot(x, norm, 'linewidth', 2, 'color','r')
title('Normalized Daily Drift vs. Expected Brownian');
saveas(gcf, 'brownianHist.png');

figure();
qqplot(drift);
title('QQ Plot to Test Normality');
saveas(gcf, 'qqplot.png');
\end{lstlisting}

\begin{figure}[!ht]
	\centering
	\begin{minipage}{.5\textwidth}
		\centering
		\includegraphics[width=0.8\linewidth]{qqplot}
		\caption{Determining Brownian Fit}
		\label{fig:test1}
	\end{minipage}%
	\begin{minipage}{.5\textwidth}
		\centering
		\includegraphics[width=0.8\linewidth]{brownianHist}
		\caption{Normalized Distribution}
		\label{fig:test2}
	\end{minipage}
\end{figure}

\section{Expected return}
Expected return is the log ratio of the return from investing today (t=0) and some future time t minus the market rate. This will determine the premium return.

\begin{align}
E(return) =& log(\frac{X(t)e^{-\alpha t}}{X(0)})\\
=& (\mu + \sigma^2/2 - \alpha)t\\
=& (0.6275+ 0.2174/2 - 0.375)t\\
=& 0.6987 \qquad \square
\end{align}

The likelihood of seeing a 5\% return next year would require an actual rate of return of 8.75\%. Given the fact that we can approximate this using Brownian motion, the probability of the return being greater than 8.75\% can be written as:

\begin{align}
P\{return > .0875\} =& 1-\Phi(\frac{.0875 - 0.6275}{\sqrt{ 0.2174}})\\
=& 1-\Phi(-1.15814)\\
=& 87.66\%
\end{align}

\section{Risk neutral measure}
Risk, or beta, is essentially a measure of volatility of the stock above market rate. It can be equated as:
$$\alpha - \sigma^2/2 = .0375 - 0.2174/2 = -0.0712$$

The volatility of the stock remains 0.2174.

\section{Expected return for risk neutral measure}
Given that the expectation of risk-neutral return, $\mu$, is negative, I would choose not to invest, therefore, my return would be \$0. I would instead invest in the market and see a return of $\alpha$, 3.75\%.

\section{Derive Black-Scholes formula}

\begin{align}
\mathbb{E}_q[e^{-\alpha t }[X(t)-K]^+-c]=&0\\
c =& e^{-\alpha t } \mathbb{E}_q[X(t)-K]^+\\
=& e^{-\alpha t } \mathbb{E}_q[X_0e^{Y(t)}-K]^+\\
=& e^{-\alpha t } \frac{1}{\sqrt{2\pi t \sigma^2}}\int_{-\infty}^{\infty} (X_0e^{Y(t)}-K)^+e^{\frac{-(y-(\mu t)^2)}{2t\sigma^2}}\\
=& e^{-\alpha t } \frac{1}{\sqrt{2\pi t \sigma^2}}\int_{log(K/X_0)}^{\infty} (X_0e^{Y(t)}-K)^+e^{\frac{-(y-(\mu t)^2)}{2t\sigma^2}}\\
\text{Given the following change of variabeles:}&\\
\text{let } z =  (y - \mu t)/\sqrt{t\sigma^2}&\\
y = \sqrt{t\sigma^2}z + \mu t&\\
dy = \sqrt{t\sigma^2}dz&\\
\text{int limit: } a = \frac{log(K/X_0)-\mu t}{\sqrt{t\sigma^2}}&\\
=& e^{-\alpha t } \frac{1}{\sqrt{2\pi}}\int_{a}^{\infty} (X_0e^{\sqrt{t\sigma^2}z + \mu t}-K)^+e^{\frac{z^2}{2}}dz\\
let \quad l1 =& \frac{1}{\sqrt{2\pi}}\int_{a}^{\infty} (X_0e^{\sqrt{t\sigma^2}z + \mu t})e^{\frac{z^2}{2}}dz\\
=& \frac{X_0e^{\mu t+t\sigma^2/2}}{\sqrt{2\pi}}\int_{a}^{\infty} e^{-(z-\sqrt{t\sigma^2})/2)}dz\\
\text{Given a change of variables: }=& \\
let \qquad u = z-\sqrt{t\sigma^2}&\\
du = dz&\\
\text{int limit: } b = \frac{log(K/X_0)-\mu t}{\sqrt{t\sigma^2}} - \sqrt{t\sigma^2}&\\
=& \frac{X_0e^{\mu t+t\sigma^2/2}}{\sqrt{2\pi}}\int_{b}^{\infty} e^{-u/2}du\\
=&X_0e^{\mu t+t\sigma^2/2}Q(b)\\
let \quad l2 =&\frac{K}{\sqrt{2\pi}}\int_{a}^{\infty}e^{\frac{z^2}{2}}dz = KQ(a)\\
c =& e^{-\alpha t }(l1-l2) = e^{-\alpha t }(X_0e^{\mu t+t\sigma^2/2}Q(b) - KQ(a))\\
c=& X_0Q(a - \sqrt{t \sigma^2}) - e^{-\alpha t }KQ(a) \qquad \square
\end{align}

\section{Determine option price}

Using the following code, I determined c to be 0.7190, 0.2941, and 0.1243 for K =0.8, 1, 1.2 respectively.

\begin{lstlisting}
alph = .0375;

X0 = close_price(1);
Ex = X0*exp(mu+var/2);
K = [0.8 1 1.2]*Ex;
c = zeros(1, length(K));
risk_nuetral = alph - var/2;

for k=K
a = (log(k/X0)-risk_nuetral)/sqrt(var);
b = a - sqrt(var);
QA = 1-normcdf(a, 0, 1);
QB = 1 - normcdf(b, 0, 1);

c(K==k) = X0*QB-exp(-alph)*k*QA;
end

disp(option_price);
\end{lstlisting}

\end{document}


\documentclass{article}
\usepackage{tocloft}
\include{common_symbols_and_format}
\renewcommand{\cfttoctitlefont}{\Large\bfseries}

\begin{document}
\logo
\rulename{Bollinger Bands} %Argument is name of rule
\tblofcontents

\ruledescription{Bollinger Bands developed by John Bolinger in the 1980s, is a technical indicator that is used to identify when an asset is overbought and oversold. The bands comprise a volatility indicator that measures the relative high or low of a security’s price in relation to previous trades. It is comprised of the upper, middle, and lower band which envelopes the price range levels. An asset is considered oversold when it breaks below the lower band and considered overbought when it breaks above the upper band.
}

\howtotrade
{The strategy is to identify asset's price cycles.
Bullish Reversal - when Price is above the upper band \&
Bearish Reversal - when Price is below the lower band.
}

\ruleparameters %You can include however many arguments (in groups of 4) as you want!
{Look Back Length}{20}{Look back length used to compute MA.}{$\lookbacklength$}
{Number of standard deviations}{2}{Number of standard deviation used to compute Bollinger Band limits.}{$m$}
\stoptable %must be included or Tex engine runs infinitely

\newpage
\section{Equation}
Below are the equations which govern how this specific trading rule calculates a trading position.

\begin{equation}
    P_{n} = \frac{H_{n} + L_{n} + C_{n}}{3}
\end{equation}
\begin{equation}
    MA(P_{n}, n) = \frac{1}{n} \sum_{i = 1}^{n} P_{i}
\end{equation}
\begin{equation}
    B_{u} = MA(P_{n}, \lookbacklength) + m \times \sigma[P_{n}, \lookbacklength]
\end{equation}
\begin{equation}
    B_{l} = MA(P_{n}, \lookbacklength) - m \times \sigma[P_{n}, \lookbacklength]
\end{equation}
\\ % creates some space after equation
where:

$P_{n}$: is the price at $n$th period.

$H_{n}$: is the highest price at $n$th period.

$L_{n}$: is the lowest price at $n$th period.

$C_{n}$: is the closing period at $n$th period.

$m$: is the number of standard deviations.

$\sigma[P_{n}, \lookbacklength]$: is the standard deviation in $P_{n}$ within \lookbacklength \ periods.

\keyterms
\furtherlinks %The footer
\end{document}