\section{[Internal use] : Weaknesses people may ask about at LVC}

* Prior on sky is skymap

* Discrete sky grid: biases? What about high SNR? (A: just use adapted resolution; not ready yet)

* Uniform weighting

* Not discarding values after burnin

* Adaptation algorithm: why parameters chosen?

* Computational cost: is this really favorable in CPU-hours?  Are we wasting CPU hours to achieve low-latency that's not needed?

** A: Not clear our overall cost is higher...we may be more accurate than 1000 samples, the usual metric for MCMC...we need a clear comparison

* Rotation of the earth

\subsection{Runtime}

* Might be ram-limited in python below 10-ish Hz (multiple harmonics and IFOs, sampled at 16kHz for 30 min)
%	1byte/sample * 16384*30min *60s/min ~ 28 Mb

\subsection{Applications}

\textbf{These need to be mentioned on the CBC project page}

* Einstein at home scaling?

* GRB project with Alex (already on table)

* [partial] Port likelihood into lalinference, see if they can get any improvement with it, mainly for precessing systems.

\section{[Internal use] Postprocessing details}
Notes, possibly for integration with main text
\begin{widetext}

\noindent \textbf{Doing the integral: Polar coordinates for linear spoked integration}: Our spoked mass grid is
naturally interpolated in polar coordinates $r,\theta$.  Keeping in mind the change-of-coordinates to align the
principal axes of our error ellipsoid with the data, we can evaluate the evidence integral as
\begin{align}
(\mc,\eta)_a &= [\sqrt{\Gamma}]_{ab}  (r\cos \theta,r\sin \theta)_b \\
\int d\mc \eta p(\mc,\eta) \LikeRed(\mc,\eta) &= \int |\Gamma| r d\theta \; p(\mc(r,\theta),\eta(r, \theta)) \LikeRed(\ldots)
\end{align}
where the matrix square root of the Fisher matrix is used to change coordinates to (scaled) principal axes of the ellipse:
\[
\sqrt{\Gamma}_{ab} = \sqrt{\gamma}_1 \hat{v}_{1,a}\otimes v_{1,a} + \sqrt{\gamma_2} v_{2b}\otimes v_{2b}
\]
where $\{\gamma_k,v_k\}$ is the eigensystem of $\Gamma$.

\end{widetext}

\section{[Internal use] Monte Carlo integration}

\subsection{Skymaps}
\noindent \textbf{[Skymaps]  Discretizing a volume} We integrate over the sky by translating a continuous distribution into
discrete, equal-area pixels.  Specifically, we start with the abstract process of breaking an integral over a volume
$\Omega$ in the sky coordinates ($\gamma$)  up into nonoveralapping
volumes $A_n$ with $\cup A_n = \Omega$  , where the extrinsic volumes are small enough that we can treat any expected
function (e.g., $L,p$) can be approximated as nearly independent of $\gamma$ in $A_n$:
\begin{widetext}
\begin{align}
\LikeRed(\lambda)
& = \int_{V\times\Omega}\Like(\lambda,\theta,\gamma)p(\theta)p(\gamma) d \theta d\gamma
%\\&
=  \sum_{n=1}^N \int_{A_n} [\int_V  L(\gamma,\theta,\gamma) p(\theta) ] p(\gamma)d\gamma
%\\ &
 \simeq  \sum_{n=1}^N  \left [\int_V  L(\gamma,\theta,\gamma_n) p(\theta) \right] P_n
\end{align}\end{widetext}
where $P_n \equiv \int_{A_n} d\gamma p(\gamma)$ is the prior probability for the volume $A_n$, and $\gamma_n$ is a
representative point in $A_n$.   Note
\begin{eqnarray}
\sum_n P_n =1
\end{eqnarray}
We are \emph{particularlly} interested in uniform distributions (equal-area tilings of the sky), so
\[
P_n = \frac{1}{N} \qquad   \text{Area}(A_n) = \frac{4\pi}{N}
\]

\noindent \emph{Monte Carlo integration with discrete volumes}: As above, but apply to Monte Carlo integration, with a
sampling prior $p_s(\theta)p_s(\gamma)$ in the sky $\gamma$ and all other extrinsic variables $\theta$:
\begin{align}
\LikeRed(\lambda)
& \simeq  \sum_{n=1}^N \int_{A_n} \left [\int_V  L(\gamma,\theta,\gamma_n) p(\theta)p(\gamma) \right]
\end{align}
Proceed as above, but approximate $p(\gamma_n)/p_s(\gamma_n) \simeq P_n/P_{s,n}$, to find
\begin{align}
\LikeRed(\lambda)
& =\sum_n \left[\int_{V}\Like(\lambda,\theta,\gamma_n)\frac{p(\theta)}{p_s(\theta)}p_s(\theta)d\theta \right] \frac{ P_n }{P_{s,n}} P_{s,n}
\end{align}
Now imagine performing a Monte Carlo integral in this form.   Monte Carlo integration corresponds to selecting a random
integer $n$ and a random sample $\theta$ drawn from $p_s(\theta)$.  Indexing the $N$ choices as $n_{\alpha}$ and
$\theta_\alpha$, we find
\begin{align}
\LikeRed(\lambda)
& = \sum_{\alpha}\frac{1}{N}
\Like(\lambda,\theta_{\alpha},\gamma_{n,\alpha})\frac{p(\theta_\alpha)}{p_s(\theta_\alpha)} \frac{ P_{n_\alpha}
}{P_{s,n}}
\end{align}
In other words, when performing a Monte Carlo integral over a discrete subset (e.g., skymap), we use the
\emph{integrated} probabilities $P_n$ as our ``PDF''.

\emph{Concrete discrete random sampling}: For a discrete uniform equal-area tiling, we will use a
\textbf{discrete} probability distribution, not a continuous one.   Specifically, our code
\begin{itemize}
\item \emph{Prior PDF}: Since we always assume uniform on the sky,  provide $P_n=1/N$ at the $n$th
point (i.e., given the sky position of the $n$th tile), \emph{not} $p(\gamma_n)$.

\item \emph{Skymap sampling PDF}: Bayestar provides normalized weighted $s_{s,n}$ already.
%% After fixing a
%%   \textbf{normalization issue} (below), we use
%% \[
%% P_{s,n} = s_n \frac{N'}{N}
%% \]
%% where $N,N'$ are defined below.

\item \emph{Skymap random samples via CDF inverse}: We generate random integers $n$ using random numbers on $[0,1]$, finding the
  closest integer $n$ to the  cumulative of $P_n$; see the text.
\end{itemize}

\noindent \textbf{[Skymaps] Finite-area skymaps and truncation}: We will use skymaps with
$p_s(\gamma)=0$  in a volume $U$ where $p(\gamma)\ne 0$.  With restricted support, we can't integrate
integrate generic functions $L$ with $L>0$ with $p_s=0$.  To be concrete, if we split up the integral over the sky ($\Omega$) into $U$ and
$\Omega-U$, we are ignoring the contribution from $\Omega-U$:
\begin{align}
\LikeRed(\lambda) &= \int_U d\gamma p_s(\gamma)
 [\int d\theta p_s(\theta)
  \frac{p(\theta)p(\gamma)}{p_s(\theta)p_s(\gamma)}\Like]
  \nonumber \\ &
 + \int_{\Omega-U}d\gamma p(\gamma) p(\theta)\Like
\nonumber \\ &
\simeq \int_U d\gamma p_s(\gamma) [\int d\theta p_s(\theta)
  \frac{p(\theta)p(\gamma)}{p_s(\theta)p_s(\gamma)}\Like]
\end{align}
The second step introduces systematic error, assumed small.

The second integral is a conventional Monte Carlo integral over a region $U$ using normalized sampling distributions $p_s$ and
can be approximated in the usual way.

Note the ratio of $p/p_s$ \emph{automatically} includes any ``normalization factor'' associated with only drawing
samples from $U$ rather than from the whole volume $V$

%% , rescaling the total number of samples $N_s$ drawn from $p_s$ by the
%% ratio of volumes of the prior and of $p_s$ over that region $U$
%% \begin{eqnarray}
%% N_{\rm corr} = N / \int_U p_s(\gamma) d\gamma
%% \end{eqnarray}
%% We will apply this correction to the returned values of $p_s$, returning ``rescaled'' probabilities
%% % mcsampler.py  : self._renorm  in pseudo_pdf  and self._expand_valid_points.
%% \begin{eqnarray}
%% p_s \rightarrow \frac{p_s}{\int_U p(\gamma)  d\gamma}
%% \end{eqnarray}
%% Specifically,  if $s_n$ are the  skymap weights reported by \BS, normalized so $\sum_n s_n=1$, we use an ``effective''
%% sampling ``probability'' $P_{s,n}$ of
%% \begin{eqnarray}
%% P_{s,n} = s_n \frac{N'}{N}
%% \end{eqnarray}
%% where $N'$ is the number of nonzero samples in $s$ and $N$ is the number of pixels in $s$.

\noindent \textbf{[Skymaps] Subtle implementation issues: cutoffs and lookup tables}: As described in the text, we use a
discretized lookup table.  As not described in the text, this table de facto \emph{discards} points with weights the
quantized probability.   For real skymaps, the choice of probability quantum $P_{\rm min}$ can be what determines
precisely what points are sampled and hence what part of $\int L p d\theta d\gamma$ are are not sampling \emph{at all}.

As a concrete example, with $n=12*64^2$ sky points, a uniform weighting gives each pixel $P_{n}
=1/n \simeq 2\times 10^{-5}$.  A qunatum of probability $P_{\rm min} $ should \emph{definitely} be smaller than this, to
allow the option of recovering a uniform skymap.
%
More broadly, a skymap with a peak and a wide tail can have that tail truncated if $P_{\rm min}$ is too small.  For
example, if $5\%$ of the skymap probability is distributed across a wide area $\Omega$ of the skymap, we need $P_{\rm
  min} < 0.05 (4\pi/\Omega)/n$, to be sure that area is sampled.
%
For safety, ROS recommends adopting a small probability quantum $\frac{10}{N}<P_{\rm min} < 0.01/n$, where $n$ is the
number of sky pixels and $N$ is the number of iterations in ILE.\footnote{This choice ensures that when ILE is run, at
  least a few draws occur even from the lowest probability pixels.  Does this matter?  Principally for tests, or if
we expect the \BS{} skymap might be wrong.}  \textbf{This choice requires self-consistent use of
  multiple code parameters, some currently hardcoded. }

Once chosen, the quantum of probability defines what pixes de facto have zero probability and are never sampled.
%% \noindent \emph{Convergence issues and sampling bounds on  small probability quanta}: Small probability quanta introduce their own
%% potential problems.  The smaller the probability quantum, the more likely you are to have a pixel with $P_{s,n} \simeq
%% P_{\rm min}$.  On average, that pixel will be sampled every  $1/P_{\rm min}$ sky iterations -- a number that might be in
%% excess of all samples drawn in one ILE run!
Ideally, the likelihood will be zero in  low-probability pixels, no matter what $\lambda,\theta$ are, circumventing
any issues with undersampling and convergence rate.

\subsection{Monte Carlo integration}
\textbf{Non-uniform, variance-weighted weighting}: By default we combine results with different sampling priors treating
them equally (e.g., as if they have converged to the same sampling prior) when evaluating the integral at each mass
point.

That doesn't minimize variance and can be particularly
ineffective when one ILE run has a problem.  Really, we should use variance-reducing weighting: if $x=ax_1 +b x_2$ with $a+b=1$ and
$\left<x_1\right> =\left<x_2\right> =\left<x\right>$, we can minimize the variance of $x$ by choosing weight $a$ to
minimize $a^2 \sigma_1^2 + (1-a)^2 \sigma_2^2$, so
\begin{eqnarray}
x = \frac{\sigma_2^2}{\sigma_1^2+\sigma_2^2} x_1 +  \frac{\sigma_1^2}{\sigma_1^2+\sigma_2^2} x_2
\end{eqnarray}
A straightforward generalization exists with arbitrary numbers of weights:
\[
x = \frac{x_k/\sigma_k^2}{\sum_q 1/\sigma_q^2}
\]
We could and should implement this scheme when recombining results at each mass point.
%% a^2 s1^2 + b^2 s2^2 + (1 - a - b)^2 s3^3
%% Solve[{D[%, a] == 0, D[% == 0, b]}, {a, b}]

\begin{widetext}
\noindent \textbf{Error estimates for $P(<x)$: Single-job}:  ILE reports on 1d cumulative distributions $P(<x)$ from
each job.  Both the numerator and denominator are Monte Carlo integrals using $N$ samples, allowing us to generate an
error estimate.  Keeping in mind the samples are independent and identically distributed, we can evaluate the standard
deviation of the numerator and denominator in the large-$N$ limit
\begin{align}
\hat{P} &= \frac{\sum_k w_k \theta(x-x_k)}{\sum_k w_k} \equiv \frac{ I(<x)}{I} \\
\left< \hat{P} \right> &\simeq \frac{1}{I} \sum_k \left<w_k \theta(x-x_k) \right>  \\
\left<I(<x)^2\right>& \equiv \left<[\sum_k w_k \theta(x-x_k)]^2\right>
  = \sum_k[ \left<w_k^2 \theta(x-x_k)\right>  -\left<w_k\theta(x-x_k)\right>^2
  + \left<\sum_{k} w_k \theta(x-x_k)\right>^2]  \\
\text{estimate: }\sigma_{I(x)}^2 &= \sum_k w_k^2 \theta(x-x_k)  - I^2 P(x)^2 \\
\text{estimate: } \sigma_{I} &= \sum_k \left<w_k^2\right> - I^2 \\
\sigma_P^2 & \simeq \frac{\sum_k w_k^2 \theta(x-x_k)}{I^2} - P(x)^2 + (\text{value at } x_{\rm max})
\end{align}
\end{widetext}
where the last expression adds errors in quadrature from the numerator and denominator, assuming \emph{uncorrelated}
errors.    (In fact, the two are correlated \editremark{fixme: important for scaling the error as $P(1-P)$ })

\subsection{Marginalizing over parameter}

\noindent \textbf{Skymap marginalization: A proposal}: A discrete skymap usually has relatively concentrated support -- roughly
$10^3$ pixels or less, say.  Rather than perform Monte Carlo integration to approximate $\int P_k
\Like(\lambda,\theta_k)$ over the sky, we can \emph{directly} integrate, via a discrete operation:

\emph{Is this a good idea?}: This procedure requires evaluating the likelihood on  \emph{every pixel in the skymap}, so
is of considerable advantage only for very
narrowly targeted (3-IFO, high-SNR) sky localizations.  In such a narrowly-focused case, an adaptive integrator will almost certainly
discover the best-fitting single sky position.  That said, if we're worried about \emph{strongly-separated peaks}, this
procedure will provide an (expensive) backup.

\section{[Internal use] Results: Targeted studies}

\begin{itemize}
\item Single event, 100 noise realizations, plus comparison with Fisher matrix

\item Zero-spin 2015 MDC: [\textbf{Not started}]  For completeness, we propose to reproduce the 2015 MDC,  using
  injections with exactly zero spin.

\end{itemize}

\subsection{Detailed investigation of one event}
Sample run results: Single event, with full DAG, done multiple times (for complete consistency/reproducibility)

Sample run results: extrinsic parameters (for one and several noise realizations)

Sample run results: $L_{\rm red}(\mc,eta)$ (for one and several noise realizations). Expected accuracy at each mass
point.. Translating into  $p(m_1,m_2)$ using a uniform mass prior.

Comparison with Fisher and MCMC

\begin{figure}
\caption{\label{fig:TargetedEvent:LikelihoodVersusMchirpEta}\textbf{Targeted study: Posterior distribution
    versus component masses, for several noise realizations}: : The 90\% confidence interval derived from $L_{\rm red}$.  For
comparison, the prediction from a Fisher matrix is shown as a solid black curve.
 \textbf{PLACEHOLDER/INTENT}
}
\end{figure}

\section{[Internal use] Results: Additional production-environment investigations}

\begin{itemize}
\item 2016 BNS MDC: [\textbf{Not started}]

\end{itemize}

\section{[Internal use] Incomplete or not-yet-implemented investigations}

\subsection{Measures of convergence}

Measures to assess whether we're done
\begin{itemize}
\item * Integral: Classic MC error estimate (central limit); reproducibility across runs and via subsamples (e.g., chisquared)

\item * 1d posterior:  Monte carlo error (see elsewhere); $n_{\rm eff}$; $L^2$ or KL divergence between subsamples or
  across runs, compared to statistics.

\item * sampling distribution: similarly
\end{itemize}

\subsection{Semianalytic marginalization over distance}

\section{[Internal use] Future directions}

Not for distribution!

\subsection{Short GRBs}

Alex

\subsection{Adding dimensions}

Tides [Les]

Aligned spin

\subsection{Even more speed}

Because the precompute phase is \emph{much} faster than marginalizing over extrinsic parameters, we could accelerate
code performance in a number of ways, potentially enabling us to tackle higher dimensions

\noindent \textbf{Einstein at home}: Transmit the precomputed data (10ms of timeseries and a few scalars) to the
Einstein at home grid, giving each distant host (say) 20 mass points to handle.  With $10^5$ cores, that means roughly
$20\times10^5/10$ intrinsic points per day...enough to handle precessing spin faster than any other extant method.

\noindent \textbf{Interpolate $Q,U$}: Perform the precompute step several times, building up a grid for $Q$ and $U,V$,

\subsection{Tigher search integration}

\noindent \textbf{Search filters as input}
The searches provide filter outputs.  We ought to be able to use those as inputs to construct $Q_{k,lm}$ for any
$\lambda,l,m$, via a suitable lookup table.   This could completely eliminate the startup/precompute cost.

\noindent \textbf{As detection statistic?}: To what extent is this approach useful compared to coherent PTF?  We can
also do timeslides at a fixed sky location.

\section{[Internal use] Results: Targeted studies}

\begin{itemize}
\item Single event, 100 noise realizations, plus comparison with Fisher matrix

\item Zero-spin 2015 MDC: [\textbf{Not started}]  For completeness, we propose to reproduce the 2015 MDC,  using
  injections with exactly zero spin.

\end{itemize}

\section{[Internal use] Results: Targeted studies}

\begin{itemize}
\item Single event, 100 noise realizations, plus comparison with Fisher matrix

\item Zero-spin 2015 MDC: [\textbf{Not started}]  For completeness, we propose to reproduce the 2015 MDC,  using
  injections with exactly zero spin.

\end{itemize}

\subsection{Detailed investigation of one event}
Sample run results: Single event, with full DAG, done multiple times (for complete consistency/reproducibility)

Sample run results: extrinsic parameters (for one and several noise realizations)

Sample run results: $L_{\rm red}(\mc,eta)$ (for one and several noise realizations). Expected accuracy at each mass
point.. Translating into  $p(m_1,m_2)$ using a uniform mass prior.

Comparison with Fisher and MCMC

\section{[Internal use] Results: Additional production-environment investigations}

\begin{itemize}
\item 2016 BNS MDC: [\textbf{Not started}]

\end{itemize}

\section{[Internal use] Incomplete or not-yet-implemented investigations}

\subsection{Measures of convergence}

Measures to assess whether we're done
\begin{itemize}
\item * Integral: Classic MC error estimate (central limit); reproducibility across runs and via subsamples (e.g., chisquared)

\item * 1d posterior:  $n_{\rm eff}$; $L^2$ or KL divergence between subsamples or across runs

\item * sampling distribution: similarly
\end{itemize}

\subsection{Semianalytic marginalization over distance}

\section{[Internal use] Future directions}

Not for distribution!

\subsection{Short GRBs}

\section{[Internal use] Incomplete or not-yet-implemented investigations}

\subsection{Measures of convergence}

Measures to assess whether we're done
\begin{itemize}
\item * Integral: Classic MC error estimate (central limit); reproducibility across runs and via subsamples (e.g., chisquared)
\end{itemize}

\section{[Internal use] Future directions}

Not for distribution!

\subsection{Short GRBs}

Alex

\subsection{Adding dimensions}

Tides [Les]

Aligned spin

\subsection{Even more speed}

Because the precompute phase is \emph{much} faster than marginalizing over extrinsic parameters, we could accelerate

\section{[Internal use] Patches pending or desired}

Persons/proposing supporting  a change listed as [name].

\noindent \textbf{Next push}

* Bugfix on ILE MC variance

* Uniform on sky prior with bayestar skymaps

\noindent \textbf{Other important updates}

* [ROS] Add monte carlo error to $P(<x)$ estimate plots (optional) and description in text

*  [ROS]  $b_k \ne 0$ for any sky position (else complications arise re normalization).

* [ROS] Interpolation of $L$ vs masses using spokes (higher order) -- current plots are mathematica

\noindent \textbf{Minor code tweaks}

* [ROS] Larger nchunk

* [ROS] Change tempering exponent definition from $(Lp/p_s)^\beta$ to $L^\beta p/p_s$, to facilitate analysis

* [ROS] Variance weighting when recombining samples from different runs at the same mass point

\noindent \textbf{Infrastructure}

* [ROS] Change DAG (number of jobs, max iterations): current solution with skymap is overkill

* [ROS] For automated end-to-end tests, add \gstlal{} coinc generation and \BS{} skymaps to \texttt{stage\_injections}

** Approximate solution: use bayestar tool to generate ``fake'' coincs and skymaps from sims:
\href{https://www.lsc-group.phys.uwm.edu/ligovirgo/cbcnote/ParameterEstimationModelSelection/BAYESTARHowTo}{Bayestar docs}

* [ROS] Add posterior in mchirp, eta accounting for prior.

* [Patrick] Logging

* [Richard/Chris] Report hashtag in log and process\_params

* [ROS] Warnings about implicit cutoffs re skymaps (\texttt{min\_p} plus the \texttt{cdf} target, both of which define
regions to be ignored).

* [ROS] Modularized version that lets us change mass quickly (e.g., to loop over mass points; to test mass points
against known-good extrinsic parameters; etc)
%load all wights stored
%loop and eval likelihood with stored samples only
%useful for exploring mass space using fixed extrinsic grid set by previous
%only need a few samples to get an idea

\noindent \textbf{Exploration}

* [ROS] Region library (i.e. ellipsoids): test if in; perform integral (mcsampler); return points (grid/unstructured)

** mcsampler: return dump of params, so we can construct marginalized posteriors/PDFs using usual infrastructure

* [ROS] Refinement stage: a single job after a mass point, using the best-suited

\section{Methods  paper 2: Spin}

Adding the spin dimensions is important and worth a detailed study.

Very early; keep thoughts organized in outline, below

\begin{itemize}

  \item Introduction
  \item Revised methods
    \begin{itemize}
        \item  $h_{lm}$ in $J$ frame
       \item  Single spin: approximate harmonic method (treat each sideband $e^{im\alpha+im\gamma}h_{22}^{ROT}$ as a
         seperate filter, and $\beta$ as an extrinsic parameter.  De facto means adding only one additional dimension
         ($\kappa$) over aligned spin;
         all else extrinsic)
        \item Spin dynamics and dimensional reduction/reorganization.  ``PN resonance coordinates.''
        \item Role of prior for spin (6d vs 3d vs 2d prior for 2spin, 1spin, aligned spin) has effect
        \item New approximately extrinsic parameters (e.g., $\alpha$ as approximately extrinsic)
    \end{itemize}

 \item Discussion and sample results: Aligned spin

   * single event in detail: 3d posterior; evidence for spin vs zero spin

 \item Discussion and results: Single precessing spin

   * ``fixed $\beta$

   * single event in detail

 \item Discussion and results: Double spin

  * challenge of higher dimensions
\end{itemize}


\documentclass{article}
\usepackage{graphicx}
\usepackage[utf8]{inputenc}
\usepackage{amsmath, amssymb, latexsym}

\usepackage{pgfplots}
\usepackage{algorithm}
\usepackage[noend]{algpseudocode}
\usepackage{tikz}
\usepackage{nicefrac}
\usepackage{placeins}
\pgfplotsset{every axis legend/.append style={
at={(0,0)},
anchor=north east}}
\usetikzlibrary{shapes,positioning,intersections,quotes}
\usetikzlibrary{arrows.meta,
                bending,
                intersections,
                quotes,
                shapes.geometric}

\definecolor{darkgreen}{rgb}{0.0, 0.6, 0.0}
\definecolor{darkred}{rgb}{0.7, 0.0, 0.0}
\makeatletter
\def\BState{\State\hskip-\ALG@thistlm}
\makeatother
\title{Week 15}
\begin{document}
\pagenumbering{gobble}
\maketitle
\newpage
\pagenumbering{arabic}

\section*{Anomaly detection}

\begin{itemize}
  \item We can assess whether data points are anomalous by using the dataset as a baseline.
  \item if $p(x_{test}) < \epsilon \quad$, then flag this as an anomaly
  \item if $p(x_{test}) \geq \epsilon \quad$, then this is OK
  \item $\epsilon$ is a threshold probability number that we determine based on how certain we need/want to be.
\end{itemize}

\includegraphics[width=0.5\textwidth]{resources/anomaly}

\section*{Applications}
\begin{itemize}
  \item Fraud detection

        \begin{itemize}
          \item Users have activities connected with them, such as the amount of time spent online, the location of login, and the frequency with which they spend money.
          \item Using this information, we can create a model of what regular users do.
          \item What is the probability of "normal" behavior?
          \item Send atypical users' data through the model to identify them. Make a note of everything that appears unusual. Block cards/transactions automatically.
        \end{itemize}

  \item Manufacturing

        \begin{itemize}
          \item Aircraft engine example.

        \end{itemize}

  \item Monitoring computers in data center

        \begin{itemize}
          \item If you have many machines in a cluster (x1 = memory use, x2 = number of disk accesses/sec, x3 = CPU load).
          \item When you notice an anomalous machine, it is likely that it is soon to fail.
          \item Consider replacing parts of it.
        \end{itemize}
\end{itemize}

\section*{The Gaussian distribution}

\begin{itemize}
  \item $\mu$ is mean.
  \item $\sigma^2$ is variance and $\sigma$ is a standard deviation.
  \item probability of x, parameterized by the mean and variance:
\end{itemize}

$$p(x; \mu; \sigma^2) = \frac{1}{\sqrt{2\pi\sigma}}exp(-\frac{(x-\mu)^2}{2\sigma^2})$$

~\\

\includegraphics[width=0.5\textwidth]{resources/gaussian}

\begin{itemize}
  \item Assume we have a data collection of m examples.
  \item Given that each example is a real number, we plot the data on the x axis.
  \item Given the dataset can you estimate the distribution?
\end{itemize}

\includegraphics[width=0.6\textwidth]{resources/data_fit}

Seems like a good fit - data suggests a higher likelihood of being in the center and a lower likelihood of being further out.

\newpage
\section*{Anomaly detection}

\FloatBarrier
\begin{algorithm}
\caption{Anomaly detection}\label{euclid}
\begin{algorithmic}[1]
\State Choose features $x_i$ that you think might be indicative of anomalous examples.
\State Fit parameters $\mu_1, ..., \mu_n, \sigma_1^2, ..., \sigma^n$

$$\mu_j = \frac{1}{m} \sum_{i=1}^mx_j^{(i)}$$
$$\sigma_j^2 = \frac{1}{m} \sum_{i=1}^m(x_j^{(i)}-\mu_j)^2$$

\State Given new example x, compute p(x):

$$p(x)= \prod_{j=1}^n \frac{1}{\sqrt{2\pi\sigma_j}}exp(-\frac{(x_j-\mu_j)^2}{2\sigma_j^2})$$

\end{algorithmic}
\end{algorithm}
\FloatBarrier

\section*{Developing and evaluating and anomaly detection system}

\begin{itemize}
  \item You have some labeled data.

        \begin{itemize}
          \item $y=0$ for engines which were non-anomalous.
          \item $y=1$ for engines which were anomalous.
        \end{itemize}

  \item Training set is the collection of normal examples.
  \item Next define:

        \begin{itemize}
          \item Cross validation set.
          \item Test set.
          \item For both assume you can include a few examples which have anomalous examples.
        \end{itemize}

  \item In our example we have:

        \begin{itemize}
          \item 10000 good engines.
          \item 50 flawed engines.
        \end{itemize}

  \item Split into:

        \begin{itemize}
          \item Training set: 6000 good engines (y = 0).
          \item CV set: 2000 good engines, 10 anomalous.
          \item Test set: 2000 good engines, 10 anomalous.
        \end{itemize}

\end{itemize}

~\\
What's a good metric to use for evaluation?

\begin{itemize}
  \item Compute fraction of true positives/false positive/false negative/true negative.
  \item Compute precision/recall.
  \item Compute F1-score.
\end{itemize}

\section*{Multivariate Gaussian distribution}
It is a somewhat different approach that can occasionally discover anomalies that normal Gaussian distribution anomaly detection fails to detect.

\begin{itemize}
  \item Assume you can fit a Gaussian distribution to CPU load and memory use.
  \item Assume we have an example in the test set that appears to be an anomaly (e.g. x1 = 0.4, x2 = 1.5).
  \item Here memory use is high and CPU load is low (if we plot x1 vs. x2 our green example looks miles away from the others).
  \item The problem is that if we look at each characteristic individually, they may fall inside acceptable bounds - the difficulty is that we know we shouldn't obtain those types of numbers together, but they're both okay individually.
\end{itemize}

\includegraphics[width=\textwidth]{resources/mult_gauss}

~\\
What are the parameters for this new model?
\begin{itemize}
  \item $\mu$ which is an n dimensional vector (where n is number of features)
  \item $\Sigma$ which is an [n x n] matrix - the covariance matrix
\end{itemize}

$$p(x; \mu; \Sigma) = \frac{1}{(2\pi)^{n/2}|\Sigma|^{1/2}}exp(-\frac{1}{2}(x-\mu)^T \Sigma^{-1}(x-\mu))$$

~\\\\
\includegraphics[width=\textwidth]{resources/cov_matrix_sigma}

~\\
Very tall thin distribution, shows a strong positive correlation.

\subsection*{Gaussian model - summary}

\begin{itemize}
  \item Probably used more often.
  \item There is a need to manually create features to capture anomalies where x1 and x2 take unusual combinations of values.
  \item So need to make extra features and might not be obvious what they should be.
  \item Much cheaper computationally.
  \item Scales much better to very large feature vectors.
  \item Works well even with a small training set e.g. 50, 100.
\end{itemize}

\subsection*{Multivariate gaussian model - summary}

\begin{itemize}
  \item Used less frequently.
  \item Can capture feature correlation.
  \item So no need to create extra values.
  \item Less computationally efficient.
  \item Needs for m > n  i.e. number of examples must be greater than number of features.
\end{itemize}

\end{document}


\chapter{fMRI model estimation}

Model parameters can be estimated using classical (ReML - Restricted Maximum Likelihood) or Bayesian algorithms. After parameter estimation, the RESULTS button can be used to specify contrasts that will produce Statistical Parametric Maps (SPMs), Effect Size Maps (ESMs) or Posterior Probability Maps (PPMs) and tables of statistics.

\section{Select SPM.mat}

Select the SPM.mat file that contains the design specification. SPM will output the results of its analysis into this directory. This includes overwriting the SPM.mat file. When the estimation job is run, no warning will be given that the SPM.mat file will be overwritten. A warning is given at the specification stage. When it comes to estimation, SPM assumes that you've now sorted out your directory structures.

\begin{figure}
\begin{center}
\includegraphics[width=100mm]{fmri_est/est_method}
\end{center}
\caption{\em After starting SPM in fMRI mode, pressing the `Estimate' button, and then double-clicking on the `+fMRI model estimation' text, the SPM graphics window should appear as above. The options under `-fMRI model estimation' can be examined by clicking on them. A single click will bring up some help text in the lower subwindow (not shown in the above graphic). A double-click on options prefixed by a '+' will allow you to specify options at a greater level of detail. Options highlighted with a `$<$-X' are mandatory and must be filled in by the user. Each of the options shown above is described in this chapter. \label{est}}
\end{figure}

\section{Method}

There are three possible estimation procedures for fMRI models (1) classical (ReML) estimation of first or second level models, (2) Bayesian estimation of first level models and (3) Bayesian estimation of second level models. Option (2) uses a Variational Bayes (VB) algorithm that is new to SPM5. Option (3) uses the Empirical Bayes algorithm with global shrinkage priors that was also in SPM2.

To use option (3) you must have already estimated the model using option (1). That is, for second-level models you must run a ReML estimation before running a Bayesian estimation. This is not necessary for option (2). Bayesian estimation of 1st-level models using VB does not require a prior ReML estimation.

\subsection{Classical}

Model parameters are estimated using Restricted Maximum Likelihood (ReML). This assumes the error correlation structure is the same at each voxel. This correlation can be specified using either an AR(1) or an Independent and Identically Distributed (IID) error model. These options are chosen at the model specification stage. ReML estimation should be applied to spatially smoothed functional images. See \cite{peb1,peb2} for further details of the ReML estimation scheme. After estimation, specific profiles of parameters are tested using a linear compound or contrast with the T or F statistic. The resulting statistical map constitutes an SPM. The SPM{T}/{F} is then characterised in terms of focal or regional differences by assuming that (under the null hypothesis) the components of the SPM (ie. residual fields) behave as smooth stationary Gaussian fields.

The rest of this chapter describes the Bayesian estimation options. So, please skip to the next chapter if you are interested only in classical estimation and inference.

\subsection{Bayesian 1st-level}

Model parameters are estimated using Variational Bayes (VB). This allows you to specify spatial priors for regression coefficients and regularised voxel-wise AR(P) models for fMRI noise processes. The algorithm does not require functional images to be spatially smoothed. Estimation will take about 5 times longer than with the classical approach. This is why VB is not the default estimation option. The VB approach has been described in a number of papers \cite{vb_fmri_ar,vb2,vb3,will_bayes_srglm}.

After estimation, contrasts are used to find regions with effects larger than a user-specified size eg. 1 per cent of the global mean signal. These effects are assessed statistically using a Posterior Probability Map (PPM) \cite{karl_posterior}.

\begin{figure}
\begin{center}
\includegraphics[width=100mm]{fmri_est/bayes_options}
\end{center}
\caption{\em After choosing Bayesian 1st-level under `Method' and then double-clicking on the `+Bayesian 1st-level' text, the SPM graphics window should appear as above. Each of the options shown above is described in this chapter. \label{bayes_options}}
\end{figure}

\subsubsection{Analysis Space}

Because estimation can be time consuming, an option is provided to analyse selected slices rather than the whole volume.

\paragraph{Volume}

You have selected the Volume option. SPM will analyse fMRI time series in all slices of each volume.

\paragraph{Slices}

Enter Slice Numbers. This can be a single slice or multiple slices. If you select a single slice or only a few slices you must be aware of the interpolation options when, after estimation, displaying the estimated images eg. images of contrasts or AR maps. The default interpolation option may need to be changed to nearest neighbour (NN) (see bottom right hand of graphics window) for your slice maps to be visible.

\subsubsection{Signal priors}

\begin{itemize}

\item{[GMRF] Gaussian Markov Random Field. This spatial prior is the recommended option. Regression coefficients at a given voxel are (softly) constrained to be similar to those at nearby voxels. The strength of this constraint is determined by a spatial precision parameter that is estimated from the data. Different regression coefficients have different spatial precisions allowing each putative experimental effect to have its own spatial regularity. }

\item{[LORETA] Low Resolution Tomography Prior. This spatial prior is very similar to the GMRF prior and is a standard choice for MEG/EEG source localisation algorithms. It does, however, have undesirable edge effects.}

\item{[Global] Global Shrinkage prior. This is not a spatial prior in the sense that regression coefficients are constrained to be similar to neighboring voxels. Instead, the average effect over all voxels (global effect) is assumed to be zero and all regression coefficients are shrunk towards this value in proportion to the prior precision. This is the same prior that is used for Bayesian estimation at the second level (see also \cite{karl_posterior}), except that here the prior precision is estimated separately for each slice. }

\item{[Uninformative] A flat prior. Essentially, no prior information is used. If you select this option then VB reduces to Maximum Likelihood (ML) estimation. This option is useful if, for example, you do not wish to use a spatial prior but wish to take advantage of the voxel-wise AR(P) modelling of noise processes. In this case, you would apply the algorithm to images that have been spatially smoothed. For P=0, ML estimation in turn reduces to Ordinary Least Squares (OLS) estimates, and for P$>$0, ML estimation is equivalent to a weighted least squares (WLS) algorithm but where the weights are different at each voxel. This reflects the different noise correlations at each voxel. }

\end{itemize}

\subsubsection{AR model order}

An AR model order of 3 is the default. Cardiac and respiratory artifacts are periodic in nature and therefore require an AR order of at least 2. In previous work, voxel-wise selection of the optimal model order showed that a value of 3 was the highest order required \cite{vb_fmri_ar}.

Higher model orders have little effect on the estimation time. If you select a model order of zero this corresponds to the assumption that the errors are Independent and Identically Distributed (IID). This AR specification overrides any choices that were made in the model specification stage.

Voxel-wise AR models are fitted separately for each session of data. For each session this therefore produces maps of AR(1), AR(2) etc coefficients in the output directory.

\subsubsection{Noise priors}

There are three noise prior options.

\begin{itemize}

\item{[GMRF] Gaussian Markov Random Field. This is the default option. This spatial prior is the same as that used for the regression coefficients. Spatial precisions are estimated separately for each AR coefficient eg. the AR(1) coefficient over space, AR(2) over space etc.}

\item{[LORETA] Low Resolution Tomography Prior. See comments on LORETA priors for regression coefficients.}

\item{[Tissue-type] This provides an estimation of AR coefficients at each voxel that are biased towards typical values for that tissue type (eg. gray, white, CSF). If you select this option you will need to then select files that contain tissue type maps (see below). These are typically chosen to be Grey Matter, White Matter and CSF images derived from segmentation of registered structural scans.}

\end{itemize}

Previous work has shown that there is significant variation in AR values with tissue type. However, GMRF priors have previously been favoured by Bayesian model comparison \cite{will_bayes_srglm}.

\subsubsection{ANOVA}

Perform 1st or 2nd level Analysis of Variance.

\paragraph{First level}

This is implemented using Bayesian model comparison as described in \cite{will_bayes_srglm}. For example, to test for the main effect of a factor two models are compared, one where the levels are represented using different regressors and one using the same regressor. This therefore requires explicit fitting of several models at each voxel and is computationally demanding (requiring several hours of computation). The recommended option is therefore NO.

To use this option you must have already specified your factorial design during the model specification stage.

\paragraph{Second level}

This option tells SPM to automatically generate the simple contrasts that are necessary to produce the contrast images for a second-level (between-subject) ANOVA. Naturally, these contrasts can also be used to characterise simple effects for each subject.

With the Bayesian estimation option it is recommended that contrasts are computed during the parameter estimation stage (see 'simple contrasts' below). The recommended option here is therefore YES.

To use this option you must have already specified your factorial design during the model specification stage.

If you wish to use these contrast images for a second-level analysis then you will need to spatially smooth them to take into account between-subject differences in functional anatomy ie. the fact that one persons V5 may be in a different position than anothers.

\subsubsection{Simple contrasts}

`Simple' contrasts refers to a contrast that spans one-dimension ie. to assess an effect that is increasing or decreasing.

If you have a factorial design then the contrasts needed to generate the contrast images for a 2nd-level ANOVA (or to assess these simple effects within-subject) can be specified automatically using the ANOVA-$>$Second level option.

When using the Bayesian estimation option it is computationally more efficient to compute the contrasts when the parameters are estimated. This is because estimated parameter vectors have potentially different posterior covariance matrices at different voxels and these matrices are not stored. If you compute contrasts post-hoc these matrices must be recomputed. This uses an approximate reconstruction based on a Taylor series expansion described in \cite{vb3}. It is therefore recommended to specify as many contrasts as possible prior to parameter estimation.

If you wish to use these contrast images for a second-level analysis then you will need to spatially smooth them to take into account between-subject differences in functional anatomy ie. the fact that one persons V5 may be in a different position than anothers.

\paragraph{Simple contrast}

\subparagraph{Name}

Name of contrast eg. `Positive Effect'

\subparagraph{Contrast vector}

These contrasts are used to generate PPMs which characterise effect sizes at each voxel. This is different to SPMs in which eg. maps of t-statistics show the ratio of the effect size to effect variability (standard deviation). SPMs are therefore a-dimensional. This is not the case for PPMs as the size of the effect is of primary interest. Some care is therefore needed about the scaling of contrast vectors. For example, if you are interested in the differential effect size averaged over conditions then the contrast $[0.5, 0.5, -0.5, -0.5]$ would be more suitable than the $[1, 1, -1, -1]$ contrast which looks at the differential effect size summed over conditions.

\subsection{Bayesian 2nd-level}

Bayesian estimation of 2nd level models. This option uses the Empirical Bayes algorithm with global shrinkage priors that was previously implemented in SPM2. It is described in detail in \cite{karl_posterior}.

Use of the global shrinkage prior embodies a prior belief that, on average over all voxels, there is no net experimental effect. Some voxels will respond negatively and some positively with a variability determined by the prior precision. This prior precision can be estimated from the data using Empirical Bayes.

\section{Output files}

After estimation a number of files are written to the output directory. These are

\begin{itemize}
\item{An \verb!SPM.mat! file containing specification of the design and estimated model parameters}
\end{itemize}

\subsection{Classical 1st-level}

For classical 1st-level models the following files are also produced

\begin{itemize}

\item{Images of estimated regression coefficients  \verb!beta_000k.img! where $k$ indexes the $k$th regression coefficient.}

\item{An image of the variance of the error \verb!ResMS.img!.}

\item{An image \verb!mask.img! indicating which voxels were included in the analysis.}

\item{The image \verb!RPV.img!, the estimated resels per voxel.}

\item{If contrasts have been specified SPM also writes \verb!con_000i.img!  if the $i$th contrast is a t-contrast and the extra sum of squares image \verb!ess_000i.img! if it is an F-contrast.}

\end{itemize}

Type \verb!help spm_spm! at the matlab command prompt for further information.

\subsection{Bayesian 1st-level}

For Bayesian 1st-level models the following files are also produced

\begin{itemize}

\item{Images of estimated regression coefficients  \verb!Cbeta_000k.img! where $k$ indexes the $k$th regression coefficient. These filenames are prefixed with a `C' indicating that these are the mean values of the `Conditional' or `Posterior' density.}

\item{Images of error bars/standard deviations on the regression coefficients \verb!SDbeta_000k.img!.}

\item{An image of the standard deviation of the error \verb!Sess1_SDerror.img!.}

\item{An image \verb!mask.img! indicating which voxels were included in the analysis.}

\item{If a non-zero AR model order is specified then SPM also writes images \verb!Sess1_AR_000p.img! where $p$ indexes the $p$th AR coefficient.}

\item{If contrasts have been specified SPM also writes \verb!con_000i.img! and \verb!con_sd_000i.img! which are the mean and standard deviation of the $i$th pre-defined contrast.}

\end{itemize}

Each of these images can be inspected using the `Display' button. Type \verb!help spm_spm_vb! at the matlab command prompt for further information.

\section{Model comparison}

Once you have estimated a model you can use SPM's results button to look at the results. You can also extract fMRI data from regions of interest using the ROI button. You can then compare GLMs based on different hemodynamic basis sets using the Bayesian model evidence.

This is described in \cite{will_bayes_srglm} and implemented using the command line option `spm\_vb\_roi\_basis'. This requires a VOI filename (created using the ROI button) and an SPM data structure. Type `help spm\_vb\_roi\_basis' at the matlab command prompt for further information. Figure~\ref{basis} shows an example output from the function indicating that, for the data in this brain region, an informed basis set has the highest model evidence.

\begin{figure}
\begin{center}
\includegraphics[width=150mm]{fmri_est/basis}
\end{center}
\caption{\em This plot shows the model evidence for a number of different hemodynamic basis sets: Inf1 - Canonical HRF, Inf2 - Canonical plus temporal derivative, Inf3 - Canonical plus temporal and dispersion derivatives, F - Fourier, FH - Fourier with a Hanning Window, Gamm3 - 3 Gamma basis functions and FIR - a Finite Impulse Response function. An informed basis set provides the best model of the data for the selected region.  \label{basis}}
\end{figure}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{pgfplots}
\usepackage{fancyhdr}
\usepackage{enumitem}
\usepackage{tikz}
\usepackage{xparse}
\usepackage{siunitx}

\pgfplotsset{width=10cm,compat=1.9}

\pagestyle{fancy}
\fancyhf{}
\lhead{Steven Glasford}
\chead{Homework 1.4}
\rhead{Page \thepage}

\title{Homework 1.4}
\author{Steven Glasford}
\date{\parbox{\linewidth}{\centering%
    \today\endgraf\medskip
    Math-451-M001}}
\newcommand{\rpm}{\sbox0{$1$}\sbox2{$\scriptstyle\pm$}
  \raise\dimexpr(\ht0-\ht2)/2\relax\box2 }

\newlist{steps}{enumerate}{1}
\setlist[steps, 1]{label = Step \arabic*:}
\ExplSyntaxOn
\newcommand*{\prlen}[1]{%
   % round to 1 digit:
    \pgfmathparse{round(10)/10.0}%
    %\pgfkeys{/pgf/number format/precision=1}
    %\pgfmathresult
    \pgfmathprintnumber[fixed, precision=2]{\pgfmathresult}
}
\ExplSyntaxOff

\begin{document}

\maketitle

I choose to do problems 3 and 9 from the options of 3, 4, 5 and 9.

\section{Problem 3}

\begin{enumerate}[label=\alph*]
    \item
    \begin{tikzpicture}
        \begin{axis}[
            axis lines = left,
            xlabel = Time in $t$ day,
            ylabel = Price per pig $P(t)$,
        ]
        \addplot [
            domain=0:15,
            samples=100,
            color=red,
        ]{.65 - .01*x + .00004*x^2};
        \addlegendentry{New: $.65-.01t+.00004t^2$}

        \addplot[
            domain=0:15,
            samples=100,
            color=blue,
        ]{.65-.01*x};
        \addlegendentry{Old: $.65-.01*x$}

        \end{axis}
    \end{tikzpicture}
    The graph above shows the price per pig using both the old linear model (blue) and the newer model (the red line). The red line is more accurate to the actual price per pig in real life. But as is obvious in the graph the values for the old equation are very similar to the more accurate model within the appropriate time domain.
    \item
        \begin{center}
            \begin{tabular}{ |c|c|c| }
            \hline
            Variables & Constants & Assumptions \\
            \hline
            $t$ - Time (Days) & $w_0 = 200$ initial weight (pounds) & $w=w_0+5t$ \\
            $f$ - profit (in dollars) &  & $p=.65-.01t+.00004t^2$ \\
            $w$ - Weight (pounds) && $c=.45t$ \\
            $p$ - Price per pound && $r=p*w$\\
            $c$ - cost (in dollars) && $f=r-c$\\
            $r$ - Revenue (in dollars) &&\\
            \hline
            \end{tabular}
        \end{center}
        If we want to maximize $f(t)$ then we should try to add all of the assumptions into a single equation, this way all of the assumptions are considered. We will try to use $t$ as the isolating factor. If $$f=r-c$$ \parbox{\linewidth}{\centering%
            $r=pw$ \hspace*{3cm} $c=.45t$ \endgraf \bigskip
            $f=pw-.45t$ \endgraf\bigskip
            $p=.65-.01t$ \hspace*{3cm}
            $w=w_0+5*t$ \endgraf \bigskip
            $f=(.65-.01t+.00004t^2)(w_0+5t)-.45t$ \endgraf\bigskip
            $w_0=200$ \endgraf \bigskip
            $f=(.65-.01t+.00004t^2)(200+5t)-.45t$ \endgraf\bigskip
        }
        Which simplifies to:$$f(t)=.0002t^3-.042t^2+.8t+130$$
        Now we need to take the derivative of $f(t)$ and find where it is equal to zero. $$\frac{df}{dt}=\frac{3t^2}{5000}-\frac{21t}{250}+\frac{4}{5}$$

        \begin{tikzpicture}
        \begin{axis}[
            axis lines = left,
            xlabel = Time in $t$ day,
            ylabel = $y$,
        ]

        \addplot[
            domain=0:20,
            samples=100,
            color=blue,
        ]{(3*x^2-420*x+4000)/5000};
        \addlegendentry{$\frac{df}{dt}$}
        \draw[ultra thin] (axis cs:\pgfkeysvalueof{/pgfplots/xmin},0) -- (axis cs:\pgfkeysvalueof{/pgfplots/xmax},0);
        \end{axis}

    \end{tikzpicture}
    \endgraf
    Using the quadratic equation we try to determine the extreme somewhere between 8 and 14, as that point will be a maximum, this is known since the derivative is a quadratic and that point the derivative goes from positive to negative, indicating a maxima. The other root would in turn be a minima.
    $$\frac{df}{dt}=0=\frac{3t^2-420t+4000}{5000}$$ $$t=\frac{10(21+\sqrt{321})}{3},\frac{10(21-\sqrt{321})}{3}$$
    $$t \approx \pgfmathparse{(10*(21+321^(1/2)))/3}\pgfmathresult , \pgfmathparse{(10*(21-321^(1/2)))/3}\pgfmathresult $$
    \textbf{Therefore, the best day to sell your pig is at about 10 days.}

    \item \underline{Sensitivity analysis}
    \endgraf
    $$p(t)=.65-.01t+.00004t^2$$
    $$f(t)=.0002t^3-.042t^2+.8t+130$$
    $$y=(.65-.01x+xt^2)(200+5x)-.45x$$
    $$y=5rx^3-.05x^2+200rx^2+.8x+130=0$$
    $$\frac{dy}{dx}=15rx^2+400rx-\frac{x}{10}+\frac{4}{5}=0$$
    $$x=\frac{-4000r+1+ 10\sqrt{160000r^2-128r+1/100}}{300r}$$
    $$\frac{dx}{dr}=\frac{-6400r+1+ \sqrt{16000000r^2-12800r+1}}{300r^2\sqrt{16000000r^2-12800r+1}}$$
    \begin{center}
            \begin{tabular}{ |c|c|c| }
            \hline
            r & x & $\frac{\Delta x}{\Delta r} * 100$ \\
            \hline
            .00002 & 297.709385518&
            %\pgfkeys{/pgf/fpu} \pgfmathparse{}\pgfmathresult
            %\edef\tmp{\pgfmathresult}
            %\pgfmathresult
            %\pgfkeys{/pgf/fpu=false} &
            %1/(-6400*.00004+1+sqrt(16000000*.00004^2-128*.00004+1))/(300*.00004^2*sqrt(16000000*.00004^2-12800*.00004+1))
            %(-6400*r+1+\sqrt{16000000*r^2-128*r+1})/(300*r^2*\sqrt{16000000*r^2-12800r+1})

            \\
            .00003 & 185.997481072  & -1117119044.46
            %\pgfmathparse{((-4000*.00003+1+10*(160000*.00003^2-128*.00003+1/100)^(1/2))/(300*.00003)-(-4000*.00002+1+10*(160000*.00002^2-128*.00002+1/100)^(1/2))/(300*.00002))/(.00003-.00002)*100}\pgfmathresult &
            \\
            .00004 &129.721576224&-562759048.485
           % \pgfmathparse{(-4000*.00004+1+10*(160000*.00004^2-128*.00004+1/100)^(1/2))/(300*.00004)}\pgfmathresult &
           % \pgfmathparse{(-4000*.00004+1-10*(160000*.00004^2-128*.00004+1/100)^(1/2))/(300*.00004)}\pgfmathresult &
           % \pgfmathparse{((-4000*.00004+1+10*(160000*.00004^2-128*.00004+1/100)^(1/2))/(300*.00004)-(-4000*.00003+1+10*(160000*.00003^2-128*.00003+1/100)^(1/2))/(300*.00003))/(.00004-.00003)*100}\pgfmathresult &
           % \pgfmathparse{((-4000*.00004+1-10*(160000*.00004^2-128*.00004+1/100)^(1/2))/(300*.00004)-(-4000*.00003+1-10*(160000*.00003^2-128*.00003+1/100)^(1/2))/(300*.00003))/(.00004-.00003)*100}\pgfmathresult
            \\
            .00005 & 95.4970354689 &-342245407.55
            %\pgfmathparse{(-4000*.00005+1+10*(160000*.00005^2-128*.00005+1/100)^(1/2))/(300*.00005)}\pgfmathresult &
           % \pgfmathparse{(-4000*.00005+1-10*(160000*.00005^2-128*.00005+1/100)^(1/2))/(300*.00005)}\pgfmathresult &
           % \pgfmathparse{((-4000*.00005+1+10*(160000*.00005^2-128*.00005+1/100)^(1/2))/(300*.00005)-(-4000*.00004+1+10*(160000*.00004^2-128*.00004+1/100)^(1/2))/(300*.00004))/(.00005-.00004)*100}\pgfmathresult&
           % \pgfmathparse{((-4000*.00005+1-10*(160000*.00005^2-128*.00005+1/100)^(1/2))/(300*.00005)-(-4000*.00004+1-10*(160000*.00004^2-128*.00004+1/100)^(1/2))/(300*.00004))/(.00005-.00004)*100}\pgfmathresult
            \\
            .00006 &72.1191645491&-233778709.199
            %\pgfmathparse{(-4000*.00006+1+10*(160000*.00006^2-128*.00006+1/100)^(1/2))/(300*.00006)}\pgfmathresult &
           % \pgfmathparse{(-4000*.00006+1-10*(160000*.00006^2-128*.00006+1/100)^(1/2))/(300*.00006)}\pgfmathresult &
           % \pgfmathparse{((-4000*.00006+1+10*(160000*.00006^2-128*.00006+1/100)^(1/2))/(300*.00006)-(-4000*.00005+1+10*(160000*.00005^2-128*.00005+1/100)^(1/2))/(300*.00005))/(.00006-.00005)*100}\pgfmathresult&
           % \pgfmathparse{((-4000*.00006+1-10*(160000*.00006^2-128*.00006+1/100)^(1/2))/(300*.00006)-(-4000*.00005+1-10*(160000*.00005^2-128*.00005+1/100)^(1/2))/(300*.00005))/(.00006-.00005)*100}\pgfmathresult
            \\
            \hline
            \end{tabular}
    \end{center}
    \textbf{Therefore the best sort of sensitivity is somewhere between .00004 and .00005}

    %% \pgfmathparse{(10*(21-321^(1/2)))/3}\pgfmathresult
    \item The robustness is fairly good since the values obtained from both the linear and the quadratic models are roughly similar.

\end{enumerate}

\section{Problem 9}

\begin{enumerate}[label=\alph*]
    \endgraf
    \item\endgraf\bigskip

        \begin{steps}
            \endgraf
            \item \emph{Ask the Question, determine variables, constants, assumptions}\endgraf
            \begin{center}
                \begin{tabular}{ |c|c|c|c| }
                \hline
                Variables & Constants & Assumptions \\
                \hline
                $s$ - Subscribers & $i_0 = 1.50$ & $f=sp$\\
                $p$ - Subscription price & $s_0=80000$&$s=s_0-50000(p-p_0)$\\
                $f$ - Profits &&$p\geq0$\\
                && $s\geq0$\\

            \hline
            \end{tabular}
        \end{center}
            \item \emph{Select the model}\endgraf
            One-variable optimization
            \item \emph{Formulate the model} \endgraf
            $$f=sp$$
            $$s=s_0-50000(p-p_0)$$
            $$f=p(s_0-50000(p-p_0))$$
            \parbox{\linewidth}{\centering%
            $s_0=80000$ \hspace*{3cm} $p_0=1.50$ \endgraf \bigskip
            }
            $$f=p(80000-50000(p-1.5))$$
            or simplified
            $$f=-50000p^2+155000p$$
            \item \emph{Solve the model}\endgraf
            Find the extrema for f:
            $$\frac{df}{dp}=0=-100000p+155000$$
            $$p=1.55$$
            Determine if 1.55 is a max or min:
            \begin{tikzpicture}
                \begin{axis}[
                axis lines = left,
                xlabel = Price per paper,
            ]
            \addplot [
                domain=0:2,
                samples=100,
                color=red,
            ]{-100000*x+155000};
            \addlegendentry{$\frac{df}{dp}$}
            \draw[ultra thin] (axis cs:\pgfkeysvalueof{/pgfplots/xmin},0) -- (axis cs:\pgfkeysvalueof{/pgfplots/xmax},0);
        \end{axis}
    \end{tikzpicture}
        Since the derivative of $f$ is going from positive to negative at the point 1.55, \textbf{$p=1.55$ is a Maximum.}\bigskip
            \item \emph{Answer the question}\endgraf
            The best price to sell the newspaper is \$1.55 for a total of \$$120125$ per week, and a total of $77500$ subscribers.
        \end{steps}

    %% \pgfmathparse{(10*(21-321^(1/2)))/3}\pgfmathresult

    \item \endgraf \bigskip
        \emph{What if the newspaper lost a different number of subscribers, other than 5000?}
        $$f=p(80000-r*10(p-1.5))$$
        $$\frac{df}{dp}=0=-20rp+15r+80000$$
        $$p=\frac{80000+15r}{20r}$$
        replace $p$ in the $f$ equation:
        $$f=\frac{80000+15r}{20r}\left(80000-10r\left(\frac{80000+15r}{20r}-1.5\right)\right)$$
        Which doesn't really simplify down to anything worth mentioning.
        \begin{center}
                \begin{tabular}{ |c|c| }
                \hline
                Lost Subscribers ($r$) & Max Profit \\
                \hline
                3000 &\$130208.33\\
                4000 &\$122500.00\\
                5000 &\$120125.00\\
                6000 &\$120416.67\\
                7000 &\$122232.14\\

            \hline
        \end{tabular}
    \end{center}
    \item
    $$f=\frac{80000+15r}{20r}\left(80000-10r\left(\frac{80000+15r}{20r}-1.5\right)\right)$$
    $$\frac{df}{dr}=\frac{45r^2-1280000000}{8r^2}$$
    Sensitivity=$\frac{df}{dr}*\frac{r}{x}=S(p,n)$
    $$S(p,n)=\frac{45n^2-1280000000}{8n^2}\left(n/\left(\left(\frac{80000+15n}{20n}\left(80000-10n\left(\frac{80000+15n}{20n}-1.5\right)\right)\right)\right)\right)$$
    \item \emph{The Newspaper shouldn't need to change its prices.} The price is already near the optimal rate, but the optimal rate is just \$125 more than the basic, which is basically nothing when considering the amount of profit they received.

\end{enumerate}

\end{document}


\documentclass[a4paper]{article}

\usepackage{amsmath, amssymb}
\usepackage{circuitikz}
\usepackage{csvsimple}
\usepackage{pgfplots}
\usepackage{siunitx}

\title{Analyzing Circuit Resistance with Various Lengths of Nichrome Wire}
\date{1 December 2014}
\author{Tarik Onalan}

\def\mean#1{\left< #1 \right>}
\def\abs#1{\left| #1 \right|}
\numberwithin{equation}{subsection}

\begin{filecontents}{data.dat}
    Len  Len_Inv T_1   T_2   T_3   T_4   T_5   Avg   Err   Len_Err
    12.2 0.0820  0.620 0.621 0.619 0.622 0.622 0.621 0.002 0.05
    18.6 0.0538  0.406 0.430 0.423 0.422 0.423 0.421 0.017 0.05
    31.6 0.0316  0.232 0.231 0.229 0.234 0.233 0.232 0.003 0.05
    42.8 0.0234  0.210 0.208 0.206 0.206 0.208 0.208 0.002 0.05
    48.6 0.0206  0.182 0.181 0.180 0.179 0.181 0.181 0.002 0.05
\end{filecontents}

\begin{document}
    \maketitle
    \section{Introduction}
        \textbf{How does the length of nichrome wire in a circuit affect the current
        of the circuit?}\\
        The goal of this lab was to analyze how varying lengths of a resistor, nichrome
        wire, affected current flow in a circuit. My hypothesis is that as resistor
        length increases, current flow within the circuit will decrease, as the more
        resistance there is in a circuit, the lower the current will be. For the experiment,
        increasing lengths of nichrome wire will be inserted into a circuit; if the
        current in the circuit decreases as the length of the nichrome wire increases,
        my hypothesis will be validated. The manipulated and responding variables are
        the length of the nichrome wire and the current through the circuit, respectively.
        The controlled variables are the battery used (for power level), the voltage of
        the battery, the length of the low-resistance wire, the ammeter used, and the
        sensitivity setting used on the ammeter.

    \section{Materials}
        \begin{itemize}
            \item 1 Ammeter
            \item 1 Battery
            \item 1 Section of Low-Resistance Wire
            \item 5 Sections of Nichrome Wire (Varying lengths)
        \end{itemize}

    \section{Procedure}
        \begin{enumerate}
            \item Set up ammeter with battery and low-resistance wire:
            \begin{itemize}
                \item Positive terminal of battery connected to positive terminal
                    of ammeter
                \item Negative terminal of battery connected to negative terminal
                    of ammeter
                \item Low-resistance wire attached to one of two terminals, other
                    terminal reserved for nichrome wire
                \begin{itemize}
                    \item Note: Use \SI{1}{\A} sensitivity instead of \SI{5}{\A}
                        sensitivity
                \end{itemize}
            \end{itemize}
            \item Attach ends of nichrome wire to open terminals of battery and ammeter
            \begin{itemize}
                \item For simplicity, start with the shortest length of nichrome wire
            \end{itemize}
            \item Record amperage shown on ammeter
            \item Repeat steps 2-3 as necessary for data averaging
            \item Repeat steps 2-4 with increasing nichrome wire lengths
        \end{enumerate}

    \section{Diagram}
        \begin{circuitikz}
            \draw
                (0,0) to [battery]  (0,4)
                      to [ammeter]  (4,4) -- (4,0)
                      to [vR] (0,0)
                ;
        \end{circuitikz}

    \section{Data}
        \begin{tabular}{|c|||c|c|c|c|c||c|}
            \hline
            \bfseries Length & \bfseries Trial 1 & \bfseries Trial 2 & \bfseries Trial 3 & \bfseries Trial 4 & \bfseries Trial 5 & \bfseries Average
            \\\hline
            \SI{12.2}{\cm} & \SI{0.620}{\A} & \SI{0.621}{\A} & \SI{0.619}{\A} & \SI{0.622}{\A} & \SI{0.622}{\A} & \SI{0.621}{\A}
            \\\hline
            \SI{18.6}{\cm} & \SI{0.406}{\A} & \SI{0.430}{\A} & \SI{0.423}{\A} & \SI{0.422}{\A} & \SI{0.423}{\A} & \SI{0.421}{\A}
            \\\hline
            \SI{31.6}{\cm} & \SI{0.232}{\A} & \SI{0.231}{\A} & \SI{0.229}{\A} & \SI{0.234}{\A} & \SI{0.233}{\A} & \SI{0.232}{\A}
            \\\hline
            \SI{42.8}{\cm} & \SI{0.210}{\A} & \SI{0.208}{\A} & \SI{0.206}{\A} & \SI{0.206}{\A} & \SI{0.208}{\A} & \SI{0.208}{\A}
            \\\hline
            \SI{48.6}{\cm} & \SI{0.182}{\A} & \SI{0.181}{\A} & \SI{0.180}{\A} & \SI{0.179}{\A} & \SI{0.181}{\A} & \SI{0.181}{\A}
            \\\hline
        \end{tabular}
        \\
        \begin{tabular}{|c|c|}
            \hline
            \bfseries Length Error & \bfseries Amperage Error
            \\\hline
            0.05 & 0.002
            \\\hline
            --- & 0.017
            \\\hline
            --- & 0.003
            \\\hline
            --- & 0.002
            \\\hline
            --- & 0.002
            \\\hline
        \end{tabular}
        \\
        \begin{tikzpicture}[trim axis left, trim axis right]
            \begin{axis}[
                scale=1.75,
                title={Current Relative to Resistor Length},
                xlabel={Length [\si{\cm}]},
                ylabel={Current [\si{\A}]},
                xmin=0.0, xmax=50.0,
                ymin=0.0, ymax=0.7,
                legend pos=north east,
                ymajorgrids=true,
                grid style=dashed
            ]
                \addplot[
                    color=blue,
                    mark=*,
                    only marks
                ] plot [
                    error bars/.cd,
                        x dir=both,
                        y dir=both,
                        x explicit,
                        y explicit
                ] table [
                    x=Len,
                    y=Avg,
                    x error=Len_Err,
                    y error=Err
                ]{data.dat};

                \addplot[
                    color=red,
                    mark=none,
                    domain=0:50
                ]{6.28197/x^0.926264};

                \addlegendentry{Average}
                \addlegendentry{\(6.28197x^{-0.926264}\)}
            \end{axis}
        \end{tikzpicture}
        \\
        \begin{tikzpicture}
            \begin{axis}[
                scale=1.75,
                title={Current Relative to Inverse of Resistor Length},
                xlabel={\(\frac{1}{Length [\si{\cm}]}\)},
                ylabel={Current [\si{\A}]},
                xmin=0.0, xmax=0.1,
                ymin=0.0, ymax=0.7,
                x dir=reverse,
                legend pos=north east,
                ymajorgrids=true,
                grid style=dashed
            ]
                \addplot[
                    color=blue,
                    mark=*,
                    only marks
                ] plot [
                    error bars/.cd,
                    x dir=both,
                    y dir=both,
                    x explicit,
                    y explicit
                ] table [
                    x=Len_Inv,
                    y=Avg,
                    y error=Err
                ]{data.dat};

                \addplot[
                    color=red,
                    mark=none,
                    domain=0:0.1
                ]{7.2379*x+0.0265815};

                \addlegendentry{Average}
                \addlegendentry{\(7.2379x+0.0265815\)}
            \end{axis}
        \end{tikzpicture}

        \section{Calculations}
            \centerline{For below calculations, \(t_{i}=t_{1}\)}
            \subsection{Average}
                \begin{equation} \label{eq:avg}
                    t_{avg}=\mean{t_{i}}
                \end{equation}
                \centerline{where \(t_{i}\) is the set of results for wire \(i\).
                Therefore,}
                \begin{equation}
                    t_{avg}=\mean{[0.620,0.621,0.619,0.622,0.622]}=0.621
                \end{equation}

            \subsection{Error}
                \begin{equation} \label{eq:err}
                    t_{err}=\pm\abs{t_{avg}-t_{E}}
                \end{equation}
                \centerline{where \(t_{E}\) is the element of set \(t_{i}\) with
                the highest deviation from \(t_{avg}\). Therefore,}
                \begin{equation}
                    t_{err}=0.621-0.619=\pm0.002
                \end{equation}

        \section{Conclusion}
            It can be observed through the data that the current through the example
            circuit decreased as the length of the nichrome wire increased. When
            the nichrome wire was \SI{12.2}{\cm} long, the average current was
            \SI{0.621}{\A}. However, when the length increased to \SI{18.6}{\cm},
            the current decreased to \SI{0.421}{\A} This was to be expected, as we
            observed through tests with bulbs that as bulbs were added to the circuit,
            resistance increased, lowering the current of the circuit. In this case,
            however, bulbs are replaced by nichrome wire, and the number of bulbs
            is analogous to the length of the wire. The graphs show the inverse
            relationship between resistor length and current; the best-fit graph
            is nearly equivalent to an inverse function: \(y=ax^{-1}\) and
            \(y=6.28197x^{-0.926264}\). This validates my prediction that current
            would decrease as resistance increased.\\
            One of the major difficulties with the lab was getting an accurate length
            for the nichrome wire. As our group used longer lengths of wire, the wire
            curved more, making accurate measurement of its length more difficult. This
            is slightly reflected in the graph, with more deviation from the best-fit
            line as the length of the wire increased.\\
            The goal of this lab is to understand how resistance affects current flow
            in a circuit. Noting that, the length of nichrome wire in a circuit seems
            less important. Instead, the effective \textit{resistance} of the wire
            seems more important. So instead of using resistors of inconsistent resistance,
            it would be simpler to use resistors of constant resistance, and simply
            add more resistors in series to increase the resistance. This would remove
            the variable of the length of the wire, removing a source of possible error.
\end{document}


\chapter{Conclusions}\label{conclusionsChapter}

This thesis describes LeapGesture, which is the hand gesture recognition library dedicated for Leap Motion Controller.
This library includes modules for static and dynamic gesture recognition and also fingers recognition.
Developed library provides recognition of all types of action gestures.
Static action gestures are supported by static gesture processing module, while the dynamic gesture processing module provides recognition of dynamic action gestures.
Additionally, comparison of existing gesture recognition methods, implementation of additional modules enabling the recording and reviewing of gestures, creation of sample gestures database and performation of tests has been conducted.
%This thesis contains a thorough evaluation of proposed approaches to the gesture recognition.

The results obtained for the static gesture recognition suggest than those gestures are easy to recognize using the SVM with appropriate pre and postprocessing.
The differences between the recognition rates for different feature sets even values reaching up to $13\%$ show that choosing different feature sets can have a major impact on the performance of recognition.
Additionally, it is important not to undermine the influence of the data preprocessing.
Even the usage of the relatively simple median filter allowed to boost the recognition rate by $6\%$.
Although, the authors believe that the proposed parameters of processing module work well in many scenarios, there might be some specific cases that need different parameters or different feature sets to achieve good results.
It is also worth noting, that this and any another approach will not work properly if during the performance of static gestures, hands and fingers are not correctly detected by the Leap Motion.
Therefore it is recommended to always choose gestures that have visible and separated fingers from the sensor's perspective.
For the proposed approach, recognition rate of $99\%$ for five classes of gestures and $85\%$ for ten classes of gestures in the static gesture recognition task were achieved. Satisfactory results of $93\%$ were also obtained for the task of fingers recognition for $15$ of $32$ classes of fingers arrangements.

When it comes to the task of dynamic gesture recognition, Leap Motion might not be the best sensor choice.
Leap Motion provides great accuracy for static hand and fingers, but the data for dynamically moved hands are usually noisy.
The problems arise with short, lost finger tracking or temporal finger occlusions that are hard to detect and cope with on the library-level.
The proposed preprocessing module tries to alleviate those negative impacts, but the preprocessed data is still far from ideal.
Nevertheless the proposed approach with Hidden Markov Models allowed to recognize five classes of dynamic gestures with $80\%$ accuracy.

As part of the further development of the library, the authors intend to improve the efficiency of dynamic gesture recognition by testing other possible feature sets and to develop methods that will allow to preprocessed data derived from Leap Motion more efficiently. Supporting of parameterized gestures is also envisaged.


\section{General concepts using 1D DC resistivity inversion}\label{sec:dc1d}
{\em See cpp/py files in the directory \file{doc/tutorial/code/dc1d}.}
\subsection{Smooth inversion}\label{sec:dc1dsmooth}
{\em Example file \file{dc1dsmooth.cpp}.}\\
Included into the GIMLi library are several electromagnetic 1d forward operators.
For direct current resistivity there is a semi-analytical solution using infinite sums that are approximated by Ghosh filters.
The resulting function calculates the apparent resistivity of any array for a given resistivity and thickness vector.
There are two main parameterisation types:
\begin{itemize}
	\item a fixed parameterisation where only the parameters are varied
	\item a variable parameterisation where parameters and geometry is varied
\end{itemize}

Although for 1d problems the latter is the typical one (resistivity and thickness), we start with the first since it is more general for 2d/3d problems and actually easier.
Accordingly, in the file dc1dmodelling.h/cpp two classes called \cw{DC1dRhoModelling} and \cw{DC1dBlockModelling} are defined.
For the first we first define a thickness vector and create a mesh using the function \cw{createMesh1d}.
The the forward operator is initialized with the data.

\begin{lstlisting}[language=C++]
    RMatrix abmnr; loadMatrixCol( abmnr, dataFile ); //! read data
    RVector ab2 = abmnr[0], mn2 = abmnr[1], rhoa = abmnr[2]; //! 3 columns
    RVector thk( nlay-1, max(ab2) / 2 / ( nlay - 1 ) ); //! const. thickn.
    DC1dRhoModelling f( thk, ab2, mn2 ); //! initialise forward operator
\end{lstlisting}

Note that the mesh generation can also be done automatically using another constructor.
However most applications will read or create a mesh from in application and pass it.

By default, the transformations for data and model are identity transformations.
We initialise two logarithmic transformations for the resistivity and the apparent resistivity by
\begin{lstlisting}[language=C++]
    RTransLog transRho;
    RTransLog transRhoa;
\end{lstlisting}

Alternatively, we could set lower/upper bounds for the resistivity using
\begin{lstlisting}[language=C++]
    RTransLogLU transRho( lowerbound, upperbound);
\end{lstlisting}
Appendix \ref{app:trans} gives an overview on available transformation functions.

Next, the inversion is initialized and a few options are set
\begin{lstlisting}[language=C++]
    RInversion inv( data.rhoa(), f, verbose );
    inv.setTransData( transRhoa );           //! data transform
    inv.setTransModel( transRho );           //! model transform
    inv.setRelativeError( errPerc / 100.0 ); //! constant relative error
\end{lstlisting}

A starting model of constant values (median apparent resistivity) is defined
\begin{lstlisting}[language=C++]
    RVector model( nlay, median( data.rhoa() ) ); //! constant vector
    inv.setModel( model );                        //! starting model
\end{lstlisting}

Finally, the inversion is called and the model is retrieved using \lstinline|model = inv.run();|

A very important parameter is the regularisation parameter $\lambda$ that controls the strength of the smoothness constraints (which are the default constraint for any 1d/2d/3d mesh).
Whereas $w^c$ and $w^m$ are dimensionless and 1 by default, $\lambda$ has, after eq. (\ref{eq:min}), the reciprocal and squared unit of $m$ and can thus have completely different values for different problems\footnote{The regularisation parameter has therefore to be treated logarithmically.}.
However, since often the logarithmic transform is used, the default value of $\lambda=20$ is often a first guess.
Other values are set by
\begin{lstlisting}
    inv.setLambda( lambda ); //! set regularisation parameter
\end{lstlisting}

In order to optimise $\lambda$, the L-curve \citep{guentherruecker06,guentherdiss} can be applied to find a trade-off between data fit and model roughness by setting \lstinline|inv.setOptimizeLambda(true);|.
For synthetic data or field data with well-known errors we can also call \lstinline|model = inv.runChi1();|, which varies $\lambda$ from the starting value such that the data are fitted within noise ($\chi^2=1$).
We created a synthetic model with resistivities of 100(soil)-500(unsaturated)-20(saturated)-1000(bedrock) $\Omega$m and thicknesses of 0.5, 3.5 and 6 meters.
A Schlumberger sounding with AB/2 spacings from 1.0 to 100\,m was simulated and 3\% noise were added.
Data format of the file \file{sond1-100.dat} is the unified data format\footnote{See \url{www.resistivity.net?unidata} for a description.}.

\begin{figure}[htbp]
\includegraphics[width=\textwidth]{sond1-100-3lambdas}\\[-3ex]
~~~a\hfill ~~~b \hfill ~~~c \hfill ~
\caption{Smooth 1d resistivity inversion results for a) $\lambda=200\Rightarrow \chi^2=11.1$/rrms=10.1\%, b) $\lambda=20\Rightarrow \chi^2=1.2$/rrms=3.3\%, and c) $\lambda=2\Rightarrow \chi^2=0.6$/rrms=2.4\%, red-synthetic model, blue-estimated model}\label{fig:dc1d-3lambda}
\end{figure}

Figure~\ref{fig:dc1d-3lambda} shows the inversion result for the three different regularisation parameters 300, 30 and 3.
Whereas the first is over-smoothed, the other are much closer at the reality.
The rightmost figure over-fits the data ($\chi^2=0.3<1$) but is still acceptable.
The L-curve method yields a value of $\lambda=2.7$, which is too low.
However, if we apply the $\chi^2$-optimization we obtain a value of $\lambda=15.2$ and with it the data are neither over- nor under-fitted.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Block inversion}\label{sec:dc1dblock}
{\em Example file \file{dc1dblock.cpp} in the directory \file{doc/tutorial/code/dc1d}.}\\
Alternatively, we might invert for a block model with unknown layer thickness and resistivity.
We change the mesh generation accordingly and use the forward operator \lstinline|DC1dModelling|:
\begin{lstlisting}[language=C++]
    DC1dModelling f( nlay, ab2, mn2 );
\end{lstlisting}

\lstinline|createMesh1DBlock| creates a block model with two regions\footnote{With a second argument \lstinline|createMesh1DBlock| can create a block model with thickness and several parameters for a multi-parameter block inversion.}.
Region 0 contains the thickness vector and region 1 contains the resistivity vector.
There is a region manager as a part of the forward modelling class that administrates the regions.
We first define different transformation functions for thickness and resistivity and associate it to the individual regions.
\begin{lstlisting}[language=C++]
    RTransLog transThk;
    RTransLogLU transRho( lbound, ubound );
    RTransLog transRhoa;
    f.region( 0 )->setTransModel( transThk );
    f.region( 1 )->setTransModel( transRho );
\end{lstlisting}

For block discretisations, the starting model can have a great influence on the results.
We choose the median of the apparent resistivities and a constant thickness derived from the current spread as starting values.
\begin{lstlisting}[language=C++]
    double paraDepth = max( ab2 ) / 3;
    f.region( 0 )->setStartValue( paraDepth / nlay / 2.0 );
    f.region( 1 )->setStartValue( median( rhoa ) );
\end{lstlisting}

For block inversion a scheme after \cite{marquardt}, i.e. a local damping of the changing without interaction of the model parameters and a decreasing regularisation strength is favourable.
\begin{lstlisting}[language=C++]
    inv.setMarquardtScheme( 0.9 ); //! local damping with decreasing lambda
\end{lstlisting}

The latter could also be achieved by \begin{enumerate}
	\item setting the constraint type to zero (damping) by \lstinline|inv.setConstraintType(0)|
	\item switching to local regularization by \lstinline|inv.setLocalRecularization(true)|
	\item defining the lambda decreasing factor by \lstinline|inv.setLambdaDecrease(0.9)|
\end{enumerate}

%The choice of appropriate regularisation parameters is somewhat more complicated since the result strongly depends on the starting model and the preceding models \citep{guentherdiss}.
%The latter can be done by \lstinline|inv.setLambdaDecrease( factor );|

With the default regularization strength $\lambda=20$ we obtain a data fit slightly below the error estimate.
The model (Fig.~\ref{fig:dc1dblock-resres}a) clearly shows the four layers (blue) close to the synthetic model (red).
%Some parameters are overestimated and some are underestimated.

\begin{figure}[htbp]
\centering\includegraphics[width=0.7\textwidth]{dc1dblock-resres}\\[-3ex]
~\hfill a\hfill ~ \hfill ~~~~~b \hfill ~ \hfill ~ \hfill ~
\caption{a) Block 1d resistivity inversion result (red-synthetic model, blue-estimated model)) and b) resolution matrix}\label{fig:dc1dblock-resres}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Resolution analysis}\label{sec:dc1dresolution}
One may now be interested in the resolution properties of the individual model parameters.
The resolution matrix $\R^M$ defines the projection of the real model onto the estimated model:
\begin{equation}
	\m^{est} = \R^M \m^{true} + (\I - \R^M) \m^R + \S^\dagger\D\n\quad,
\end{equation}
\citep{guentherdiss} where $\S^\dagger\D\n$ represents the generalised inverse applied to the noise.
Note that $\m^R$ changes to $\m^k$ for local regularisation schemes \citep{friedel03}.

\citet{guentherdiss} also showed that the model cell resolution (discrete point spread function) can be computed by solving an inverse sub-problem with the corresponding sensitivity distribution instead of the data misfit.
This is implemented in the inversion class by the function \lstinline|modelCellResolution( iModel )| where \lstinline|iModel| is the number of the model cell.
This approach is feasible for bigger higher-dimensional problems and avoids the computation of the whole resolution matrix.
A computation for representative model cells can thus give insight of the resolution properties of different parts of the model.

For the block model we successively compute the whole resolution matrix.
\begin{lstlisting}
    RVector resolution( nModel ); //! create single resolution vector
    RMatrix resM;                 //! create empty matrix
    for ( size_t iModel = 0; iModel < nModel; iModel++ ) {
        resolution = inv.modelCellResolution( iModel );
        resM.push_back( resolution );  //! push back the single vector
    }
    save( resM, "resM" ); //! save resolution matrix
\end{lstlisting}

In Figure~\ref{fig:dc1dblock-resres} the model resolution matrix is shown and the diagonal elements are denoted.
The diagonal elements show that the resolution decreases with depth.
The first resistivity is resolved nearly perfect, whereas the other parameters show deviations from 1.
$\rho_2$ is positively connected with $d_2$, i.e. an increase of resistivity can be compensated by an increased resistivity.
For $\rho_3$ and $d_3$ the correlation is negative.
These are the well known H- and T-equivalences of thin resistors or conductors, respectively, and show the equivalence of possible models that are able to fit the data within noise.
%Similarly, we can obtain the resolution kernels also for smooth inversion.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Structural information}\label{sec:dc1dstruct}
{\em Example file \file{dc1dsmooth-struct.cpp} in the directory \file{doc/tutorial/code/dc1d}.}\\
Assume we know the ground water table at 4\,m from a well.
Although we know nothing about the parameters, this structural information should be incorporated into the model.
We create a thickness vector of constant 0.5\,m.
The first 8 model cells are located above the water table, so the 8th boundary contains the known information.
Therefore we set a marker different from zero (default) after creating the mesh
\begin{lstlisting}
    //! variant 1: set mesh (region) marker
    f.mesh()->boundary( 8 ).setMarker( 1 );
\end{lstlisting}
This causes the boundary between layer 8 and 9 being disregarded, the corresponding $w^c_8$ is zero and allows for arbitrary jumps in the otherwise smoothed model.
Figure~\ref{fig:dc1dsmooth-struct} shows the result, at 4\,m the resistivity jumps from a few hundreds down to almost 10.

\begin{figure}[htbp]
\centering\includegraphics[width=0.45\textwidth]{sond1-100-struct.pdf}
%\\[-3ex]
%~\hfill a\hfill ~ \hfill ~~~~~b \hfill ~ \hfill ~ \hfill ~
\caption{Inversion result with the ground water table at 4\,m as structural constraint.}\label{fig:dc1dsmooth-struct}
\end{figure}

Note that we can set the weight to zero also directly, either as a property of the inversion
\begin{lstlisting}
    //! variant 2: application of a constraint weight vector to inversion
    RVector bc( inv.constraintsCount(), 1.0 );
    bc[ 6 ] = 0.0;
    inv.setCWeight( bc );
\end{lstlisting}
or the (only existing) region.
\begin{lstlisting}
    //! variant 3: application of a boundary control vector to region
    RVector bc( f.regionManager().constraintCount(), 1.0 );
    bc[ 7 ] = 0.0;
    f.region( 0 )->setConstraintsWeight( bc );
\end{lstlisting}
Of course, in 2d/3d inverse problems we do not set the weight by hand.
Instead, we put an additional polygon (2d) or surface (3d) with a marker $\neq 0$ into the PLC before the mesh generation.
By doing so, arbitrary boundaries can be incorporated as known boundaries.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Regions}\label{sec:dc1dregion}
{\em Example file \file{dc1dsmooth-region.cpp} in the directory \file{doc/tutorial/code/dc1d}.}\\
In the latter sections we already used regions.
A default mesh contains a region with number 0.
A block mesh contains a region 0 for the thickness values and regions counting up from 1 for the individual parameters.
Higher dimensional meshes can be created automatically by using region markers, e.g. for specifying different geological units.

In our case we can divide part above and a part below water level.
In 2d or 3d we would, similar to the constraints above, just put a region marker $\neq 0$ into the PLC and the mesh generator will automatically associate this attribute to all cells in the region.

Here we set the markers of the cells $\geq8$ to the region marker 1.
\begin{lstlisting}
    Mesh * mesh = f.mesh();
    mesh->boundary( 8 ).setMarker( 1 );
    for ( size_t i = 8; i < mesh->cellCount(); i++ )
        mesh->cell( i ).setMarker( 1 );
\end{lstlisting}

Now we have two regions that are decoupled automatically.
The inversion result is identical to the one in Figure~\ref{fig:dc1dsmooth-struct}.
However we can now define the properties of each region individually.
For instance, we might know the resistivities to lie between 80 and 800\,$\Omega$m above and between 10 and 100\,$\Omega$m below.
Consequently we define two transformations and apply it to the regions.
\begin{lstlisting}
    RTransLogLU transRho0( 80, 800 );
    RTransLogLU transRho1( 10, 1000 );
    f.region( 0 )->setTransModel( transRho0 );
    f.region( 1 )->setTransModel( transRho1 );
\end{lstlisting}

Additionally we might try to improve the very smooth transition between groundwater and bedrock.
We decrease the model control (strength of smoothness) in the lower region by a factor of 10.
\begin{lstlisting}
    f.region( 1 )->setModelControl( 0.1 );
\end{lstlisting}

The result is shown in Figure~\ref{fig:dc1dsmooth-region}.
The resistivity values are much better due to adding information about the valid ranges.
Furthermore the transition zone in the lower region is clear.
\begin{figure}[htbp]
\centering\includegraphics[width=0.45\textwidth]{sond1-100-region.pdf}
\caption{Inversion result using two regions of individual range constraint transformations and regularization strength (model control).}%
\label{fig:dc1dsmooth-region}
\end{figure}


\subsection{The Keynesian cross and the Investment Saving (IS) curve}

\subsubsection{The Keynesian cross}

We have:

\(Y=C(Y-T(Y))+I(r)+G+NX(Y)\)

Where:

\begin{itemize}
\item \(Y\) is output
\item \(C\) is consumption
\item \(T\) is taxes
\item \(I\) is investment
\item \(r\) is the real interest rate
\item \(G\) is government spending
\item \(NX\) is net exports
\end{itemize}

The Keynesian cross plots:

\(Y\)

Against:

\(C(Y-T(Y))+I(r)+G+NX(Y)\)

This identifies an equilibrium level of output.

\subsubsection{The IS curve}

The IS curve plots the equilibrium level of output from the Keynesian cross against the real interest rate.

As the real interest rate rises, investment and therefore output falls.

\subsubsection{The slope of the IS curve}

The slope of the IS curve depends on taxes and net exports.