\documentclass[modern, letterpaper]{aastex63}

\usepackage{microtype}
\usepackage{amsmath}

% \include{figures/githash}

\newcommand{\package}[1]{\textsf{#1}}
\newcommand{\project}[1]{\textsl{#1}}
\newcommand{\acronym}[1]{{\small{#1}}}

\newcommand{\Gaia}{\project{Gaia}}

% Math
\newcommand{\dd}{\ensuremath{\,\mathrm{d}}}

\shorttitle{Unresolved spectroscopic binaries with \Gaia\ I}
\shortauthors{Foreman-Mackey et al.}

\begin{document}

\title{Unresolved spectroscopic binaries with \Gaia\ I: \\
	Detection \& orbital characterization using EDR3 radial velocity errors}

\correspondingauthor{Daniel~Foreman-Mackey}
\email{dforeman-mackey@flatironinstitute.org}

\author[0000-0002-9328-5652]{Daniel Foreman-Mackey}
\affiliation{Center for Computational Astrophysics, Flatiron Institute, 162 5th Ave, New York, NY 10010, USA}

\author{Quadry~Chance}
\affiliation{Center for Computational Astrophysics, Flatiron Institute, 162 5th Ave, New York, NY 10010, USA}
\affiliation{University of Florida, 211 Bryant Space Science Center, Gainesville, FL 32611, USA}

\author{Andrew~R.~Casey}
\affiliation{School of Physics \& Astronomy, Monash University, Wellington Road, Clayton 3800, Victoria, Australia}

\author{Others TBD}
\noaffiliation{}

\begin{abstract}
	The Early Data Release 3 (EDR3) from the \Gaia\ Mission includes measurements of the radial velocities (RVs) of more than 7 million targets with the goal of providing a detailed view of the phase space of the Milky Way.
	Many of these stars are, in fact, members of unresolved multiple star systems and the effects of these orbits are generally seen as a source of contamination in the RV catalog.
	In this paper, we demonstrate that the published RV error estimates can provide evidence for the existence of an unresolved stellar companion and, under the assumption of a gravitationally-bound binary system, place constraints its orbital parameters.
	This paper makes three key contributions:
	\emph{(1)} We build a flexible data-driven model to estimate \Gaia's intrinsic per-transit RV uncertainty as a function of color and apparent magnitude.
	\emph{(2)} Using these estimates, we develop a procedure for quantifying the likelihood that a given RV target is an unresolved multiple-star system, and compute apply this procedure to all targets with at least 3 RV transits in EDR3.
	\emph{(3)} Finally, we derive a physical model for inferring the orbital parameters of the binary system based only on the published RV error, the number of RV transits, and the observational baseline.
	These parameters are generally only weakly constrained, with the exception of the RV semi-amplitude, which can typically be estimated with a precision of $\sim PPP\%$.
	To demonstrate the reliability of this method and quantify its completeness, we apply our method to simulated data and catalogs of previously known spectroscopic binaries.
	We expect that this value-added catalog will be useful for population studies of binary star systems, target selection for precise RV surveys, vetting exoplanet discoveries, and other applications.
\end{abstract}

\keywords{Astrostatistics (1882) --- Binary stars (154) --- Radial velocity (1332)}

\section{Introduction} \label{sec:intro}

By the end of the \Gaia\ Mission, it will discover SOME LARGE NUMBER of exoplanet and multiple star systems based on time resolved astrometry and radial velocities of many targets.
In the meantime, we only have static measurements of the radial velocity, but it turns out that there is still information in the existing public facing catalog to place constraints on the orbital parameters of multiple systems using the \Gaia\ data.
It has been previously demonstrated that the published radial velocity and astrometric ``errors'' can be used a proxy for multiplicity or data quality.
In this paper, however, we demonstrate that it is possible to use the reported radial velocity errors and a probabilistic model to place constraints on the radial velocity amplitude and, in some cases, some other properties of the orbit.
These measurements are useful for many applications, including \emph{(a)} discovering black holes, \emph{(b)} vetting transiting exoplanet discoveries, \emph{(c)} measuring the masses of a large sample of eclipsing binaries, \emph{(d)} quantify the binary fraction across the H--R diagram, and \emph{(e)} informing constraints on exoplanet formation and evolution theory, to name a few.

The fundamental idea of this paper is that even though the \Gaia\ catalog only includes a single RV measurement and its error, we can still make calibrated probabilistic inferences about the RV time series.
In particular, we know the number of RV observations that were used to measure the RV and its uncertainty.
Similarly, we can estimate the expected per-transit RV uncertainty based on the distribution of targets of similar color and magnitude.
Using these auxillary values, we can compute the expected distribution of RV error, and compare this to the measured value.
It has been previously identified that the RV error measurements scale with RV semi-amplitude (IS THERE A CITATION; add a figure), but below we take this one step further and derive an interpretable, physical, probabilistic model for this effect that allows us to directly infer the semi-amplitude, and weak constraints on the other orbital parameters.

In the following sections, we derive and discuss the components of this model, validate and characterize the performance using simulated data, and compare to archival binary surveys.
To start, we build a hierarchical probabilistic model to estimate the per-transit RV measurement uncertainty of the \Gaia\ survey as a function of apparent magnitude and observed color.

\section{The basics}

For each RV target, the \Gaia\ EDR3 catalog reports the median RV, an estimate of the error on this median, and the number of individual RV measurements (called ``transits'') that were used in this estimate.
In our analysis, we ignore the median RV and instead only consider the RV error and number of transits.
The RV error $\epsilon_n$ for a target $n$ is computed as \citep{Katz19}
\begin{eqnarray}
	{\epsilon_n}^2 = \left(\sqrt{\frac{\pi}{2\,T_n}}\,s_n\right)^2 + 0.11^2
\end{eqnarray}
where
\begin{eqnarray}
	\label{eq:sample-var}
	{s_n}^2 = \frac{1}{N-1}\sum_{t=1}^{T_n} \left(v_{n,t} - \bar{v}_n\right)^2
\end{eqnarray}
is the sample variance of the individual RV measurements $v_{n,t}$.
In \autoref{eq:sample-var}, $t=1,\cdots,T_n$ indexes the $T_n$ transits for target $n$ and
\begin{eqnarray}
	\bar{v}_n = \frac{1}{N}\sum_{t=1}^{T_n} v_{n,t}
\end{eqnarray}
is the mean RV.
In all that follows, it is much simpler to reason about the sample variance ${s_n}^2$ instead of the error on the median ${\epsilon}_n$, but that isn't a problem because we can compute
\begin{eqnarray}
	{s_n}^2 = \frac{2\,T_n}{\pi}\left({\epsilon_n}^2 - 0.11^2\right)
\end{eqnarray}
for any target, using only values available in the catalog: $\epsilon_n$ and $T_n$.

With this out of the way, the basic intuition underlying the methods discussed below is that, for a given target $n$, if we know the number of RV transits $T_n$ and the per-transit RV uncertainty $\sigma_n$, we can estimate the sampling distribution for ${s_n}^2$.

\section{Estimating the per-transit radial velocity precision}\label{sec:noise}

A key element of our analysis is that we must make a reasonably accurate estimate of the per-transit radial velocity measurement uncertainty.
The \Gaia\ pipeline does not release a public estimate of this, but it can be estimated from the data in the public catalog.
To make this estimate, the key assumption that we make is that the RV uncertainty depends only on a target's (reddened) $G_\mathrm{BP} - G_\mathrm{RP}$ color and apparent $G$-band magnitude $m_\mathrm{G}$.
While this is certainly not the full story, we discuss several validation experiments below and argue that this is not an overly restrictive assumption given the precision of our measurements.

\subsection{The fundamental model}

To build intuition, let's start with a simplified toy model that we will extend below.
If we have a catalog of $N$ targets where only the sample variance ${s_n}^2$ of the RV time series and the number of observations $T_n$ of target $n$.
For our initial toy model, let's also assume that we know that the only source of noise is the per-transit measurement uncertainty, which we will assume to be Gaussian.
In this case, the generative model is
\begin{eqnarray}
	v_{n,t} &\sim& \mathcal{N}(\mu_n,\,\sigma^2)
\end{eqnarray}
and then $s_n$ is computed as defined above in \autoref{eq:sample-var}.
In this model, our data are the empirical sample variances ${s_n}^2$ and we want to infer the underlying variance $\sigma^2$ that is shared by all targets $n$.
In other words, we must evaluate the likelihood function
\begin{eqnarray}
	\label{eq:noise-like1}
	\mathcal{L}(\sigma^2;\,\left\{{s_n}^2\right\}) &=& p(\left\{{s_n}^2\right\}\,|\,\sigma^2) \\
	&=& \prod_{n=1}^N\int p({s_n}^2\,|\,v_{n,t})\,p(v_{n,t}\,|\,\sigma^2) \dd v_{n,t}\quad.
\end{eqnarray}
From this equation we can see that the statistic $X_n$ (WAIT, CAN WE? MORE HERE)
\begin{eqnarray}
	X_n &=& \frac{(T_n - 1)\,{s_n}^2}{\sigma^2}
\end{eqnarray}
is chi-squared distributed with $T_n - 1$ degrees of freedom.
Therefore, \autoref{eq:noise-like1} becomes
\begin{eqnarray}
	\label{eq:noise-like2}
	\mathcal{L}(\sigma^2;\,\left\{{s_n}^2\right\}) &=& \prod_{n=1}^N \left|\frac{\partial X_n}{\partial {s_n}^2}\right|\,p(X_n\,|\,\sigma^2) \nonumber\\
	&=& \prod_{n=1}^N \frac{T_n - 1}{\sigma^2}\,p(X_n\,|\,\sigma^2)
\end{eqnarray}
where $p(X_n\,|\,\sigma)$ is a chi-squared density with $T_n - 1$ degrees of freedom.
In this equation, the Jacobian term is important since $\sigma^2$ is a parameter of the model.

\subsection{Accounting for per-target variability}

In the previous discussion we assumed that the \emph{only} source of noise contributing to the RV sample variance measurement was the individual RV measurement uncertainties.
This isn't going to be true when working with real data since there will by other sources of excess noise, both systematic and astrophysical; in fact, that's the whole point of this paper!
To handle this, we add $N$ new parameters ${\delta_n}^2$ to our model, which we will add to the baseline uncertainty $\sigma^2$ when computing the expected variance for a target $n$.
With this change, our model is nearly identical to \autoref{eq:noise-like2}, with $\sigma^2$ replaced with $\sigma^2 + {\delta_n}^2$
\begin{eqnarray}
	\label{eq:noise-like3}
	\mathcal{L}(\sigma^2,\,\left\{{\delta_n}^2\right\};\,\left\{{s_n}^2\right\}) &=& \prod_{n=1}^N \frac{T_n - 1}{\sigma^2 + {\delta_n}^2}\,p(X_n\,|\,\sigma^2 + {\delta_n}^2) \quad.
\end{eqnarray}
This is a very flexible model with more parameters than data points, but it is still possible to perform parameter estimation.

\subsection{Scaling inference}

In the previous subsection, we defined a probabilistic model for simultaneously inferring the (shared) measurement uncertainty $\sigma$ and the (per-target) intrinsic variability $\delta_n$ for an ensemble of \Gaia\ RV targets.
At this point, the main parameter that we're interested in inferring is $\sigma$ and to make sensible measurements of this value, we need to place priors on $\delta_n$ and marginalize.
To do this, we use a mean-field variational inference (VI) model implemented in \package{numpyro} and \package{jax}.
In this model, the parameters are $\log \sigma$ and $\log \delta_n$ with broad zero-mean Gaussian priors and the likelihood is given be \autoref{eq:noise-like3}.
We model the full posterior density as a multivariate Gaussian in these logarithmic parameters and then optimize the Kullback--Leibler divergence between this approximate posterior and the actual posterior as described by CITE AUTODIFF VI PAPER.

We then apply this VI procedure in bins of color and magnitude (ADD SPECS OF GRID).
In each bin, we perform 5 iterations of the following procedure and average the estimated values of $\sigma$ across iterations.
For each iteration, we select up to 1000 random targets within the bin\footnote{If the bin has fewer than 1000 targets, we use all of the targets.}, run 5000 steps of VI to infer the mean of the marginalized posterior for $\log \sigma$ and return that value.
To minimize the effects of shot noise, we then lightly smooth this grid of estimated $\log \sigma$ across color and magnitude.
\autoref{fig:noise_model} shows this estimate of the per-transit RV uncertainty $\sigma$ as a function of observed color and magnitude.
In this figure, the shaded regions indicate parts of parameter space where there were fewer than 50 targets so our procedure could not be applied reliably.

\begin{figure}
	\plotone{figures/noise_model.pdf}
	\caption{A model of \Gaia's per-transit RV uncertainty as a function of its observed color and magnitude.
		This model is constructed using a data-driven hierarchical model to deconvolve the measurement uncertainty and intrinsic RV variability, by assuming that ensembles of targets with similar colors and magnitudes will have the same per-transit RV measurement uncertainty.
		The shaded regions at the edges of this figure have a sufficiently small number of targets that this ensemble model cannot be reliably used and any estimated RV uncertainties in these regions will be extrapolated. \label{fig:noise_model}}
\end{figure}

Using this grid of estimated uncertainties, we interpolate the model to the color and magnitude coordinates of each \Gaia\ RV source to provide an estimate of the baseline RV measurement uncertainty for each target.
\autoref{fig:sigma_cmd} shows the average per-transit RV uncertainty across the color--magnitude diagram, where we have estimated the distance of each source using the inverse parallax.
In this figure, there is some shot noise visible around the edges where the target density drops, but there is not otherwise a huge amount of structure.

\begin{figure}
	\plotone{figures/sigma_cmd.pdf}
	\caption{The typical per-transit RV uncertainty across the \Gaia\ color--magnitude diagram.
		The color bar indicates the binned average per-transit RV uncertainty, and the contours indicate the density of \Gaia\ targets across the same space.
		Any bins with fewer than 50 targets are masked and colored white in this figure. \label{fig:sigma_cmd}}
\end{figure}

\section{Binary probabilities}

One side effect of inferring the per-transit radial velocity uncertainty for all sources in \Gaia\ is that we also get (for free!) an estimate of the probability that the measured RV error is consistent with what we would expect if the individual RV measurements were Gaussian distributed with this uncertainty.
This is some sort of ``outlier'' probability, but we can use it as a proxy for binarity in the absence of systematic, or other un-accounted for effects.
We explore the limitations of such an assumption in the following sections, but it is useful for now to discuss the math for the idealized case here.

If each measured RV sample for a given \Gaia\ target $n$ was truly just generated as Gaussian noise with variance ${\sigma_n}^2$, the sampling distribution for the measured, normalized RV variance $X_n = (T_n - 1)\,{s_n}^2 / {\sigma_n}^2$ would be a $\chi^2$ with $T_n - 1$ degrees of freedom.
Therefore, we can compute the $p$-value for this null hypothesis as
\begin{eqnarray}
	\mathrm{Pr}(X > X_n\,|\,\mathrm{null}) &=& \int_{X_n}^\infty \chi^2 (X;\,T_n-1) \dd X \quad.
\end{eqnarray}
This is a useful number since, it can be used to construct a catalog with a specific expected false positive rate.

Specifically, we construct a binary catalog by selecting all targets with a $p$-value below some threshold $P_0$.
If our Gaussian model is correct and our estimates of the per-transit measurement uncertainty is accurate, then we would expect targets without companions to pass threshold with probability $P_0$.
In other words, setting $P_0 = 0.001$, we would expect 1 in 1,000 single targets to pass our cut and be incorrectly labelled as a binary.
This number does not directly map to the false positive rate, since that will also depend on the true binary fraction, however, it gives us a tuning parameter that can trade off precision and recall in our selection.

\autoref{fig:p_value_dist} shows the observed distribution of $p$-values across the full catalog.
Our null hypothesis here is that every target is a single star with only Gaussian noise with known variance as estimated in \autoref{sec:noise} and, if this hypothesis was correct, we would expect the $p$-values to be uniformly distributed between zero and one (as indicated by the orange dashed histogram in the left panel of \autoref{fig:p_value_dist}).
The observed distribution is significantly different from this expectation and there are a few factors causing this.
MORE HERE.

\begin{figure}
	\plotone{figures/p_value_dist.pdf}
	\caption{The distribution of $p$-values for the full \Gaia\ EDR3 catalog.
		\emph{(left)} The black histogram shows the observed distribution of $p$-values for the sample, and the orange histogram shows the distribution that would be expected for our null hypothesis: that all targets are single stars and the observed RV errors are the result of \emph{only} the per-transit RV uncertainties.
		The overdensity at large $p$ suggests that some fraction of the per-transit RV uncertainties are overestimated, and we interpret the large peak at $p \approx 0$ as an indication of multiplicity.
		\emph{(right)} The same distribution plotted near $p \approx 0$.
		We select $p \leq 0.001$ as our threshold for rejecting the null hypothesis in all the following figures.
		Therefore, the shaded region indicates the targets that would be classified as multiple systems.
		\label{fig:p_value_dist}}
\end{figure}

After computing the $p$-value for every target in our catalog, we can use these values to identify targets that are likely multi-star systems or other kinds of outliers.
\autoref{fig:binary_fraction_cmd} shows the fraction of targets with $p < 0.001$ as a function fo their location in the color--magnitude diagram.
In the absence of other sources of noise\footnote{We discuss these other noise sources in SOME SECTION.}, this can be interpreted to first order as the binary fraction.
With this interpretation, this figure shows some expected structures such as an equal-mass main sequence and an increased binary fraction at higher mass.

\begin{figure}
	\plotone{figures/binary_fraction_cmd.pdf}
	\caption{Like \autoref{fig:noise_model}, but this time the color bar shows the fraction of targets where our null hypothesis is rejected.
		This is an estimate of the fraction of \Gaia\ targets that are detectable multiple star systems as a function of color and magnitude.
		In other words, this can (with the appropriate caveats) be interpreted as the binary fraction across the color--magnitude diagram.
		Under this interpretation, the equal-mass binary main sequence is clearly visible, and there is an increased binary fraction DESCRIBE THIS.
		\label{fig:binary_fraction_cmd}}
\end{figure}

\section{Completeness \& reliability}

We would like to characterize the efficiency with which our method detects astrophysical signals from companions.
In other words we want to estimate the \emph{completeness} of our survey.
To do this, we use simulated injection and recovery tests where we generate simulated data with a known signal and attempt to recover it with our pipeline.
In particular, we will focus on characterizing the survey completeness as a function of the radial velocity measurement uncertainty and semi-amplitude, integrated over all of the other dimensions.
There are other parameters that affect the detection thresholds including, most importantly, the number of RV transits, but these are second order effects and we will marginalize over these effects using realistic distributions.

We use an approximate broken power-law model for the number of radial velocity transits:
\begin{eqnarray}
	\label{eq:nb-transits}
	p(T_n) &\propto& \left\{\begin{array}{ll}
	1                                  & T_n < T_\mathrm{break} \\
	(T_n / T_\mathrm{break})^{-\alpha} & \mathrm{otherwise}
	\end{array} \right.
\end{eqnarray}
where the parameters are set to approximately match the observed distribution.
These parameters are set as follows: the breakpoint is $T_n = 12$ and the power-law index is $\alpha = 6$.
As demonstrated by SOMEFIGURE, this approximate distribution is a reasonably good fit.

Otherwise we sample the other parameters period, RV semi-amplitude, phase, eccentricity, argument of periasteron, and per-transit measurement uncertainty from the simple distributions listed in SOMETABLE.

Then we simulate an RV time series with $T_n$ observations sampled from \autoref{eq:nb-transits} and compute the sample variance.
Then for consistency, we compute the floored and scaled ``error'' $\epsilon$ used to construct the \Gaia\ catalog (SOMEEQUATION) and, like the real catalog, mark any with $\epsilon > 20\,\mathrm{km/s}$ as non-detections.
Using this catalog, we evaluate the $p$-value for each simulated target, and then estimate its RV semi-amplitude.
Figure SOMEFIGURE shows the recovery rate as a function of RV semi-amplitude and per-transit measurement uncertainty.
At small semi-amplitude, the injected signals are not recovered since the variance is too small to distinguish from the measurement noise.
At large semi-amplitude, the signals are not recovered because the catalog is censored at large RV ``error'' ($\epsilon > 20\,\mathrm{km/s}$).
Figure SOMEOTHERFIGURE shows the inferred semi-amplitude for these simulations as a function of the true value.

These measurements don't fully capture all the limitations of this inference, but we will try to address some of those below by comparing to other binary surveys.

\begin{figure}
	\plotone{figures/completeness.pdf}
	\caption{The completeness of our binary detection pipeline calculated using a simulated catalog of binary star systems.
		The color bar indicates the fraction of simulated binaries that are recovered---where recovery is defined as the system having a $p$-value of less than 0.001.
		\emph{(left)} The recovery rate of binary targets as a function of the RV measurement uncertainty and the true RV semi-amplitude of the simulated orbit.
		At fixed measurement uncertainty, the recovery rate increases with semi-amplitude because the signal-to-noise of the error measurement increases.
		The recovery rate also decreases for large semi-amplitudes because the \Gaia\ processing pipeline discards any measurements where the final RV error is larger than 20~km/s, an effect that is included in our simulated catalog.
		\emph{(right)} The same color bar as the right-hand panel, but this time plotted as a function of the number of RV transits.
		Perhaps unsurprisingly, the recovery rate is significantly higher for targets with larger numbers of observations.
		% It's worth noting that since this figure is integrated over a simulated distribution of transit numbers, the detailed shape depends on the actual distribution.
		\label{fig:completeness}}
\end{figure}

\section{A model for the radial velocity error}

Apparently \Gaia\ reports RV error $\epsilon$ defined as:
\begin{eqnarray}
	\epsilon^2 = \left(\sqrt{\frac{\pi}{2\,N}}\,s\right)^2 + 0.11^2
\end{eqnarray}
where
\begin{eqnarray}
	s^2 = \frac{1}{N-1}\sum_{n=1}^N \left(v_n - \bar{v}\right)^2
\end{eqnarray}
is what we actually want.

We need to compute a probability density for the $s^2$ parameter above conditioned on a set of $N$ noisy observations of an RV orbit (with some given parameters).
I'm going to model the noise as Gaussian with a known, constant variance $\sigma^2$, but it's \emph{possible} (although probably tricky) that we could relax some of that.

Next, the key realization is that if we have a set of $N$ Gaussian random variables $X_n \sim \mathcal{N}\left(\mu_n,\,\sigma^2\right)$ (with known, but different, means $\mu_n$, in our case given by the RV orbit evaluated at $t_n$), then the random variable $Y = \sum X_n^2$ will be distributed as a noncentral chi-squared.
To use this to our advantage, we first argue that the argument in the variance sum $v_n - \bar{v}$ will be a Gaussian random variable
\begin{eqnarray}
	v_n - \bar{v} \sim \mathcal{N}\left(\mu_n - \bar{\mu},\,\sigma^2\right)
\end{eqnarray}
where $\mu_n$ is the RV orbit \emph{model} at time $t_n$.
This isn't totally obvious since the $v_n$ and $\bar{v}$ distributions aren't necessarily independent, but I think it's ok because [insert relevant math here] (this derivation will look similar to the sampling distirbution for the sample variance calculations so I should double check there, but this definitely works in practice).

Then we're where we need to be: $(N - 1)\,s^2$, our quantity of interest, is now the sum of squares of $N$ Gaussian random variables.
Following the Wikipedia page for the noncentral chi-squared distribution\footnote{\url{https://en.wikipedia.org/wiki/Noncentral_chi-squared_distribution}}, we need to rescale our parameters to have unit variance:
\begin{eqnarray}
	\frac{v_n - \bar{v}}{\sigma} \sim \mathcal{N}\left(\frac{v_n - \bar{\mu}}{\sigma},\,1\right)
\end{eqnarray}
where we have used the assumption that $\sigma$ is known and constant.
I think that we could allow non-constant $\sigma$ but that would result in a "generalized chi-squared" distribution which has no closed form\footnote{\url{https://en.wikipedia.org/wiki/Generalized_chi-squared_distribution}}.

Then the parameters of our noncentral chi-squared will be

\begin{eqnarray}
	k = N
\end{eqnarray}

and

\begin{eqnarray}
	\lambda = \sum \frac{N\,(\mu_n - \bar{\mu})^2}{(1 + N)\,\sigma^2}
\end{eqnarray}

And the parameter that is thus distributed is:

\begin{eqnarray}
	\xi^2 = \frac{(N - 1)\,s^2}{\sigma^2}
\end{eqnarray}

That was a bit ugly.
Let's see if this works with a simulation.
We'll simulate a single simple RV curve and then compute the distribution of $s^2$ over many different realizations of the noise.

\section{Bulk radial velocity inference}

Finally we can combine the measurement uncertainty estimates from SOME SECTION with the

\section{Artisanal radial velocities}

\section{Validation}

\section{Discussion}

This is a paper.
See \autoref{fig:noise_model}.

\begin{figure}
	\plotone{figures/recovered.pdf}
	\caption{The semi-amplitude estimated from only the RV error and number of transits for a suite of simulated binary orbits, as a function of the true simulated semi-amplitude.
		The points are colored by the eccentricity of the orbit showing a weak trend. MORE HERE?
		\label{fig:recovered}}
\end{figure}

\begin{figure}
	\plotone{figures/sb9.pdf}
	\caption{This is a cool figure. \label{fig:sb9}}
\end{figure}

\begin{figure}
	\plotone{figures/apogee-gold.pdf}
	\caption{This is a cool figure. \label{fig:apogee-gold}}
\end{figure}

\acknowledgments
The authors would like to thank the Astronomical Data Group at Flatiron for listening to every iteration of this project and for providing great feedback every step of the way.

\vspace{5mm}
\facilities{
	\project{APOGEE},
	\project{Gaia}
}
\software{
	\package{AstroPy} \citep{Astropy2013, Astropy2018},
	\package{JAX} \citep{Bradbury2018},
	\package{NumPy} \citep{Harris2020},
	\package{Matplotlib} \citep{Hunter2007},
	\package{SciPy} \citep{Virtanen2020}
}

\appendix

\section{Probably some fancy math}

\bibliography{one-datum}{}
\bibliographystyle{aasjournal}

\end{document}


\section{Overview}

In this chapter, we present a worklist algorithm
that further supports subtyping,
by introducing the top and bottom types.
Such type inference algorithms are known to be hard to design,
due to the complication of unifying variables under subtyping inequality
rather than equality.
Therefore, algorithmic existential variables that appear in subtyping judgments
might have more solutions than before,
and the eager substitution rules miss some possibilities.
Our new \emph{backtracking} algorithm extends the existing one by some overlapping rules that
non-deterministically try different sorts of solutions instead of a single sort,
improving the rate of success guessing.

Formalization of the algorithm in the Abella theorem prover shows soundness of the algorithm
with respect to our declarative specification.
Although the backtracking algorithm is still incomplete in some corner cases,
the proof script indicates that the only source of incompleteness comes from
higher-ranked subtyping relations.
Following that discovery, we formally proved partial completeness of our subtyping algorithm
under the rank-1 restriction,
and is at least comparable to local type inference algorithms.

\subsection{Type Inference in Presence of Subtyping}

In all previous chapters, ``subtyping'' refers to a relationship that compares
the degree of polymorphism between two types.
In addition to that, we also include the top and bottom types.
Both of the types have practical uses, especially in object-oriented programming languages.
The top type, $\top$, is the super type of any type,
i.e. any type is more general than $\top$
and thus can be considered as an instance of type $\top$.
In typical object-oriented programming languages, the \texttt{Object} class,
as the base class of any class, is the $\top$ type.
In contrast, the bottom type, $\bot$, is dual to $\top$.
An instance of $\bot$ can be cast to a value of any type,
which is also viewed as the uninhabited type therefore
one cannot find any value that has the $\bot$ type.
However, there are few exceptions.
One of them is a \texttt{null pointer} value (\texttt{(void *)} in C++, for example).
Another practical use for bottom types is for exceptions.
The type $\text{Exception} \to \bot$ given to the \texttt{raise}
function may pass type checkers naturally.
The type $\text{Exception} \to \all a$ is also a reasonable choice,
which in fact reveals that $\bot$ behaves almost identically as $\all a$.
From the theoretical point of view, both of them represent ``falsity''.

A number of systems and algorithms are proposed for type inference with subtyping.
Since subtyping constraints cannot be easily reduced in presence of subtyping,
it is natural to extend the syntax of types so that unsolved constraints are carried with them,
and the idea is studied by previous work~\citep{RecConstraint1995, subcon1996}.
For example, the following function
\begin{verbatim}
select p v d = if (p v) then v else d
\end{verbatim}
is inferred to have the type
$$(\alpha \to \text{bool}) \to \alpha \to \beta \to \gamma \mid \alpha \le \gamma, \beta \le \gamma$$
However, this representation usually contains a large set of constraints,
which might confuse programmers.
Besides, subtyping comparison between constraint types may introduce additional
difficulty for programmers to understand, which weakens its practical use.

\paragraph{MLsub}
MLsub~\citep{mlsub}, as introduced in Section~\ref{sec:mlsub},
is based on a type system with lattices and polar types.
Inspired by~\citep{pottier1998phd}, they separated input and output types into polar types,
which greatly simplifies the subtyping judgment.
With the further help of their biunification algorithm,
constraints are no longer carried with types,
instead they are expressed directly on types.
For example, the \verb|select| function defined above has type
$$(\alpha \to \text{bool}) \to \alpha \to \beta \to \alpha \sqcup \beta$$
in MLsub.
The $\sqcup$ symbol in the output type indicates that
any of the types might be returned when executed.
On the other hand, the $\sqcap$ symbol might occur on an input type,
when that argument is used as different types,
meaning that the type must be able to convert to all these types.

Nevertheless, there are still drawbacks in practice.
Firstly, the type inference algorithm produces types that
have comparable sizes to previous systems with constraint types.
Although a simplification algorithm is provided,
it is not guaranteed to produce the most simple form.

Secondly, MLsub always infers an expression and returns its principal type,
but some of the types are not easy to understand.
For example, the \verb|twice| function defined as
\begin{verbatim}
  twice f x = f (f x)
\end{verbatim}
has type $(\alpha \sqcup \beta \to \alpha) \to \beta \to \alpha$,
where normally people might expect $(\alpha \to \alpha) \to \alpha \to \alpha$.

Thirdly, MLsub tries to infer on almost every possible expression,
even those that might just be a mistake made by the programmer.
For instance, the function
\begin{verbatim}
  positive f x = if x > 0 then f x else f 0
\end{verbatim}
always checks the positivity of \verb|x| before applying to \verb|f|.
However, a slight mistake is made in the following definition
\begin{verbatim}
  positive' f x = if x > 0 then f x else f
\end{verbatim}
then the algorithm might still accept the program by giving the type:
$$\alpha \sqcap (\beta \to \alpha) \to \beta \sqcap \text{int} \to \alpha$$
Type inference algorithms, as logical tools that help people program,
should accept good programs and reject bad ones.
Typically, such a function is considered to be badly-written,
but unfortunately, MLsub accepts it without giving any warnings.

\subsection{Judgment List and Eager Substitution}

The judgment list algorithm we discussed in the last chapter treats subtyping
judgments mainly as unification.
However, the algorithm may lose some solutions
when top and bottom types are introduced.
For example,
the judgment $\al \le 1$ has the best solution
$\al := 1$ in the previous system,
but now $\al := \bot$ should also be allowed.
As a result, several subtyping judgments cannot be reduced easily with a single
eager substitution.

Similar incompleteness also affects the instantiation and
existential variable solving rules
\begin{gather*}
\begin{aligned}
    \Gm[\al] \Vdash \al \le A\to B &\rrule{10} [\al[1]\to\al[2]/\al] (\Gm[\al[1], \al[2]] \Vdash \al[1]\to \al[2] \le A \to B)\\
    \Gm[\al][\bt] \Vdash \al \le \bt &\rrule{12} [\al/\bt](\Gm[\al][])
\end{aligned}
\end{gather*}
The instantiation judgment $\al \le A \to B$ immediately split $\al$ into $\al[1] \to \al[2]$,
which is no longer the case --- $\al$ can also be $\bot$.

The rule that solves one existential variable against another,
$\al \le \bt$, is even more problematic,
because the shapes of the instantiations of both existential variables are unknown,
there are infinitely many possibilities if no additional constraints present.
For example, if $\al$ is finally instantiated to $\text{Int} \to \text{Int}$,
then $\bt$ may be solved to the same type as $\al$,
or other types that satisfy the subtyping relationship,
including $\text{Int} \to \top$, $\bot \to \text{Int}$ or $\bot \to \top$.
What's worse, given only this single subtyping judgment $\al \le \bt$,
we have no assumption on both existential variables.

Theoretically speaking, if we extend the type system with $\top$ and $\bot$ types,
the definition of monotypes becomes
\[ \tau ::= 1 \mid \bot \mid \top \mid \tau \to \tau \mid a\]
, which causes the subtyping problem to be undecidable~\cite{su2002firstorder}.
Therefore, it is impossible to derive a complete algorithm for the declarative system,
and we try to keep the algorithm complete for common usages of polymorphism,
but pose restrictions in some higher-ranked cases.

% \subsection{Lazy Substitution and Non-terminating Loops}
% discussed later

\subsection{Our Solution: Backtracking Algorithm}

After examining the examples where the worklist algorithm behaves incompletely,
we propose some improvements to the worklist algorithm to accept some simple cases.
We have observed that subtyping judgments like $\al \le A$ and $A \le \al$ have trivial solutions
$\al := \bot$ and $\al := \top$ respectively,
thus we can try these simple solutions before the more complicated analysis.

For example, for the judgment list $\Gm \Vdash \al \le A \to B$,
we first assign $\al := \bot$ and continue to check other judgments.
If the judgment reduction succeeds, then we know that $\al := \bot$ is a possible solution.
If the reduction failed, we continue to try the other possibility according to the
previous instantiation rule by splitting $\al$ into two existential variables $\al[1]$ and $\al[2]$.
The procedure must keep track of all backtracking points until type-checking the whole program.
Note that for the reduced judgment $\al[1] \to \al[2] \le A \to B$,
$\al[1]$ and $\al[2]$ might be solved to $\top$ or $\bot$.

For the type system, there are also interesting changes to
both the declarative system and algorithmic system.
Two algorithmic typing rules also adopt the backtracking approach:
\begin{itemize}
    \item The checking judgment $e \Lto \al$ now has a trivial solution $\al := \top$,
        because $e \Lto \top$ should be accepted for any (well-formed) expression.
    \item The introduction of top and bottom types comes with another new declarative rule
        $\appInf{\bot}{e}{\bot}$.
        Therefore the application inference judgment $\appInfAlg{\al}{e}$
        can be satisfied by $\al := \bot$,
        with output $\bot$ applied to the rest of the judgment chain $\jg$.
\end{itemize}

By applying the above modifications,
the algorithm remains sound w.r.t our new declarative system,
and at the same time, we address most issues regarding the top and bottom types,
obtaining a more complete algorithm.
Furthermore, we formally verified that the subtyping algorithm
is complete under the rank-1 restriction.
However, for rank-2 judgments like $\al \le \bt$,
a naive complete algorithm would traverse an infinitely large tree of possibilities
and diverge for some set of constraints.
% completeness is impossible for any algorithm that reduces eagerly.

Following the formal statements, we conclude that the backtracking algorithm
is complete under a rank-1 restriction, but not always on the general higher-ranked settings.
In those settings, our algorithm only consider the equality case and ignore any other possibilities.
In another perspective, our algorithm does not try to solve completely when
parametric polymorphism and subtyping polymorphism occur at the same time.
This is not theoretically satisfactory, but we argue that
this handles most common programs, and the programmer may use annotation to guide
the inference algorithm with the real type he intends to instantiate.


\documentclass[12]{scrartcl}
\usepackage{amssymb,amsmath,gensymb,dsfont,calc,multicol,fullpage}
\makeatletter
\newcommand\Aboxed[1]{
   \@Aboxed#1\ENDDNE}
\def\@Aboxed#1&#2\ENDDNE{%
   &
   \settowidth\@tempdima{$\displaystyle#1{}$}
   \setlength\@tempdima{\@tempdima+\fboxsep+\fboxrule}
   \kern-\@tempdima
   \boxed{#1#2}
}
\makeatother

\begin{document}

\title{Homework 8, Section 1.8; 4, 7, 8, 9, 15, 26, 30}
\author{Alex Gordon}
\date{\today}
\maketitle
\section*{Homework}
\subsection*{4.}
RREF Matrix results in:
$\begin{bmatrix} 1&0&0&-17 \\0&1&0&-7 \\ 0&0&1&-1  \end{bmatrix}$\\
So $x_1 = -17$\\
$x_2 = -7$\\
$x_3 = -1$\\
and $ X = \begin{bmatrix} -17 \\-7 \\ -1  \end{bmatrix}$\\
This makes it clear that X is unique.
\subsection*{7.}
Let A be a 6x5 matrix. What must a and b be in order to define $T : \mathbb{R}^a \rightarrow \mathbb{R}^b$ by $T(X) = Ax$.
The matrix A and the column vector x are multipliable. If A is a matrix of order p x q, and x is a matrix of order q x l, then the product matrix is of order p x l.
This confirms that the dimension of range space is p, meaning the number of rows of A is the dimension of the range vector space.
Since 6 is the dimension of the range space and 5 is the dimension of the domain vector space, then A is a matrix of order 6 x 5 and $T : \mathbb{R}^a \rightarrow \mathbb{R}^b$ by $T(X) = Ax$ that is A has $b = 6$ rows and $a = 5$ columns and so the co domain of $T$ is $\mathbb{R}^6$ and the domain of $T$ is $\mathbb{R}^5$.
Therefore, $a =5$ and $b = 6$.
\subsection*{8.}
Since $X \in \mathbb{R}^5$, we have x has order $ 5 \cdot 1$ \\
Since $Ax \in \mathbb{R}^7$, we have $Ax$ with order $ 7 \cdot 1$.
Thus, by the product rule of matrices the number of rows in $Ax$ and the number of columns in $X$ equal the number of rows and columns in A, so A must be a matrix with 7 rows and 5 columns.
\subsection*{9.}
RREF Matrix results in:
$\begin{bmatrix} 1&0&-4&10&0 \\0&1&-3&5&0 \\ 0&0&0&4&0  \end{bmatrix}$\\
Hence $x_4 = 0$\\
$x_3 = free$\\
$x_2 = 3x_3$\\
$x_1 = 4x_3$\\
and $ X = \begin{bmatrix} 4 \\3 \\ 1 \\ 0 \end{bmatrix}$\\
\subsection*{15.}
$ T(u) = \begin{bmatrix} 2  \\ 5  \end{bmatrix}$\\
$ T(5) = \begin{bmatrix} 4  \\ -2  \end{bmatrix}$\\
The images of $T(u)$ and $ T(v)$ are a reflection through the line $x_2 = x_1$
\subsection*{26.}

\subsection*{30.}
$T(x) = c_1, \cdot 0 + c_2 \cdot 0 + ...... + c_p \cdot 0$\\
Therefore, $T(x) = 0$ and T is linear.

\end{document}


\documentclass[12pt]{amsart}

\newcommand{\argmax}{\mathop{\rm argmax}}
\newcommand{\qm}{{\rm qm}}

\newcommand{\file}{\textbf}
\newcommand{\url}{\underline}
\newcommand{\code}{\small\texttt}

\begin{document}

\title{Och's Line Maximization Algorithm Explained}
\maketitle

{\tiny
\noindent
K-Best Rescoring Module\\
Aaron Tikuisis\\
Technologies langagi{\`e}res interactives / Interactive Language Technologies\\
Institut de technologie de l'information / Institute for Information Technology\\
Conseil national de recherches Canada / National Research Council Canada\\
Copyright 2005, Sa Majest{\'e} la Reine du Chef du Canada / Her Majesty in Right of Canada
}

%{\tiny
%\noindent
%Version info: \verb+$Id$+
%}
%\\

\section{Introduction\label{sec:intro}}
In my own experience going about understanding and implementing the line maximization algorithm described here, I found the explanation of this algorithm in Och's paper to be a bit lacking.
Hence, to aid in understanding both the algorithm and the implementation here, I attempt to provide a more comprehensive explanation of the algorithm in this document.

\section{The Statistical Model\label{sec:model}}
We use the standard log-linear model with a predetermined set of feature functions $\{h_1, \dots , h_m\}$, and parameterized by the vector $\boldsymbol{\lambda} = (\lambda_1, \dots , \lambda_M)$, so that the probability assigned to a translation is:
\[ p(e|f) = \frac
{\exp \left( \sum_{m=1}^M \lambda_m \cdot h_m(e,f) \right)}
{\sum_{e'} \exp \left( \sum_{m=1}^M \lambda_m \cdot h_m(e',f) \right)}. \]
In the denominator, the sum is taken over \textbf{all} sentences and it is assumed that this sum converges\footnote{It seems that this assumption is in fact often incorrect}.
In using the model to rescore the $K$-best translations, we may skirt this problem entirely by pretending these $K$ sentences are the only sentences:
\[ p(e_k|f \land e \in \{e_1, \cdots , e_K\}) = \frac
{\exp \left( \sum_{m=1}^M \lambda_m \cdot h_m(e_k,f) \right)}
{\sum_{k'=1}^K \exp \left( \sum_{m=1}^M \lambda_m \cdot h_m(e_{k'},f) \right)}. \]

Based on this model, the most probable translation of a sentence $f$ is:
\begin{align}
\hat{e}(f) = \hat{e}(f, \boldsymbol{\lambda}) &= \argmax_{e} p(e|f) \notag \\
&= \argmax_{e} \exp \left( \sum_{m=1}^M \lambda_m \cdot h_m(e,f) \right) \notag \\
&= \argmax_{e} \sum_{m=1}^M \lambda_m \cdot h_m(e,f) \label{eq:e(f)}
\end{align}
Again, in the general case, the argmax ranges over all possible sentences, and in practice, it ranges only over the $K$-best translations.

\section{Input and Purpose\label{sec:input}}
We are given a set $\{f_1, \dots , f_S\}$ of source sentences, called the source corpus, and for each source sentence $f_s$, a set $\{e_{s,1}, \dots , e_{s,K}\}$ of candidate translations (for simplicity, we assume that for each different source sentence, the number of candidate translations is the same).
Generally, these candidate translations should be the $K$ best, as found by some other model/decoder.

We have a function to evaluate the quality of a candidate translation $\{e_{s,k_1}, \dots , e_{s,k_S}\}$, where $e_s$ is a translation of $f_s$ for all $s$.
We write the quality metric $\qm(e_1, \dots e_S)$; using this notation, it is taken for granted that the quality metric may be a function of additional data (eg. reference translations).
BLEU is an example of such a metric.%; in fact, BLEU and all other metrics I've seen have additional properties which make their computation faster; this is discussed in section \ref{sec:scorefunc}.

Finally, we are given a line in the $M$-dimensional parameter space, specified by a point $\boldsymbol{p} \in \mathbb{R}^M$ and a direction vector $\boldsymbol{d} \in \mathbb{R}^M$.
The purpose of Och's line maximization algorithm is to find the point $\boldsymbol{\lambda}^\star = \boldsymbol{p} + \gamma^\star \boldsymbol{d}$ (where $\gamma^\star \in \mathbb{R}$) along this line that maximizes the value of the quality metric $\qm(\hat{e}(f_1, \boldsymbol{\lambda}), \dots , \hat{e}(f_S, \boldsymbol{\lambda})$.
For convenience, we abuse notation and write $\qm(\gamma) := \qm(\hat{e}(f_1, \boldsymbol{p} + \gamma \boldsymbol{d}), \dots , \hat{e}(f_S, \boldsymbol{p} + \gamma \boldsymbol{d}))$.

\section{The Algorithm\label{sec:algo}}
It may be observed that the value of the quality metric, $\qm(\gamma)$, depends only on the choices of the most probable translation, $\hat{e}(f_s, \boldsymbol{p} + \gamma \boldsymbol{d})$ for $s = 1 , \dots , S$.
Hence, as a function of $\gamma$, the quality metric is piecewise constant\footnote{In fact, as a function of $\boldsymbol{\lambda}$, $\qm(\hat{e}(f_1, \boldsymbol{\lambda}), \dots , \hat{e}(f_S, \boldsymbol{\lambda}))$ is piecewise constant with linear discontinuities}; so as a step towards maximizing the quality metric, the real line $\mathbb{R}$ is partitioned into intervals $I_0 = (-\infty, \gamma_1), I_1 = (\gamma_1, \gamma_2), \dots , I_{N-1} = (\gamma_{N-1}, \gamma_N), I_N = (\gamma_N, \infty)$ such that $\qm(\gamma)$ is constant on each interval $I_n$.

The natural approach to finding this partition is to consider each most-probable-translation function separately.
For each $s$, the function $\hat{e}(f_s, \boldsymbol{p} + \gamma \boldsymbol{d})$ is piecewise constant.
In fact, observe that the most probable translation given in \eqref{eq:e(f)} is determined by maximizing a sort of score assigned to each candidate translation, $\sum_{m=1}^M \lambda_m \cdot h_m(e,f)$.
This score is linear as a function of $\boldsymbol{\lambda}$.
In particular, in restricting $\boldsymbol{\lambda}$ to a line, $\boldsymbol{\lambda} = \boldsymbol{p} + \gamma \boldsymbol{d}$, this score is itself a line parameterized by $\gamma$.
Thus, the problem of finding the points at which $\hat{e}(f_s, \boldsymbol{p} + \gamma \boldsymbol{d})$ changes reduces to a problem of finding the intersections of these score lines.

So, for each source sentence, $f_s$, a sequence $\gamma_{s,1} < \cdots < \gamma_{s,N_s}$ may be found such that $\hat{e}(f_s, \boldsymbol{p} + \gamma \boldsymbol{d})$ is constant for $\gamma$ in each interval $(-\infty, \gamma_{s,1}), (\gamma_{s,1}, \gamma_{s,2}), \dots , (\gamma_{s,N_s-1}, \gamma_{s,N_s}), (\gamma_{s,N_s}, \infty)$.
It is straightforward to merge these ordered sequences into a single ordered sequence $\gamma_1 < \cdots < \gamma_N$ as required.

The value of the quality metric $\qm(\gamma)$ is then computed at one point in each interval (of course, the choice of point doesn't matter, since $\qm(\gamma)$ is constant on each interval) and the interval producing the best quality metric value is found.
For an arbitrary quality metric function, its value could be computed by scratch at each interval.
However, as quality metrics, the BLEU score, the NIST score, and scores given by error-counts all have certain abelian group-theoretic underpinnings which allow the sequence of their values to be computed more efficiently.

\section{A Useful Property of Score Functions\label{sec:scorefunc}}
Suppose that the function $\qm(e_1, \dots , e_S)$ is the count of the number of errors\footnote{The error-count of course should be minimized, but in practice this can be done by using its negative instead}.
In this case, we can compute the number of errors on each sentence, $\text{Error count}(e_s)$, and we get
\[ \qm(e_1, \dots , e_s) = \text{Error count}(e_1) + \cdots + \text{Error count}(e_S) \]
In the last step of the algorithm described, where the value of the quality metric is determined on each interval, this property may be exploited.

When a point $\gamma_{s, n}$ is found at which $\hat{e}(f_s, \boldsymbol{p} + \gamma \boldsymbol{d})$ changes at this point, it takes almost very little to find additionally the associated change $\Delta \qm_{s, n}$ in error count.
When merging the sequences $\{\gamma_{s, n}\}_{s=1}^{N_s}$ into the single sequence $\{\gamma_n\}_{n=1}^N$, whenever we set $\gamma_n = \gamma_{s_n, m_n}$, we correspondingly set $\Delta \qm_n = \Delta \qm_{s_n, m_n}$.
Then, since the error count for the entire translated corpus is additive in the error counts for each translated sentence, when iterating through the intervals $I_n$ in the last step, we get
\[ \qm(I_{n+1}) = \qm(I_n) + \Delta \qm_n. \]
By this method, the error-count computation at the new interval takes $O(1)$ steps instead of $O(S)$.

The BLEU and NIST scores are not additive like an error-count.
These two stores are similar, and only the BLEU score will be considered in depth here.
The BLEU score is defined on a candidate corpus as follows:
\begin{align*}
\text{BLEU} = & \min(1, e^{1-\frac{\text{best match length}}{\text{candidate corpus length}}}) \cdot \\
&\exp \left( \sum_{n=1}^N \frac{1}{N} \log \left( \frac{\text{\tiny{clipped count of $n$-gram matches}}}{\text{\tiny{number of $n$-grams in candidate corpus}}}\right) \right).
\end{align*}

The BLEU score is a function of a number of variables: the best match length, the candidate corpus length, the clipped count of $n$-gram matches for $n=1, \dots N$, and the total count of $n$-grams in the candidate corpus.
Fortunately, each of these variables is an additive; their value on the entire corpus is equal to the sum of their values on each sentence of the corpus.
The ability to decompose the BLEU score into such variables is precisely the important property that makes quality metric computations efficient.

It is assumed that the only type of quality metric function $\qm(\cdot)$ is one with the following property:
\[ \qm(e_1, \dots , e_S) = f(a(e_1) + \cdots + a(e_n)), \text{where $a(e) \in \mathbb{R}^L$} \]
As just demonstrated, this property holds for the BLEU score and error counts; it also holds for the NIST score.
In this case, let $A(\gamma) = a(\hat{e}(f_1, \boldsymbol{p} + \gamma \boldsymbol{d})) + \cdots + a(\hat{e}(f_S, \boldsymbol{p} + \gamma \boldsymbol{d}))$.
$A(\gamma)$ is constant on each interval $I_n$, for the same reason that $\qm(\gamma)$ is.

In the algorithm, when a point $\gamma_{s, n}$ is found at which $\hat{e}(f_s, \boldsymbol{p} + \gamma \boldsymbol{d})$ changes at this point, we find $\Delta A_{s, n} = a(\hat{e}(f_s, \boldsymbol{p} + \gamma^+ \boldsymbol{d}) - a(\hat{e}(f_s, \boldsymbol{p} + \gamma^- \boldsymbol{d})$, where $\gamma^-$ and $\gamma^+$ are in the intervals immediately to the left and right of $\gamma_{s, n}$ respectively.
Incindentally, so long as $\hat{e}(f_{s'}, \boldsymbol{p} + \gamma \boldsymbol{d})$ do not change at $\gamma_{s, n}$ for $s' \neq s$, then $\Delta A_{s, n} = A(\gamma^+) - A(\gamma^-)$\footnote{In the rare occurrance that $\hat{e}(f_{s})$ does change at $\gamma$ for more than one $s'$, the change in $A$ is the sum of the $\Delta A_{s, n_s}$ over all such $s$}.

When merging the sequences $\{\gamma_{s, n}\}_{s=1}^{N_s}$ into the single sequence $\{\gamma_n\}_{n=1}^N$, whenever we set $\gamma_n = \gamma_{s_n, m_n}$, we correspondingly set $\Delta A_n = \Delta A_{s_n, m_n}$.
Finally, when iterating through all the intervals $I_n$ in the last step, we get
\[ A(I_{n+1}) = A(I_n) + \Delta A_n. \]
and $\qm(I_{n+1})$ is subsequently computed as $f(A(I_{n+1}))$.
Again, this allows the quality metric computation to be reduced from $O(S)$ steps for each new interval to $O(1)$ steps.

\end{document}


\graphicspath{{sec03/images/}{sec03/code/}}
\lstset{inputpath=sec03/code/}
\subsection{Common graphs}
\begin{frame}{Graph example 1\magicPage}\relax
\twocolImg{
\lstinputlisting[linerange={9-10, 16-27}, basicstyle=\tt\tiny]{graphSample.tex}
}{graphSample}

 You can write \ccol{below=of <label>} to have a relative coordinate

\end{frame}

\begin{frame}{Chain example\magicPage}\relax
     \includegraphics[width=\textwidth]{chainSample}

     \skfootnote{\tikzc{I.5}[69]}
\end{frame}

\subsection{Trees}

\begin{frame}{Tree}

\twocolImg{
\lstinputlisting[linerange={9-9, 16-25}, basicstyle=\tt\tiny]{treeSample.tex}
}{treeSample}

We use \ccol\node\ and \ccol{child}.

\ccol{sibling distance} option provides a horizontal distance between nodes

\end{frame}


\section{Distinct Element Method}
Discontinuous systems are characterized by the existence of discontinuities that separate discrete domains within the system. In order to effectively model a discontinuous system, it is necessary to represent two distinct types of mechanical behaviour: the behaviour of the discontinuities and the behaviour of the solid material.

There exists a set of methods, referred to as discrete element methods, which provide the capacity to explicitly represent the behaviour of multiple intersecting discontinuities. The methods allow for the modelling of finite displacements and rotations of discrete bodies, including contact detachment as well as automatic detection of new contacts. Within the set of Discrete Element Methods, there are four subsets \citep{CUNDALL_1992}: Modal Methods, Discontinuous Deformation Analysis Methods, Momentum Exchange Methods, and Distinct Element Methods (DEM). In this paper, the discrete element method used is DEM.

With DEM methods, the discontinuous system is represented as an assembly of deformable blocks such that the interfaces between the blocks represent the discontinuities. With respect to NFR, the blocks can be used to represent the intact rock while the discontinuities represent the joints in the rock mass.

Consider an arbitrary deformable domain, $\Omega$, with a boundary, $\Gamma$, that is subdivided by prescribed discontinuities into $i$ number of subdomains, each denoted by $\Omega_i$ (Figure \ref{fig:DEM}). Let $\Gamma_{ij}=\Gamma_{ji}$ represent the boundary between $\Omega_i$ and $\Omega_j$. The motion of these subdomains (discrete elements) is governed by the conservation of momentum which relates the divergence of the stress field within the element, $\nabla\boldsymbol{\sigma}^m$, to the element acceleration, $\ddot{\mathbf{u}}^m$, and density, $\rho$:

\begin{equation}
\rho \ddot{\mathbf{u}}^m=\nabla\cdot\boldsymbol{\sigma}^m
\label{eqn:cauchy}
\end{equation}


\section{Structuring Elements}

\begin{frame}{Text blocks}
\framesubtitle{In plain, example, and \alert{alert} flavour}
\alert{This text} is highlighted.

\begin{block}{A plain block}
    This is a plain block containing some \alert{highlighted text}.
\end{block}

\begin{exampleblock}{An example block}
    This is an example block containing some \alert{highlighted text}.
\end{exampleblock}

\begin{alertblock}{An alert block}
    This is an alert block containing some \alert{highlighted text}.
\end{alertblock}
\end{frame}

 \begin{frame}[label=proof]{Definitions, theorems, and proofs}
\framesubtitle{All integers divide zero}
\begin{definition}
    $\forall a,b\in\mathbb{Z}: a\mid b\iff\exists c\in\mathbb{Z}:a\cdot c=b$
\end{definition}

\begin{theorem}
    $\forall a\in\mathbb{Z}: a\mid 0$
\end{theorem}

\begin{proof}
    $\forall a\in\mathbb{Z}: a\cdot 0=0$
\end{proof}
\end{frame}


\section{Circuit Design}
	\begin{figure}[ht!]
	\begin{tikzpicture}
		[->,>=stealth',semithick]
		\node[rectangle,draw] (r0) at (0, 0) {Photodiode};
		\node[rectangle,draw,text width=1cm, align=center] (r1) at (-1, -2) {RED LED};
		\node[rectangle,draw,text width=1cm, align=center] (r2) at (1, -2) {IR LED};
		\node[rectangle,draw,text width=1cm, align=center] (r3) at (0, -4) {LED Driver};
		\node[rectangle,draw,text width=3cm, align=center] (r4) at (4, 0) {Transimpedance Amplifier};
		\node[rectangle,draw,text width=2cm, align=center] (r5) at (8, 0) {Difference  Amplifier};
		\node[rectangle,draw,text width=3cm, align=center] (r6) at (8, -2) {Programmable Gain Amplifier};
		\node[rectangle,draw,text width=1cm, align=center] (r7) at (8, -4) {$\mu$C ADC};
		\node[rectangle,draw,text width=1cm, align=center] (r8) at (2, -4) {$\mu$C};
		\node[rectangle,draw,text width=1cm, align=center] (r9) at (5, -2) {$\mu$C};

		\path (r1) edge (r0);
		\path (r2) edge (r0);
		\path (r3) edge (r1)
		edge (r2);
		\path (r0) edge (r4);
		\path (r4) edge (r5);
		\path (r5) edge (r6);
		\path (r6) edge (r7);
		\path (r8) edge (r3);
		\path (r9) edge (r6);

	\end{tikzpicture}
	\caption{System Block Diagram}
	\end{figure}

	Referring the Block diagram, subsections are divided below that show the circuit and corresponding oscilloscope trace outputs.

	\subsection{Front End}

		\subsubsection{Transimpedance Stage}

			Photodiode's current is needed to be converted to a proportional voltage so that ultimately the microcontoller can convert the voltage to a digital value using an ADC for calculation of SpO\textsubscript{2}. A simple Transimpedance Amplifier configuration was used to achieve this:

			\begin{figure}[ht!]\centering
				\begin{circuitikz}[american]
					\draw

					(2,3) node[op amp,yscale=-1] (opamp) {}
					(opamp.down) -- +(0,0.3) node[vcc]{Vcc}
					(opamp.up) -- +(0,-0.3) node[ground]{}
					(0,0) node[ground]{} to[pDo, l^=$D$, f^=$I$]  (0,2.5) to (opamp.-)
					(opamp.+) node[ground]{}
					(opamp.-) -- ++(0,-1.7) -- ++(0.5,0)
					to [R, l^=$R_1$] ++(1.5,0) -- ++(0.4,0) to (opamp.out)
					(opamp.-)  ++(0,-1.7) -- ++(0,-1.3) -- ++(0.5,0)
					to [C, l^=$C_1$]  ++(1.5,0) -- ++(0.4,0) -- ++(0,1.3)
					(opamp.out) -- ++(1,0) to [open,v=$V_1$,o-o] ++(0,-3) node[ground]{};

				\end{circuitikz}
			\end{figure}

			As per led's intensity, if a current of 10$\mu$A is produced in the photodiode, and R = 100K$\Omega$, according to Ohm's law, $V_1$ = 1 V. Capacitor is used for low-pass filtering of the pulse. Cut off frequency would be:

			\begin{equation}
				F_c = \frac{1}{2\pi R_1C_1}
			\end{equation}

			If R\textsubscript{1}=576K$\Omega$, C\textsubscript{1} = 33pF,

			\[
			F_c \approx \SI{8.3}{\kilo\hertz}
			\]

			\begin{figure}[ht!]
				\centering
				\subfloat[V\textsubscript{1}
				]{\includegraphics[width=0.9\textwidth]{../common/circuit/itov.png}}
				\hfill
				\subfloat[V\textsubscript{1} zoomed in
				]{\includegraphics[width=0.9\textwidth]{../common/circuit/itov_zoom.png}
				\label{fig:itov}}
				\caption{V\textsubscript{1} Oscilloscope trace (Red \& IR pulsed sequentially) }
			\end{figure}

			Since LEDs are pulse on-off at a fast rate sequentially the pulse would look like a square wave. We need to use a pulse frequency less than 1/10\textsuperscript{th} of this cut-off to pulse the LEDs so that the square wave with all its high-frequency components is clearly visible and not filtered out.

			\[
			F_p = \SI{500}{\hertz}
			\]

			It can be seen in traces, 2 PPG signals are observable, riding on a DC level. Higher amplitude one is a result of ir led pulsing, while lower amplitude is from red led. Since the peaks correspond to heart rate, in this case as per trace, it measures out to be 1.42/sec or 85BPM. Individual pulses which construct the signal can be seen in Figure \ref{fig:itov}. Amplitude of detected red signal is lower than ir and that is due to:

			\begin{itemize}

				\item Higher intensity of ir led due to higher ir current.
				\item Photodiode's red sensitivity is 80\% of ir sensitivity.
				\item Ir penetrates into skin deeper than red, hence a better ir response is visible at photodiode\cite{penetrate}.

			\end{itemize}
			Hence to obtain a suitable response, it is important to increase led intensity until it reaches a required level so that the PPG signal has a higher amplitude.

		\subsubsection{Difference Amplifier}

			PPG signal needs to be amplified to a suitable level so that full scale ADC range can be utilized to better digitize the signal. Right now, amplitude of PPG signal is quite low and DC level is higher. We are not bothered by the DC anymore, $\mu$C can read it and store for R value calculation, we only need to amplify the AC part which is an envelope of the signal. Direct amplification would lead to gain of the DC level as well, as the required signal is riding on top of it. This will saturate the opamp and amplification level range will be quite low.

			\begin{figure}[ht!]
				\centering
				\includegraphics[width=0.8\textwidth]{../common/circuit/diff.png}
				\caption{Effect of Difference Amplifier}
				\label{fig:diff}
			\end{figure}

			If we can subtract the DC level from entire signal, only AC would remain which can be easily amplified further at higher levels. A Difference amplifier along with a 8bit Digital to analog converter (DAC) generating the DC cancellation signal can be used here. Since red and ir have different levels, separate DC levels are generated by DAC in sync with the pulsed frequency.\medskip

			As shown in Figure \ref{fig:diff} a plot generated in Octave demonstrates subtraction of required signal and diff signal. This difference can then be amplified further. Pulse width of diff signal is slightly larger than required signal so that there are no spikes on edges in output signal. Also the lower level of diff signal is slightly above 0V to allow the difference to always be negative in low states and since negative supply is grounded, output would be 0V here.

			\begin{figure}[ht!]\centering
				\begin{circuitikz}[american]
					\draw

					(4,3) node[op amp,yscale=-1] (opamp) {}
					(opamp.down) -- +(0,0.3) node[vcc]{Vcc}
					(opamp.up) -- +(0,-0.3) node[ground]{}
					(opamp.-) to [R, l^=$R_1$] ++(-2,0) node[label={left:$V_{dac1}$}] {}
					(opamp.+) to [R, l^=$R_1$] ++(-2,0) node[label={left:$V_1$}] {}
					(opamp.+) to [R, l^=$R_1$] ++(0,2) node[ground,rotate = 90]{}
					(opamp.-) -- ++(0,-1.7) -- ++(0.5,0)
					to [R, l^=$R_1$] ++(1.5,0) -- ++(0.4,0) to (opamp.out)
					(opamp.out) -- ++(1,0) to [open,v=$V_2$,o-o] ++(0,-3) node[ground]{};

				\end{circuitikz}
			\end{figure}

			\[
				V_2 = \frac{R_1}{R_1}(V_1 - V_{dac1})
			\]

			For $R_1 = 10K$,

			\[
				V_2 \approx V_1 - V_{dac1}
			\]

		\subsubsection{Programmable Gain Amplifier}

			It becomes necessary to amplify the signal as skin tone, improper finger contact with sensor or any skin pigmentation\cite{skin} can cause attenuation in signal leading to low amplitudes. This stage provides selectable variable gain which can be set by microcontroller.

			$\mu$C would determine what level of further amplification is required and drive the respective pin high so that resistor value would get selected.

			\begin{figure}[ht!]\centering
				\begin{circuitikz}[american]
					\draw
						(2,3) node[op amp,yscale=-1] (opamp) {}
						(opamp.down) -- +(0,0.3) node[vcc]{Vcc}
						(opamp.up) -- +(0,-0.3) node[ground]{}
						(opamp.+) to ++(-2,0) node[label={left:$V_2$}] {}
						(opamp.-) -- ++(0,-1.7) -- ++(1,0)
						to [R, l^=$10K$] ++(1,0) -- ++(1,0) -- ++(0,2.2) to (opamp.out)
						(opamp.-) ++(0,-1.7) -- ++(0,-1.5) to [R, l^=$10K$] ++(0,-1) -- ++(0,-0.5)
						++(0,-0.5) node[npn] (Q1){}
						(Q1.E) node[ground]{}
						(Q1.B) to [R, l^=$4.7K$] ++(-1.5,0) node[label={left:S1}] {}

						(opamp.-) ++(0,-1.7) ++(0,-0.75) -- ++(3.5,0) -- ++(0,-0.75)
						to [R, l^=$4.7K$] ++(0,-1) -- ++(0,-0.5)
						++(0,-0.5) node[npn] (Q2){}
						(Q2.E) node[ground]{}
						(Q2.B) to [R, l^=$4.7K$] ++(-1.5,0) node[label={left:S2}] {}

						(opamp.-) ++(0,-1.7) ++(0,-0.75) ++(2,0) -- ++(5,0) -- ++(0,-0.75)
						to [R, l^=$2.35K$] ++(0,-1) -- ++(0,-0.5)
						++(0,-0.5) node[npn] (Q3){}
						(Q3.E) node[ground]{}
						(Q3.B) to [R, l^=$4.7K$] ++(-1.5,0) node[label={left:S3}] {};

				 \end{circuitikz}
			\end{figure}

			\begin{center}
				\begin{tabular}{ |c|c|}
					\hline
					\textbf{Switch} & \textbf{Gain}   \\
					\hline
					all off & 1 	\\
					\hline
					S1 & 2  		\\
					\hline
					S2 & 3  		\\
					\hline
					S1+S2 & 4  		\\
					\hline
					S3 & 5  		\\
					\hline
					S1+S3 & 6	  	\\
					\hline
					S2+S3 & 7  		\\
					\hline
				\end{tabular}
			\end{center}

	\subsection{LED Driver}

		As discussed previously, the detected signal levels can be very low after I to V stage. LED Driver needs to increase current to the leds as required. Following circuit is used as a voltage controlled current source, voltage is set by the DAC as instructed by the $\mu$C.

		\begin{figure}[ht!]\centering
			\begin{circuitikz}[american]
				\draw

				(4,3) node[npn] (Q1){}
				(Q1.E) to [R, l^=20] ++(0,-1.5) node[ground]{}
				(Q1.B) to [R, l^=4.7K] ++(-1.5,0) node[label={left:$V_{dac2}$}] {}
				(Q1.C) -- ++(-1,0) -- ++(0,0.5) ++(0,0.5)  node[npn] (Q2){}
				(Q2.B) to [R, l_=10K] ++(-1.5,0) node[label={left:Red pulse}] {}
				(Q2.C) -- ++(0,1) node[]{} ++(0,0.2) node[]{Red cathode}
				(Q1.C) -- ++(1,0) -- ++(0,0.5) ++(0,0.5) node[npn,xscale=-1] (Q3){}
				(Q3.B) to [R, l^=10K] ++(1.5,0) node[label={right:IR pulse}] {}
				(Q3.C) to ++(0,1) node[]{} ++(0,0.2) node[]{Ir cathode};
			\end{circuitikz}
		\end{figure}

		Leds get activated by a single bridge configuration, whenever red/ir pulse is active, corresponding red/ir leds get active (led anodes connected to Vcc). Amount of current would depend on $V_{dac2}$ \& $R_1$.

		$V_{dac2}$ is also set individually for red and ir as per the pulsed frequency. If intensity is not sufficient for signal detection, it is necessary to increase $V_{dac2}$. $\mu$C would determine the thresholds and modulate the control voltage accordingly.

		Shown below is final output after the PGA stage.
		The gained up PPG signal is clearly visible for both red and ir which is at a suitable level for ADC capture and usage in algorithms for calculations.

	\begin{figure}[ht!]
		\centering
		\subfloat[Final output - green, V\textsubscript{1} - blue
		]{\includegraphics[width=1\textwidth]{../common/circuit/final_out.png}}
		\hfill
		\subfloat[Final output measurement
		]{\includegraphics[width=1\textwidth]{../common/circuit/final_out_meas.png}}
		\caption{Final output}
	\end{figure}


% corrected LN 96
\onecolumn
\newpage
\clearpage
\section{Appendix}
\textbf{Training on English dataset:}

% train on 13311 samples and testing on 4438 samples

% \begin{enumerate}[label=\arabic*.]
%   \item Feedforward:
    \begin{figure}[H]
      \centering
      \subfloat[Feedforward's accuracy using SGD.]{{\includegraphics[scale=0.5]{sgdffa.png} }}%
      \qquad
      \subfloat[Feedforward's accuracy using Adam.]{{\includegraphics[scale=0.5]{adamffa.png} }}%
      \caption{Accuracy comparison between feedforward using SGD and Adam.}%
      \label{ffa}
    \end{figure}
    \begin{figure}[H]
      \centering
      \subfloat[Feedforward's loss using SGD.]{{\includegraphics[scale=0.5]{sgdffl.png} }}%
      \qquad
      \subfloat[Feedforward's loss using Adam.]{{\includegraphics[scale=0.5]{adamffl.png} }}%
      \caption{Loss comparison between feedforward using SGD and Adam.}%
      \label{ffl}
    \end{figure}
  % \item RNN:
    \begin{figure}[H]
      \centering
      \subfloat[RNN's accuracy using SGD.]{{\includegraphics[scale=0.5]{sgdrnna.png} }}%
      \qquad
      \subfloat[RNN's accuracy using Adam.]{{\includegraphics[scale=0.5]{adamrnna.png} }}%
      \caption{Accuracy comparison between RNN using SGD and Adam.}%
      \label{rnna}
    \end{figure}
    \begin{figure}[h]
      \centering
      \subfloat[RNN's loss using SGD.]{{\includegraphics[scale=0.5]{sgdrnnl.png} }}%
      \qquad
      \subfloat[RNN's loss using Adam.]{{\includegraphics[scale=0.5]{adamrnnl.png} }}%
      \caption{Loss comparison between RNN using SGD and Adam.}%
      \label{rnnl}
    \end{figure}
    \begin{figure}[h]
      \centering
      \includegraphics[scale=0.4]{300rnna.png}
      \caption{Training and validation accuracy of RNN model.}
      \label{300rnna}
    \end{figure}

    \begin{figure}[h]
      \centering
      \includegraphics[scale=0.3]{300rnnl.png}
      \caption{Training and validation loss of RNN model.}
      \label{300rnnl}
    \end{figure}
  % \item LSTM:
    \begin{figure}[H]
      \centering
      \subfloat[LSTM's accuracy using SGD.]{{\includegraphics[scale=0.5]{sgdlstma.png} }}%
      \qquad
      \subfloat[LSTM's accuracy using Adam.]{{\includegraphics[scale=0.5]{adamlstma.png} }}%
      \caption{Accuracy comparison between LSTM using SGD and Adam.}%
      \label{lstma}
    \end{figure}
    \begin{figure}[H]
      \centering
      \subfloat[LSTM's loss using SGD.]{{\includegraphics[scale=0.5]{sgdlstml.png} }}%
      \qquad
      \subfloat[LSTM's loss using Adam.]{{\includegraphics[scale=0.5]{adamlstml.png} }}%
      \caption{Loss comparison between LSTM using SGD and Adam.}%
      \label{lstml}
    \end{figure}
  % \item GRU:
    \begin{figure}[H]
      \centering
      \subfloat[GRU's accuracy using SGD.]{{\includegraphics[scale=0.5]{sgdgrua.png} }}%
      \qquad
      \subfloat[GRU's accuracy using Adam.]{{\includegraphics[scale=0.5]{adamgrua.png} }}%
      \caption{Accuracy comparison between GRU using SGD and Adam.}%
      \label{grua}
    \end{figure}
    \begin{figure}[H]
      \centering
      \subfloat[GRU's loss using SGD.]{{\includegraphics[scale=0.5]{sgdgrul.png} }}%
      \qquad
      \subfloat[GRU's loss using Adam.]{{\includegraphics[scale=0.5]{adamgrul.png} }}%
      \caption{Loss comparison between GRU using SGD and Adam.}%
      \label{grul}
    \end{figure}
% \end{enumerate}

\textbf{Training on Luxembourghish dataset.}\\
\begin{figure}[h]
  \centering
  \includegraphics[scale=0.5]{luxffa.png}
  \caption{Training and validation accuracy of feedforward model trained on
  Luxembourgish dataset.}
  \label{luxffa}
\end{figure}

\begin{figure}[h]
  \centering
  \includegraphics[scale=0.5]{luxffl.png}
  \caption{Training and validation loss of feedforward model trained on
  Luxembourgish dataset.}
  \label{luxffl}
\end{figure}

\begin{figure}[h]
  \centering
  \includegraphics[scale=0.5]{luxrnna.png}
  \caption{Training and validation accuracy of RNN model trained on
  Luxembourgish dataset.}
  \label{luxrnna}
\end{figure}

\begin{figure}[h]
  \centering
  \includegraphics[scale=0.5]{luxrnnl.png}
  \caption{Training and validation loss of RNN model trained on Luxembourgish
  dataset.}
  \label{luxrnnl}
\end{figure}

\begin{figure}[h]
  \centering
  \includegraphics[scale=0.5]{luxlstma.png}
  \caption{Training and validation accuracy of LSTM model trained on
  Luxembourgish dataset.}
  \label{luxlstma}
\end{figure}

\begin{figure}[h]
  \centering
  \includegraphics[scale=0.5]{luxlstml.png}
  \caption{Training and validation loss of LSTM model trained on Luxembourgish
  dadataset.}
  \label{luxlstml}
\end{figure}

\begin{figure}[h]
  \centering
  \includegraphics[scale=0.5]{luxgrua.png}
  \caption{Training and validation accuracy of GRU model trained on Luxembourgish
  dadataset.}
  \label{luxgrua}
\end{figure}

\begin{figure}[h]
  \centering
  \includegraphics[scale=0.5]{luxgrul.png}
  \caption{Training and validation loss of GRU model trained on Luxembourgish
  dadataset.}
  \label{luxgrul}
\end{figure}

\begin{figure}[h]
  \centering
  \lstinputlisting{sections/scientific/fr1/mfccsnippet.py}
  \caption{Code snippet to extract MFCCs from the dataset.}
  \label{mfccsnip}
\end{figure}


\documentclass[simplex.tex]{subfiles}
% NO NEED TO INPUT PREAMBLES HERE
% packages are inherited; you can compile this on its own
\begin{document}
\subsection{Robust Law of Large Graphs}

In order to see a bias-variance tradeoff phenomenon with respect to the parameter $q$ in the ML$q$E estimator, we change our simulation setting as following:
We consider the 2-block SBM with respect to the exponential distributions parameterized by
\begin{equation*}
B = \begin{bmatrix}
4 & 2 \\
2 & 7
\end{bmatrix}
,\qquad \rho = \begin{bmatrix}
0.5 & 0.5
\end{bmatrix}.
\end{equation*}
And let the contamination also be a 2-block SBM with the same structure parameterized by
\begin{equation*}
B^{\prime} = \begin{bmatrix}
9 & 6 \\
6 & 13
\end{bmatrix}
,\qquad \rho = \begin{bmatrix}
0.5 & 0.5
\end{bmatrix}.
\end{equation*}

Figure~\ref{fig:q} shows the mean squared error in average by varying the parameter $q$ in ML$q$E with fixed $n = 100$, $m = 20$ and $\epsilon = 0.1$ based on 1000 Monte Carlo replicates. Different types of lines represent the simulated MSE associated with four different estimators. From the figure, we can see that the ASE procedure takes advantage of the graph structure and improves the performance of the corresponding estimators independent of the selection of $q$. Moreover, within a proper range of $q$, the ML$q$E wins the bias-variance tradeoff and shows the robustness property compare to the MLE. And as $q$ goes to 1, ML$q$E goes to the MLE as expected.

\begin{figure}[!htb]
\centering
\includegraphics[width=0.7\textwidth]{../../figs/sim_q.pdf}
\caption{Mean squared error in average by varying the parameter $q$ in ML$q$E with fixed $n = 100$, $m = 20$ and $\epsilon = 0.1$ based on 1000 Monte Carlo replicates. Different types of lines represent the simulated MSE associated with four different estimators.
1. ASE procedure takes advantage of the graph structure and improves the performance of the corresponding estimators independent of the selection of $q$;
2. Within a proper range of $q$, ML$q$E wins the bias-variance tradeoff and shows the robustness property compare to the MLE. Also as $q$ goes to 1, ML$q$E goes to the MLE as expected.}
\label{fig:q}
\end{figure}

\clearpage
\end{document}


\section{Comparing Shapley Effects and Morris Indices} \label{comparison}

In this section, I want to connect and further discuss the results presented in \cref{shapley_rust_model,morris_rust_model}. Both, Shapley effects and independent Morris indices, succeed in identifying the correct input ranking, even for a relatively small sample size.

Interestingly, in the case of two inputs and when setting $N_I = 3$ as recommended
by \citet{SNS16}, the computational burden of both algorithms are parallel, given that $N = N_O$. Then they differ by $N_V$, the number of Monte Carlo simulations used to estimate the total output variance, only. One practical drawback
of the Shapley effects as in the implementation of \citet{SNS16} is that more than one sample size needs to be set. For applying the Morris method, we need to specify only one sample size.

With respect to the estimation precision, Morris indices are subject to larger uncertainty
than Shapley effects. As argued above, this estimation precision is not necessarily needed: If
the identification of uninfluential inputs is the objective of sensitivity analysis, estimates need to be precise
only in the sense that they yield nonzero indices if an input is important and zero if they
are not influential. If input ranking is the goal, the situation is slightly different: Estimates need to reliably rank the inputs.
% The simulation study presented in this paper with two inputs only may not be the right trial to investigate this.
In the case of the Rust model, a ranking based on the full Morris indices can be misleading since estimates are very volatile.

With respect to convergence, I find that Shapley effects converge even for small sample sizes. While the same is true for independent Morris indices, the convergence of full Morris indices is ambiguous. For the sample sizes I applied, convergence was not achieved. Since the convergence behaviour is unclear definite conclusions cannot be drawn here.

Since the estimation of Shapley effects is more costly and acknowledging that Morris
indices can yield correct input rankings for the Rust model when interpreted cautiously, we can resort to using the Morris
method for input importance ranking. Morris indices for the Rust model imply that the influence of $RC$ is due to its dependence on $\theta_{11}$ only. Both Shapley effects and Morris indices imply that more effort should be exerted into reducing uncertainty in $\theta_{11}$ if the derivation of a more stable model output is the goal.

Further, Morris indices are informative about the model structure, i.e. whether there exist large interaction effects among inputs. Hence, we cannot only successfully rank the inputs but also learn more about the model itself.

Since Shapley effects yield only one sensitivity index per input, they are easier to interpret than the four indices in the case of the Morris method. By having these four indices, the Morris method is applicable in more settings, as discussed in \cref{var_based_sa,screening}.

My findings are broadly in line with the ones found in \citet{GM17} with the restriction that under small sample sizes full indices should not be used in isolation to derive the input ranking. % The extension of the Morris method to dependent inputs is an important contribution.
Since full Morris indices in isolation do not suffice to provide a ranking of each input variable, the extension to dependent inputs allows to extract this information easily. Depending on the objective of sensitivity analysis, the additional information can be of great importance in assessing the validity of assumptions.


\documentclass[12pt]{cdblatex}
\usepackage{exercises}
\usepackage{fancyhdr}
\usepackage{footer}

\begin{document}

% --------------------------------------------------------------------------------------------
\section*{Exercise 1.7 Subtleties of {\tt foo = bah} and {\tt foo := @(bah)}}

\begin{cadabra}
   {a,b,c,d,e,f,h#}::Indices.

   foo := B_{b} A_{a}.
   bah := A_{a} C_{c}.

   # cdbBeg(print.0107)
   print("foo = "+str(foo))
   print("bah = "+str(bah)+"\n")

   print("type foo = "+str(type(foo)))
   print("type bah = "+str(type(bah))+"\n")

   print("id foo = "+str(id(foo)))
   print("id bah = "+str(id(bah))+"\n")

   bah = foo

   print("foo = "+str(foo))
   print("bah = "+str(bah)+"\n")

   sort_product (foo)

   print("bah = "+str(bah)+"\n")

   print("id foo = "+str(id(foo)))
   print("id bah = "+str(id(bah))+"\n")

   bah := @(foo).

   print("id foo = "+str(id(foo)))
   print("id bah = "+str(id(bah))+"\n")
   # cdbEnd(print.0107)
\end{cadabra}

\clearpage

\IfFileExists{ex-0107.cdbcopy}%
{\lstinputlisting[backgroundcolor=\color{white}]{ex-0107.cdbcopy}}%
{Where is {\tt ex-0107.cdbcopy}?}

\vskip 1cm
Note that the line numbers referenced in the following are those of the output above not
those of the Cadabra source.

\begin{itemize}
   \item Lines 7 and 8 show that the objects \verb|foo| and \verb|bah| point to distinct
         areas of memeory (i.e., they point to different objects).
   \item Lines 10 and 11 show the result of the statement \verb|bah = foo|.
   \item Line 13 shows that \verb|bah| has changed after the statement
         \verb|sort_product (foo)|.
   \item Lines 15 and 16 verifies that \verb|foo| and \verb|bah| point to the same object
         (so changes in \verb|foo| will be seen by \verb|bah|, as just noted).
   \item Lines 18 and 19 shows that after \verb|bah := @(foo)| the symbols \verb|bah| and
         \verb|foo| no longer point to the same object.
\end{itemize}

\end{document}


\documentclass[11pt, a4paper]{article}

\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{csquotes}
\usepackage{listings}
\usepackage{booktabs}

\usepackage{color}

\definecolor{pblue}{rgb}{0.13,0.13,1}
\definecolor{pgreen}{rgb}{0,0.5,0}
\definecolor{pred}{rgb}{0.9,0,0}
\definecolor{pgrey}{rgb}{0.46,0.45,0.48}

\lstset{language=Java,
  showspaces=false,
  showtabs=false,
  breaklines=true,
  frame=single,
  showstringspaces=false,
  breakatwhitespace=true,
  commentstyle=\color{pgreen},
  keywordstyle=\color{pblue},
  morekeywords={assert},
  stringstyle=\color{pred},
  basicstyle=\ttfamily\footnotesize
}

\title{SINF2224 --- Assignment 3}
\author{Florian Thuin \and Syméon Malengreau}
\date{\today}

\begin{document}
    \maketitle

    \section{Analytical expression of the number of different test cases}

    The analyze of the function \verb#concurrentTransactions#$(N_t, N_a, M_i, M_t)$
    to find the number of states $N_s$ can be separated in two parts:

    \begin{itemize}
        \item The first \verb#while# loop can create $N_a$ accounts with a
        random value between 1 and $M_i$, such that:
            $$N_s = {M_i}^{N_a}$$
        \item The second \verb#while# loop can create from two random accounts
        (identified by their unique id from 1 to $N_a$) a random tranfer from 1
        to $M_t$
            $$N_s = {(N_a \times N_a \times M_t)}^{N_t}$$
    \end{itemize}

    The result for the whole function can be expressed as:

    $$N_s = {M_i}^{N_a} \times {(N_a \times N_a \times M_t)}^{N_t}$$

    \section{Experiments}

    Those experiments were made in Salle Intel, the computers are becoming really
    old and we can't remove all other processes, such that we had very poor results
    and we decided to use the modified test harness (see the next section).
    It just removes the non-relevant states so it won't affect the
    relation with the analytical expression. You can reproduce them by
    running \verb#test.sh#.

    \begin{tabular}{cccccc}
        \toprule
        $(N_t, N_a, M_i, M_t)$ & New & Visited & End & Time (mm:ss) & Memory \\
        \toprule
        (2,2,2,2)            & 15393 & 11031  & 1718 & 00:07 & 166MB \\
        \midrule
        (3,2,2,2)           & 397703 & 320454 & 12860 & 01:27 & 377MB \\
        \midrule
        (2,3,2,2)           & 254696 & 206144 & 26966 & 00:45 & 166MB \\
        (2,4,2,2)           & 2336533 & 2006172 & 236338 & 06:45 & 515MB \\
        \midrule
        (2,2,3,2)           & 33855 & 24429 & 3728 & 00:08 & 166MB \\
        (2,2,4,2)           & 59325 & 43007 & 6462 & 00:12 & 166MB \\
        (2,2,5,2)           & 91803 & 66765 & 9920 & 00:17 & 166MB \\
        (2,2,6,2)           & 131289 & 95703 & 14102 & 00:25 & 356MB \\
        (2,2,7,2)           & 177783 & 129821 & 19008 & 00:32 & 377MB \\
        (2,2,8,2)           & 231285 & 169119 & 24638 & 00:40 & 377MB \\
        (2,2,9,2)           & 291795 & 213597 & 30992 & 00:51 & 356MB \\
        (2,2,10,2)          & 359313 & 263255 & 38070 & 01:02 & 288MB \\
        \midrule
        (2,2,2,3)           & 29190 & 23024 & 3067 & 00:07 & 166MB \\
        (2,2,2,4)           & 46587 & 38805 & 4736 & 00:10 & 288MB \\
        (2,2,2,5)           & 68537 & 59060 & 6755 & 00:14 & 190MB \\
        (2,2,2,6)           & 94424 & 83270 & 9100 & 00:20 & 377MB \\
        (2,2,2,7)           & 124853 & 111944 & 11787 & 00:25 & 377MB \\
        (2,2,2,8)           & 159150 & 144498 & 14798 & 00:31 & 288MB \\
        (2,2,2,9)           & 198063 & 181590 & 18151 & 00:39 & 247MB \\
        (2,2,2,10)          & 240783 & 222509 & 21834 & 00:47 & 377MB \\
        \bottomrule
    \end{tabular}
    \bigskip

    So as we can see in this table, the exponent factors in the analytic
    formula really play an important role in the state-space explosion. We can
    conclude that, even if it is not a perfect formula (because many other
    factors come from what Java does internally and how JPF handles it),
    it globally reflects where will be the limits of the proofs.

    \section{Modified test harness}

    We can delete the printf inside the tests method to reduce the number of
    states. We can also delete all unused variables (that were needed by
    those printf). Only by doing that, with (2, 2, 2, 2) as parameters, the
    number of visited states decreases from 57835 to 11031 and the computation
    time is already divided by 5. \newline

    This is a very easy improvement, there might be many others directly
    related to the program (as not trying to transfer from an account that
    already has a balance of 0 to withdraw 0 and deposit 0) but we gave a try
    to them and it only increased the number of states.

    \section{Ensures balance never becomes negative}

    First, we have to be sure that the initialization never sets a negative
    balance by adding this line in the constructor:

    \begin{lstlisting}
assert 0 <= initialAmount;
    \end{lstlisting}

    We also have to add it to the methods that modifies the balance, such as
    \verb#withdraw# and \verb#deposit#:

    \begin{lstlisting}
private void withdraw(final int amount) {
    assert balance - amount >= 0;
    balance = balance - amount;
}
private void deposit(final int amount) {
    assert balance + amount >= 0;
    balance = balance + amount;
}
    \end{lstlisting}

    \section{Verifying merge method}

    To verify \verb#merge# method, we can create more threads in the
    \verb#concurrentTransactions# that will call it (concurrently from the ones
    that do the \verb#transferTo#), we don't really care about the number of
    merge (because it is not a parameter) so you can pay no attention to that.
    Here is what we can write:

    \begin{lstlisting}
i = 0;
while (i != nbrOfTransfer) {
    final Thread t = new Thread() {
        @Override
        public void run() {
            final Account a1 = accounts[random.nextInt(accounts.length)];
            final Account a2 = accounts[random.nextInt(accounts.length)];
            if (a1.id() != a2.id()) a1.merge(a2);
        }
    };
    t.start();
    i = i + 1;
}
    \end{lstlisting}

    But as JPF will output, we just created a deadlock\ldots So let's add some
    code to the merge method:

    \begin{lstlisting}
synchronized (getFirstSync(other)) {
    synchronized (getSecondSync(other)) {
        this.balance = this.balance + other.balance;
        other.balance = 0;
    }
}
    \end{lstlisting}

    Now everything works fine for this method.

    \section{Program errors found}

    In this section, we explain the way we corrected the program following the
    warning and errors given by running JPF on TestHarness. \newline

    The first program error was a \textbf{computation mistake} in \verb#transferTo#,
    such that balance could have been negative and the assertion would not
    be true when the amount to transfer is less than the account balance
    (JPF tells us it is a \verb#java.lang.AssertionError#):

    \begin{lstlisting}[language=Java]
withdraw(amount);
other.deposit(maxAmount);
    \end{lstlisting}

    has to become:
    \begin{lstlisting}[language=Java]
withdraw(maxAmount);
other.deposit(maxAmount);
    \end{lstlisting}

    The second program error was a \textbf{missing synchronization} on the access of the
    field \verb#balance# in the method \verb#balance()# which resulted in an
    \textit{unprotected field access}:

    \begin{scriptsize}
    \begin{verbatim}
[SEVERE] unprotected field access of: bank.Account@272.balance in thread: Thread-1 (bank/Account.java:49)
    \end{verbatim}
    \end{scriptsize}

    To correct that, we just have to synchronize it:

    \begin{lstlisting}[language=Java]
public int balance() {
    synchronized (this) {
        return balance;
    }
}
    \end{lstlisting}

    But correcting this error leads to create a deadlock (in fact, JPF now
    finds it) and there is still an assertion failure. \newline

    The third program error is that the \textbf{assertion fails} because the assignment
    of \verb#initialSum# is outside the synchronized part such that it can still
    change before the synchronisation. We have to move this line inside the scope
    of the synchronisation. \newline

    \begin{lstlisting}[language=Java]
final int initialSum = this.balance() + other.balance();
    \end{lstlisting}

    The fourth program error was a \textbf{synchronization} mistake, as we saw
    in the lab session with the dining philosophers problem, we have to lock
    and release the resources in a way that doesn't create deadlock (use the
    same order to lock and then release in the reverse order). Here, we can
    use the unique id of each account to lock first the account with the lower
    id and then the account with the greatest id. \newline

    \begin{lstlisting}[language=Java]
synchronized (getFirstSync(other)) {
    synchronized (getSecondSync(other)) {
        ...
    }
}
private Account getFirstSync(Account other) {
    if (this.id() < other.id()) return this;
        else return other;
}
private Account getSecondSync(Account other) {
    if (this.id() > other.id()) return this;
        else return other;
    }
    \end{lstlisting}

    \section{Conclusion}

    We didn't find JPF to be very very verbose because the errors returned were
    usually rather vague, but we tried our best to show that we understood how
    JPF works and that it helped us to submit a program that is errors-free.
    It was fun to try a program that is able to understand when two states
    of the program are equals and that can detect errors with threads.

\end{document}


\documentclass{article}

% the purpose of this particular document is to serve as a staging area potential blog posts, for the purpose of editing, revision, and review.

% packages
	\usepackage{amsmath, amsthm, mathtools}
	\usepackage{mdframed}
	\usepackage{geometry, enumerate}
	\usepackage{parskip}
	\usepackage{graphicx}
	\usepackage{dsfont} % for math bf text

% theorems and such
  	\newtheorem{theorem}{Theorem}
  	\newtheorem{corollary}{Corollary}
  	\newtheorem{lemma}[theorem]{Lemma}
  	\newtheorem*{remark}{Remark}
  	\newtheorem*{exe}{Exercise}
  	\newtheorem{prop}{Proposition}
  	\newtheorem{example}{Example}
  	\newtheorem{definition}{Definition}

% custom commands
	\newcommand{\ceil}[1]{\left \lceil #1 \right \rceil}
	\newcommand{\floor}[1]{\lfloor #1 \rfloor}
	\newcommand{\X}[1]{\, \text{mod} \, #1}
	\newcommand{\divv}{\,|\,}
	\newcommand{\GCD}[2]{GCD\,(#1, #2)}
	\newcommand{\LCM}[2]{LCM\,(#1, #2)}

\begin{document}

\title{Elementary Number Theory : \S 3}
\author{Henry Slayer $|$ University of California, Santa Cruz}
\date{}
\maketitle
\section*{Primes and Prime Decomposition}
Primes are often referred to as the atoms of the integers, and the analogy could be no less fitting. All integers and rational numbers can be decomposed into primes, and just as a molecule is defined by its distinct atomic signature, the prime decomposition belonging to each number is unique. Some of this may be familiar from previous courses, or even from high school. We develop existence and uniqueness of prime decomposition with the following collection of facts and observations.
\begin{mdframed}
\begin{definition}
A prime number is a natural number $p$ such that (i) $p$ is not zero or 1, and (ii), the only divisors of $p$ are 1 and itself.
\end{definition}
\begin{prop}
Every nonzero integer has finitely many divisors.
\end{prop}
\begin{proof}
For the proof, we consider divisors in terms of absolute value. Suppose that we have two numbers, $a$ and $b$. If $a\divv b$, then $a\leq b$. As the greatest divisor of any number is itself, it follows that any and all other divisors must be smaller than the number itself. So, there must be finitely many such divisors.
\end{proof}
\begin{prop}
Every nonzero number that isn't 1 has a prime factorization.
\end{prop}
\begin{proof}
We can consider positive integers not equal to zero or 1, as we are considering divisibility primarily in terms of absolute value. Walking through the numbers, we can see that every number we encounter is either prime (2, 3, 5, \dots), or composite. When we arrive at a prime number, we know by the definition of a prime that it is its own decomposition. When we arrive at a composite number, we stop, and break that number down into composite factors, dividing and dividing and dividing our divisors and dividends until we have found the elements that can no longer be divided, which are primes. We know that we'll eventually bottom out, because each number has finitely many divisors, and we know that all of these numbers are prime, because we've divided as much as we possibly can. So, every integer is itself prime, or the product of primes.
\end{proof}
\end{mdframed}
\begin{mdframed}
\begin{prop}
Let $N$ be a natural number greater than 1. If $N$ is composite, then $N$ has a prime factor $p \leq \floor{\sqrt{N}}$.
\end{prop}
\begin{proof}
Assume that $N$ is composite. So, $N=ab$ for a pair of positive integers $a$ and $b$, where both $a$ and $b$ are greater than 1. We claim that $a\leq \floor{\sqrt{N}}$ or $b \leq \floor{\sqrt{N}}$. Suppose, to the contrary, that this wasn't the case. Then, $ a \geq b > \floor{\sqrt{N}}$, which would imply that $ab > N$, which is impossible. Now, as $a > 1$, $a$ must have some prime factor $p$, such that $p\leq a\leq \floor{\sqrt{N}}$. As $p\divv a$, and $a\divv N$, it follows that $p\divv N$, and so $N$ must have a prime factor $p \leq \floor{\sqrt{N}}$.
\end{proof}
\end{mdframed}
Now, try to convince yourself why any integer $N$ not equal to zero or one cannot have a prime factor \textit{larger} than $\floor{\sqrt{|N|}}$, unless $N$ itself is prime. \\
Having established that every number \textit{has} a factorization in terms of primes, we now want to investigate whether or not this factorization is unique.
\begin{mdframed}
\begin{lemma}[Euclid's Lemma] Suppose that $a, b, c$ are integers, and $\GCD{a}{b} = 1$. Then, if $a\divv bc$, it follows that $a\divv c$.
\end{lemma}
\begin{proof} As the $GCD$ of $a$ and $b$ is 1, we know from the previous section that $ax+by = 1$, for some choice of integers $x$ and $y$. Scaling by $c$,
\[(ac)x + (bc)y = c\]
As $a\divv acx$ and $a\divv bcy$ (by assumption), we know that $a$ must divide $c$, too.
\end{proof}
\begin{lemma}[Euclid's Lemma, now for primes] Let $p, c, b$ be integers, where $p$ is a prime number. Then, if $p\divv bc$, $p\divv b$ or $p\divv c$ (or both.
\end{lemma}
\begin{proof}
Suppose that $p\divv b$. Then we're done. If $p\not\divv b$, then $p$ and $b$ must be coprime, and so $p\divv c$ by Euclid's lemma.
\end{proof}
\end{mdframed}
The lemmas above suggest an important observation. If a prime number divides any product of integers, say $p\divv a_1a_2a_3\cdots a_n$, then $p\divv a_i$ for some $1\leq i \leq n$. This makes sense. If this weren't the case, then somehow parts of $p$ would need to be distributed across one or more integers. However, numbers can't share fractions of a prime between each other, because a prime can't be split into such factors. \\
Having gathered these observations, we are ready to prove the uniqueness of prime decomposition. From here forwards, we will typically denote the prime factorization of a number as $a = 2^{e_2}3^{e_3}\cdots$, where each $e_i$ is the exponent of a prime factor.
\begin{mdframed}
\begin{theorem}[Uniqueness of Prime Decomposition]
Each number has a unique factorization into primes. That is, if $e_i$ and $f_i$ are natural numbers, and
\[ N = 2^{e_2}3^{e_3}5^{e_5}\cdots p^{e_p} = 2^{f_2}3^{f_3}5^{f_5}\cdots p^{f_p}\]
Then $e_p = f_p$ for each prime factor $p$.
\end{theorem}
\begin{proof}
Suppose that $N$ is some integer, with two prime decompositions.
\[N = 2^{e_2}3^{e_3}5^{e_5}\cdots p^{e_p} = 2^{f_2}3^{f_3}5^{f_5}\cdots p^{f_p}\]
We want to show that the exponents are equal, across both factorizations. For any given pair of exponents $e_p, f_p$, either $e_p \leq f_p$ or $f_p \leq e_p$. So, we'll assume without loss of generality that $e_p \leq f_p$, and we let $d$ denote the difference, $d = f_p - e_p$. So, what happens if we divide $N$ by $p^{e_p}$?
\[\frac{N}{p^{e_p}} = 2^{e_2}3^{e_3}5^{e_5}\cdots 1 = 2^{e_2}3^{e_3}5^{e_5}\cdots p^{d}\]
We claim that $d$ must be zero. Why is this the case? \\
If it weren't the case that $d$ was zero, then $p$ would divide the product on the right hand side. If $p$ divided the product on the right hand side, then it would also need to divide a product on the left. But, the left has no factors of $p$, because they've already been divided out. By Euclid's lemma, we know that $p\divv a_1a_2\cdots a_n$ if and only if $p\divv a_i$ for one of the factors $a_i$. All of the factors in the left-hand product are prime, so $p$ divides none of them. Thus, $d$ must be zero, which implies that $e_p = f_p$, as desired.
\end{proof}
\end{mdframed}

\subsubsection*{Decomposition and Divisibility}
The beauty of prime decomposition is that it enables us to view the integers in terms of a signature string of exponents, the trace of the prime decomposition. This allows us to re-discover the properties of divisibility in a new and powerful way, and recast the $LCM$ and $GCD$ in terms of minimum and maximum exponents. \\\\
If $a$ divides $b$, then every prime power of $a$ must divide every prime power of $b$ in their respective decompositions. In other words, if $a = 2^{e_2}3^{e_3}\cdots$ and $b = 2^{f_2}3^{f_3}\cdots$, and $a\divv b$, then $e_p\leq f_p$ for every prime $p$.
\begin{mdframed}
\begin{prop}
Let $a$ and $b$ be positive integers with prime decompositions
\[a = 2^{e_2}3^{e_3}\cdots \qquad b = 2^{f_2}3^{f_3}\cdots \]
Then, $a\divv b$ if and only if $e_p \leq f_p$ for each prime $p$.
\end{prop}
\begin{proof}
Divisibility always implies a third party. Assume $a\divv b$. Then, $b = ac$ for some other integer $c$. $c$, in turn, possesses its own unique prime decomposition, say $c = 2^{g_2}3^{g_3}\cdots$. \\ If $b = ac$, then it follows that \[ac = 2^{e_2 + g_2}3^{e_3 + g_3} \cdots = 2^{f_2}3^{f_3}\cdots = b\]
By what we know of prime factorization, this implies that $e_p + g_p = f_p$, which is only possible if $e_p\leq f_p$. Conversely, suppose that $e_p \leq f_p$ for each prime $p$. Denoting the difference by $d_p = f_p - e_p$, it follows that we can construct a factor $c = 2^{d_2}3^{d_3}\cdots$ such that $ac = b$, which implies that $a\divv b$.
\end{proof}
\end{mdframed}
Prime decomposition enables us to consider divisibility in terms of inequality, and multiplicity in terms of addition. This perspective offers an alternative and equivalent definition for the greatest common divisor and least common multiple.
\begin{mdframed}
\begin{corollary}
Let $a$ and $b$ be integers, with prime factorizations $a = 2^{e_2}3^{e_3}\cdots, \, b = 2^{f_2}3^{f_3}\cdots$. Then,
\[\GCD{a}{b} = 2^{min(e_2, f_2)}3^{min(e_3, f_3)}\cdots \]
\[\LCM{a}{b} = 2^{max(e_2, f_2)}3^{max(e_3, f_3)}\cdots \]
Where $min$ and $max$ represent the minimum, and maximum of each pair of exponents, respectively.
\end{corollary}
\begin{proof}
We prove the case for the $GCD$, leaving the $LCM$ for the reader. Suppose that $g = \GCD{a}{b}$. Naturally, as $g$ is an integer, it can be factored into primes, say
\[g = 2^{h_2}3^{h_3}\cdots\]
What can we say about $g$? We know that $g$ divides both $a$ and $b$, certainly, so $h_p \leq e_p, \, h_p \leq f_p$. But we also know that $g$ is divided by every other divisor of $a$ and $b$. Among all divisors, it is the maximum. So, in terms of exponents, $h_p$ cannot be greater than $e_p$ or $f_p$, and yet, it must be the greatest possible between the two, which is precisely $min(e_p, f_p)$.
\end{proof}
\end{mdframed}
In summary, all integers (and rational numbers, as we will see) afford a unique decomposition into primes. In turn, this decomposition allows us to explore divisibility relationships as a consequence of the exponents, giving new perspective on some old definitions.
\begin{mdframed}
\begin{exe}
Let $a$ and $b$ be integers. Then, if $\GCD{a}{b} = 1$, $\GCD{a^m}{b^n} = 1$ for all positive integers $m, n$. \\\\ (Hint: $\GCD{a}{b} = 1 \,\Rightarrow\, min(e_p, f_p) = 0$).
\end{exe}
\end{mdframed}
\end{document}