% ---------------------------------------------------------
% Project: PhD KAPPA
% File: discussion.methods.tex
% Author: Andrea Discacciati
%
% Purpose: Percentile + RAP + GOF meta (discussion)
% ---------------------------------------------------------

\section{Survival percentiles when age at the event is the relevant time scale}

When describing survival data from observational epidemiologic studies, the standard approach is to focus on the hazard function, while differences in survival between groups of individuals are usually summarized by means of HRRs. The reason for the ubiquitous presence of HRRs in the epidemiologic literature is arguably due to the wide availability of statistical tools to directly model the hazard as a function of some covariates of interest, and not  to the fact that HRRs are inherently ``better'' measures of association than others \citep{hernan_hazards_2010, uno_moving_2014}.

A different approach to describe survival data consists in focusing on survival percentiles, which thoroughly describe the distribution of survival time. Survival percentiles provide the link between the proportion of study subjects that has experienced the event of interest and the time by which that proportion is reached.

As survival percentiles can be obtained from survival curves, both hazard models and the Kaplan-Meier estimator can be utilized to ``indirectly'' estimate them.\footnote{The word ``indirectly'' is used to highlight the fact that, both in the case of hazard models and Kaplan-Meier estimator, what is actually being estimated are survival probabilities given study time.} As seen in section \ref{sec:phapproach}, however, calculating survival curves following PH models can sometimes give a ``misleading impression'' \citep{kalbfleisch_statistical_2002} and is impractical if one wants to relax the PH assumption by employing time-dependent coefficients.  Moreover, calculation of confidence intervals for survival percentiles obtained in this way is not straightforward \citep{burr_comparison_1994, lai_confidence_2006}. The Kaplan-Meier estimator, on the other hand, is often inadequate in the context of observational epidemiology, where modeling is generally preferred over simple stratification.

Quantile regression for censored data, on the other hand, focuses directly on survival percentiles  posing no restrictions on the shape of the survival function and offers all those advantages of statistical modeling that are so important in the analysis of observational data.

In \citetalias{bellavia_using_2015}, we proposed an intuitive and simple approach to extend the use of Laplace regression to those situations in which investigators want to use age as the underlying time scale. We showed that delayed entries, introduced by the change in the time scale, complicate the interpretation of survival percentiles obtained from survival curves. In fact, in the presence of delayed entries, it is no longer true that the $100p$-th percentile of attained age, say $a(p)$, can be interpreted as the age by which $100p\%$ of the study population has experienced the event of interest. It may even occur, for example, that by age $a(p)$ less than $100p\%$ of study participants have been enrolled in the study.

By conditioning the quantile regression model on age at entry (baseline age), the $100p$-th survival percentile of age at the event can now be interpreted as the age by which $100p\%$ of the study participants have experienced the event, given a fixed baseline age. Moreover, the model coefficients express the difference in the $100p$-th percentile of age at the event between exposed and unexposed individuals, always conditional on baseline age. We think that this extension can be particularly useful for the analysis of observational studies, where attained age is often a more meaningful and natural time scale than time elapsed since some arbitrary baseline event.

As previously written, survival percentiles thoroughly describe the distribution of survival time, and likewise PDs give a full picture of the differences in the distribution of age-at-event between exposed and unexposed individuals. This can be, however, a double-edged sword. In fact, unless one is interested in a pre-specified PD (as it could be the case for clinical trials), the richer picture provided by PDs also requires more information to be reported (in terms of figures, words and/or numbers). Focusing on  one single arbitrary chosen PD  can give a ``misleading impression'' of the association just like reporting one single HRR when the PH does not seem to hold. In the illustrative example in \citetalias{bellavia_using_2015}, we reported differences in the 25th, 50th, and 75th percentiles of age at death according to smoking status for this reason.

\subsubsection{Laplace regression}

Among the different available methods for quantile regression for censored data, in this thesis and in \citetalias{bellavia_using_2015}, we employed Laplace regression, whose principal characteristic is probably the Laplacian distributional assumption of the error term. This assumption is shared by other methods for quantile regression \citep{liu_mixedeffects_2009, farcomeni_quantile_2010, lee_bayesian_2010, yuan_bayesian_2010}. Other proposed techniques do not make distributional assumptions on the error term but are subject to other assumptions, such as global linearity \citep{portnoy_censored_2003, peng_survival_2008}, not required by Laplace regression. The method by \citet{wang_locally_2009} overcomes the global-linearity assumption but its computational algorithm requires subjectively setting a smoothing function, which may ultimately have an impact on inference.

 Laplace regression has been observed to be robust to violations of the Laplacian distributional assumption. In simulation studies where the error term was generated from a number of different distributions --- including normal, lognormal, Student's $t$, and exponential distribution --- the performances of the Laplace estimator in terms of bias and coverage were observed to be remarkably good, even at high quantiles and with large rates of censoring \citep{bottai_laplace_2010,bottai_authors_2011,bottai_command_2013,bottai_gradient_2015}.

\section{Risk and rate advancement periods}

RAP reflects the exposure impact in the timing of the disease occurrence. For rates, RAP quantifies how much sooner $e_1$-exposed individuals reach the same IR of a certain disease as $e_0$-exposed individuals, under the assumptions of a monotonically increasing age-disease association and disease-free survival to baseline age.

The fact that RAP is a time-based measure of exposure impact makes it very appealing in risk communication. For example, in a recent paper published in the British Medical Journal, \citet{mons_impact_2015} wrote that ``risk communication is [...] crucial [...] and risk advancement periods could be easier to grasp for the general public than other epidemiologic risk measures such as relative risks of years of life lost''. Whether this is actually true or not remains to be seen, as no empirical evidence on the usefulness of RAP in risk communication with lay audiences has been collected yet.

What we have observed, however, is the fact that correct interpretation of RAP has eluded many of the studies that have employed this measure so far. In particular, we have identified three major conceptual problems. First, equating RAP with the difference in mean survival time; second, interpreting RAP as the time by which the survival curve for the $e_1$-exposed individuals is shifted with respect to the survival curve for the $e_0$-exposed individuals; third, equating the RAP concept to that of RRs. Furthermore, we have highlighted a too often neglected statistical problem, which is the high sensitivity of RAP estimates to the form of the age-disease association. All these problems can be quite serious in realistic examples.

The reason why RAP has been repeatedly misinterpreted  in the literature is most likely multi-factorial, but a possible explanation is that RAP is a measure involving three variables (exposure, baseline age, and binary outcome) instead of two as traditional measures of association. As RAP evaluates the impact of the exposure on the relation of age to the outcome, it is more similar to an interaction measure than to a measure of association, although it is not equivalent to a product term in the model. This supplementary complexity might partially explain why RAP has been repeatedly misinterpreted.

\subsubsection{RAP in prostate cancer epidemiology}

A crucial assumption for sensible use of RAP is that the rate of the outcome strictly increases with age, given exposure and confounders. If this assumption is not met, one should avoid use of RAP. As seen in the COSM  (figure \ref{fig:ir_cosm}) and in other populations \citep{ferlay_cancer_2015}, the incidence of prostate cancer  increases until around 70--74 years of age and declines thereafter. This is likely due to less use of PSA-testing in men over 80 years of age \citep{williams_prostatespecific_2011} and, at the same time, to a constant increase over time in PSA-testing among younger men   \citep{salinas_prostate_2014}. Although the reasons behind the non-monotonicity of the age-incidence curve are unlikely to be biological, this limits  the application of RAP in this context. A possible solution is to restrict RAP analysis to men younger than 70--74 years of age to ensure that the assumption of a monotonic increase in disease rate over age holds.

In contrast, the monotonicity assumption seems to hold for prostate cancer mortality \citep{ferlay_cancer_2015}. In the example reported in section \ref{sec:example_rap_bmi} using data from the COSM, obese men were observed to experience the same prostate cancer MR over the follow-up period 1998--2012 as that of normal-weight men who were 18 months older at baseline. Despite the lack of evidence of a non-linear association between age and log--mortality rate of prostate cancer, RAP estimates were somewhat sensitive to the functional form with which baseline age entered into the model.

Lastly, interpretation of RAP as the time period by which attainment of a particular rate level can be postponed by eliminating the exposure requires the (conditional) independence of the study disease from competing events. As \citet{brenner_risk_1993} wrote ``this limitation is negligible if the RAP is derived for young and middle-aged individuals and if the RAP is not too long''. In the case of prostate cancer mortality, on the other hand, since the vast majority of men dying of the disease are elderly, interpretation of RAP estimates becomes more theoretical.

%--

\section{Goodness of fit assessment in dose--response meta-analysis}

\subsection{Why goodness of fit assessment is important}

Statistical modeling in the context of dose--response meta-analysis can be thought of as `descriptive modeling' \citep{rosenthal_statistical_1985, shmueli_explain_2010}. Citing the words by \citet{shmueli_explain_2010} ``this type of modeling is aimed at summarizing or representing the data structure in a compact manner''. %This as opposed to `explanatory modeling' and to `predictive modeling' \citep{shmueli_explain_2010}.

In order to give a fair representation of the data structure --- that is, of the information regarding a certain dose-risk relation --- one should make sure that the posited dose--response meta-analysis models actually provide an adequate description of the data at hand. Therefore, we argue that the natural `third stage' of a dose--response meta-analysis should be the evaluation of its goodness of fit, which can be done in practice by measuring the degree of agreement between fitted and observed data.

A poor fit can  raise doubts about the ability of a certain model to summarize the available data or as \citet{greenland_invited_1994} put it ``if one views statistical estimates as data summaries, then one can view significant lack of fit as a warning that the fitted model structure and model-based estimates are poor data summaries''.

Despite its importance, however, the issue of how to evaluate goodness of fit of dose--response meta-analysis models has, to the best of our knowledge, never been specifically addressed. As a consequence, assessment of the goodness of fit  is rarely, if ever, carried out in practice. As \citet{sutton_recent_2008} pointed out “little formal assessment of the goodness-of-fit of meta-analysis models to the data is carried out. This may be partly because many non-statisticians conduct meta-analysis, and to such applied researchers meta-analysis may be seen as a necessary data-processing procedure rather than a model-fitting exercise”.

The aforementioned degree of agreement has been sometimes evaluated by simply overlaying the study-specific RRs to the pooled dose--response curve \citep[see, for example,][]{wcrf_continuous_2014, liao_blood_2015}. However, although praiseworthy, this approach can be highly misleading as the correlation among the reported logRRs means that even a well-fitting dose--response curve might not pass through the data points. This was clearly illustrated, in the simple case of trend estimation for one single study, in figure \ref{fig:visual_wrong}.

Furthermore, study-specific dose--response relations are sometimes illustrated by plotting the reported RRs at their assigned doses and connecting them with straight lines (segments) \citep{wcrf_continuous_2014}. This can be misleading for the very same reasons given above. In fact, visually inferring the dose--response relation from this kind of plot may lead to wrong conclusions, given the that the correlation between RRs is not taken into account.

\subsubsection{A digression on goodness of fit}

The statistical modeling carried out for the description of COSM data and for the comparison with Swedish national data (section \ref{sec:materialsepi}) can also be viewed as descriptive modeling and the same considerations about goodness of fit apply. In fact, we overlaid to the model predictions the observed (summarized) data, so that the goodness of fit of the models could be, at least qualitatively, evaluated.

Statistical modeling in \citetalias{discacciati_body_2011}, on the other hand, was aimed at `explaining' --- that is, testing hypotheses about theoretical constructs \citep{shmueli_explain_2010}. In `explanatory modeling' one is not usually interested in assessing the overall absolute goodness of fit of a model, as ``the fit of the model must be evaluated only against the specific part of the variation [of the outcome] which is relevant to the subset of effects of interest. An overall test of the model fit is too general for this purpose and does not answer the right question'' \citep{hagquist_goodness_1998}.

\subsection{Deviance, coefficient of determination, and visual assessment}

To highlight the need of assessing the goodness of fit in dose--response meta-analysis, we presented and discussed in \citetalias{discacciati_goodness_2015} three goodness of fit tools (deviance, coefficient of determination, and \rveplot). These tools can be useful for testing, quantifying, and visually displaying the fit of dose--response meta-analysis models while taking into account the correlation structure of the study-specific logRRs.

The examples given in \citetalias{discacciati_goodness_2015} and in section \ref{sec:examplegof} showed how these tools can give important indications regarding the fit of the candidate models. In particular, these tools can help identify dose--response patterns, investigate sources of heterogeneity, and eventually assess wether the pooled dose--response association adequately summarizes the published results. Their use can strengthen the conclusion drawn from a dose--response meta-analysis or, conversely, raise doubts about its ability to describe in an adequate manner the available evidence. However, one should also be aware that these tools come with some  limitations.% and, more in general, that goodness of fit assessment is not an easy task to carry out.

\subsubsection{Limitations and caveats}

First, while a small $p$-value from the test for model specification (deviance) is an indication that the considered model fails at explaining the observed variation in the reported logRRs, a large $p$-value shall not lead to the conclusion that the model adequately explains all the observed variability. Moreover, although a small $p$-value is an indication that the the tested model is unsatisfactory and needs to be modified (or completely replaced), it provides no indication as to how to proceed. Lastly, a major drawback of all global tests of fit is their low power to detect problems in the model \citep{hosmer_comparison_1997}.

Second, when the dose--response meta-analysis models are specified in a data-dependent fashion (also known as data dredging), $p$-values from the global goodness of fit cannot be formally regarded as valid.

Third, although a $R^2=1$ does correspond to a perfect fit, interpreting a low coefficient of determination is more complicated. In fact, $R^2$ can be close to 0 because of different reasons: the model fits poorly the data, the  exposure and the logRRs are not associated, or simply  --- even under the ``correct'' model --- the GRSS is close to the GTSS.

Fourth, the interpretation of the \rveplot{} is based on the visual recognition of patterns in the distribution of the decorrelated residuals by levels of the exposure, which is a subjective experience \citep{greenland_invited_1994}. In the extreme situation of sparse data, the meta-analyst can ``recognize'' almost any pattern in the plot.

%All these limitations are particularly relevant if one wants to assess the goodness of fit of study-specific dose--response models.

In conclusion, a good fit alone does not imply that the ``correct'' model has been selected. In extreme cases, especially when the number of data points is small, very different pooled dose--response curves may appear to be equally satisfactory from a goodness-of-fit point of view. This is not surprising, as it simply reflects the fact that the total amount of available information is limited. In such cases, subject matter knowledge --- that is, prior information --- will play a crucial role in selecting the final model. At the same time, goodness of fit assessment can help to weed out those models that fail at adequately summarizing the available evidence.


\documentclass[10pt,conference]{ieeeconf}

%\usepackage{algpseudocode}% http://ctan.org/pkg/algorithmicx
\usepackage{xcolor}
\usepackage{subfigure}
\usepackage{amsmath,amssymb,amsbsy}
\usepackage{pdfpages}
\usepackage{epstopdf}
\usepackage{color}
\usepackage{fixmath}
\usepackage{optidef}
\usepackage{cite}
\usepackage{array,tabularx}
\usepackage{booktabs}
\usepackage[hidelinks]{hyperref}
\usepackage{cleveref}

\usepackage{color}

\ifpdf
  \RequirePackage{graphicx}
  \RequirePackage{epstopdf}
  \DeclareGraphicsExtensions{.pdf,.jpeg,.png,.jpg}
\else
  \RequirePackage[dvipdfmx]{graphicx}
  \RequirePackage{bmpsize}
  \DeclareGraphicsExtensions{.eps,.pdf,.jpeg,.png,.jpg}
\fi
\graphicspath{{pics/},{./Images/}}

\RequirePackage{algorithm}
\RequirePackage{algorithmic}
\renewcommand{\algorithmicrequire}{\textbf{Input:}}
\renewcommand{\algorithmicensure}{\textbf{Output:}}
\providecommand{\algorithmautorefname}{Algorithm}

\RequirePackage{subfigure}
\RequirePackage{empheq}
\providecommand{\subfigureautorefname}{\figureautorefname}
\Crefname{table}{Table}{Table}
\Crefname{figure}{Fig.}{Fig.}
\Crefname{algorithm}{Algorithm}{Algorithm}

\providecommand{\rm}[1]{\mathrm{#1}}
\providecommand{\od}{\mathrm{d}}

\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}

\IEEEoverridecommandlockouts
\begin{document}

\title{Multichannel Deconvolution of Skin Conductance Data: Concurrent Separation of Tonic and Phasic Component}
% should be concise and informative to broad readers and attract attention from potential readers.

\author{Yuchen Jin, Jin Lu, Md. Rafiul Amin, and Rose T. Faghih \thanks{Yuchen Jin, Jin Lu, Md. Rafiul Amin, and Rose T. Faghih are with the Department of Electrical and Computer Engineering at the University of Houston, Houston, TX 77004 USA (e-mail: \tt\small  yjin4@uh.edu, jlu28@uh.edu, mamin@central.uh.edu, rtfaghih@uh.edu).}
}

\maketitle

\begin{abstract}

Understanding the electrodermal activities (EDA) is an essential step in neuroscience. To retrieve the neural events from EDA, one of the key step is the deconvolution. In this work, we decompose the skin conductance (SC) data as a combination of the phasic component and the tonic component. To eliminate the influence of the noise, we utilize multi-channel data and use a multi-channel state-space model to describe the phasic component. To improve the fitness of the tonic extraction, we propose an adaptive bases method and ensure that the appearance of peaks in the tonic component is relatively later than the appearance of the phasic component. To perform the optimization, we adopt the concurrent coordinate descent algorithm, where the phasic component and the tonic component are solved by the interior point method and FOCUSS+ respectively. To determine the proper value of the regularization ratio, we introduce the generalized cross-validation (GCV) in both methods. The experiments show that our method could achieve a good performance. Compared to the results of the conventional tonic extraction method, the fitness of our results is significantly improved.

\end{abstract}

\section{Introduction} \label{introduction}

Analyzing the electrical activities of neurons is (an important aspect in the neuroscience) of great interest in biomedical signal processing. By placing the electrodes, intramuscular needles or some other wearable devices, the neuron activities could be detected by different kinds of indirect methods including the electroencephalogram, electrodermal activities~\cite{savazzi2019estimation,jain2016compressed,amin2019robust}, electromyography signals~\cite{biagetti2016homomorphic}, calcium imaging, and some other approaches. The study of the neural activities could be utilized in many applications, like predicting people's emotions~\cite{feng2018wavelet}, monitoring the health-related abnormal events~\cite{said2017wearable}, and developing the brain-computer interface~\cite{de2018use}.

In this work, we concentrate on the study of electrodermal activities (EDA). One example of EDA is the skin conductance (SC) data, which is collected by detecting activities of people's sweat glands. The sensors for detecting SC could be placed on either the hand palms, fingers, legs or feet. Previous works~\cite{fowles1981publication,amin2019robust} show that the SC of different body parts is controlled by the universal neural signal from the autonomic nervous system (ANS). Within this hypothesis, the whole collection of SC could be modeled as a group of multichannel signals, where one channel represents the collected SC from a specific body part. For any channel $n$, the signal could be formulated by
\begin{align} \label{fml:scmodel}
  y_{\rm{SC}_\mathit{n}}(t) = p_n(t) + s_n(t) + \nu_n(t),
\end{align}
%
where $s_n(t)$ is the tonic component representing the influence of the thermoregulation and general arousal of the
body, $p_n(t)$ is the phasic component thought as reflected from the activity of ANS, and $v_n(t)$ is the additive noise. Different from the slowly drifting tonic component, the phasic component is fast-varying, which makes it possible to separate the two signals by formulating two different mathematical models. Either the tonic component and the phasic component could be regarded as a linear composition of different basis functions, or a group of kernels convoluted with weighted impulse trains. It is possible to formulate a deconvolution problem with the physiological interpretation of SC, which requires a solution for the ANS activation given the composed signal  $y_{\rm{SC}_n}(t)$  is known.

\subsection{Deconvolution}

The neural stimuli from the ANS is though as spike signals. While after the signal is conducted to the sweat glands, the signals are smoothly filtered due to the (physiological effects) physiological systems. After applying the deconvolution, the estimated neural stimuli could be used for denoising, emotion predictions, or health monitoring. \textit{Caruelle et al.}~\cite{caruelle2019use} summarizes the applications of the EDA analysis in recent years. For more details, \textit{Posada et al.}~\cite{posada2020innovations} provides a review on the previous deconvolution methods with the EDA data. Similar to EDA, the other kinds of signals could be also used for studying the neuron activities by deconvolution. In \Cref{tab:literature}, we inspect on several works on deconvolution with different data. For the simplest case, the spike signal could be retrieved by applying a threshold and performing the local maximum searching~\cite{kaur2016remote,subramanian2019systematic}. In most works, the deconvolution is solved by formulating an optimization problem based on a linear model, including the Bateman function~\cite{savazzi2019estimation,greco2014electrodermal,greco2015cvxeda,amin2019tonic,hernando2017feature,wickramasuriya2019skin}, the AR-1 model~\cite{friedrich2017fast}, and the state-space model~\cite{kazemipour2017fast,amin2019robust}.

\begin{table*}[htbp]
  \centering
  \normalsize
  \caption{A summary of the previous works about deconvolution.}
  \label{tab:literature}
  \begin{tabular}{m{0.14\textwidth}<{\raggedright}m{0.14\textwidth}<{\raggedright}m{0.65\textwidth}}
    \toprule
    \multicolumn{1}{c}{Study} & \multicolumn{1}{c}{Data} & \multicolumn{1}{c}{Methodology} \\ \midrule
    \textit{Friedrich et. al.}~\cite{friedrich2017fast} & Calcium imaging data & The conversion from the calcium spikes to the observed fluorescenc signals is described as an AR-p model. The authors propose an algorithm, OASIS for solving the constrained optimization problem in the AR-1 case. \\
    \textit{Kazemipour et. al.}~\cite{kazemipour2017fast} & Calcium imaging data & Develop a compressible state-space model for describing the data. The spikes are designed as the innovation sequence of the state. The optimization problem is solved by a nested Expectation-Maximization algorithm. \\
    \textit{Kaur et. al.}~\cite{kaur2016remote} & Electro-cardiogram & Introduce two methods. The first method is based on curve fitting and local maximum searching. The second method is Blind Deconvolution, where the kernel and the spikes are optimized alternatively. \\
    \textit{Mesin}~\cite{mesin2019single} & Electromyogram & Model the EMG signal by motor unit kernels. The deconvolution is performed by solving an $\ell_1$ regularized optimization problem by the iterative reweighted least square algorithm. \\
    \textit{Antink et. al.}~\cite{friedrich2017fast} & ECG, ABP and PPG & The author use the blind deconvolution to solve the heart beat spikes. This paper introduces a multi-observation scheme by assuming a universal spike data. \\
    \textit{Subramanian et. al.}~\cite{subramanian2019systematic} & EDA & Use a low-pass FIR filter to isolate the tonic component, the spike is found by searching local maximum in phasic component. \\
    \textit{Savazzi et. al.}~\cite{savazzi2019estimation} & EDA & The tonic component and noise is removed by a low-pass FIR filter. The phasic component is modeled by Bateman function. The spike is acquired by optimization. \\
    \textit{Jain et. al.}~\cite{jain2016compressed} & EDA & Involve the compress sensing into the deconvolution. The optimization is performed on the differential signal to eliminate the influence of sudden changes of the tonic component. \\
    \textit{Greco et. al.}~\cite{greco2014electrodermal} & EDA & The tonic component is estimated by Gaussian smoothing and interpolation. The spike is solved by modeling the phasic component with Bateman function. \\
    \textit{Greco et. al.}~\cite{greco2015cvxeda}, \textit{Amin and Faghih}~\cite{amin2019tonic} & EDA & Model the phasic component and the tonic component by Bateman function and spline function respectively, then formulate the deconvolution as a joint constrained convex optimization problem. \\
    \textit{Hernando et. al.}~\cite{hernando2017feature} & EDA & The tonic and phasic components are decomposed by two groups of pre-defined bases. The phasic component basis is designed by using Bateman function. \\
    \textit{Wickramasuriya et. al.}~\cite{wickramasuriya2019skin} & EDA & The tonic component is isolated by using cvxEDA~\cite{greco2015cvxeda}. The phasic component is decomposed by a basis based on Bateman function. The deconvolution is solved by a hybrid algorithm based on GCV and FOCUSS+. \\
    \textit{Amin and Faghih}~\cite{amin2019robust} & EDA & The authors propose a multichannel deconvotion scheme. The tonic component is removed by using cvxEDA~\cite{greco2015cvxeda} on each channel, and the phasic component is formulated by a state-space model. The deconvolution is solved by the hybrid algorithm GCV-FOCUSS+. \\ \bottomrule
  \end{tabular}
\end{table*}

The complexity of the deconvolution differs in different applications. In \cite{greco2015cvxeda}, the basis determining the shape of the phasic component is fixed, while in \cite{hernando2017feature}, the phasic component is modeled by several fixed bases. In \cite{greco2014electrodermal,amin2019tonic,wickramasuriya2019skin,kazemipour2017fast,amin2019robust}, although there is a single basis, the shape of the basis could be learned. Particularly, some researches assume that the linear model is totally unknown. By only giving an initial guess, the modeling function is also optimized by blind deconvolution~\cite{kaur2016remote,friedrich2017fast}. For a large scale application, the problem requires to be formulated properly according to the computational consumption.

Most of the mentioned works deal with single channel data, while in \cite{friedrich2017fast,amin2019robust}, a multichannel scheme is introduced for improving the robustness of the predictions. In the practice, observations are contaminated with the noise, which may influence the accuracy of the deconvolved spikes. The motivation of introducing multichannel data is based on the hypothesis that the same neural stimuli is shared by the data acquired by different observations. When the multichannel data is deconvolved jointly, the noise from different channels could be attenuated by each other, thus the problem could converge towards a better solution.

\subsection{Tonic separation}

Previously there some works on the tonic component separation. In \cite{van1967skin}, the raw EDA signal is decomposed as absolute change in conductance, change as a ratio of the basic conductance level (BLC), and log conductance change. In \cite{green2014development}, the author proposes a method of automated analysis of SCR data in the contexts of event-related cognitive tasks and nonspecific responding to complex stimuli. \cite{greco2014electrodermal} formulates the separation as a quadratic programming problem, where the phasic component and the tonic component are decomposed by the basis of the biexponential Bateman functions and the B-spline functions respectively. To solve the problem that the tonic component may overlap each other, \textit{Vartak et. al.}~\cite{vartak2009estimation} proposes a mathematical model fitting procedure is used to separate these overlapping components. Features are extracted using the mathematical model fitting procedure. In~\cite{amin2019tonic}, the author proposes a generalized-cross-validation-based block coordinate descent approach. In this approach, the tonic component is regarded as a summation of a series of P shifted and weighted cubic B-spline functions. Then the problem is solved by minimizing an $\ell_p$ penalized loss function. Especially, inspired by compress sensing, \textit{Jain et. al.}~\cite{jain2016compressed} builds a model for separating the difference of the signal instead of the raw signal. This idea is effective for eliminating the interference caused by sudden changes of the tonic component.

In some previous works, the tonic component separation and the phasic component deconvolution are performed individually~\cite{greco2014electrodermal,amin2019robust}. Since both the modeling method and the inversion method in \cite{amin2019robust} are designed for the multichannel case, the previous tonic separation works designed for single-channel data may cause deviation for the solution. In addition to the motivation, we are interested in multi-channel deconvolution inorder to obtain more accurate results in presence of noise. As tonic component separation is performed separately, inaccuracy in separation in presence of artifact and noise may lead to inaccurate deconvolution of ANS activation. A multi-channel tonic and phasic decomposition may lead to a more robust decomposition in presence of noise. In our work, we extend the multichannel deconvolution into the joint optimization for solving the tonic component in the multichannel case. With this scheme, the neural stimuli from the ANS is though to be shared, while the solution for the composition of tonic basis is assumed to be different in different channels. To improve the fitness of the tonic component, we propose a new method for generating the B-spline bases based on the solve neural stimuli. The results show that our method could achieve a better performance.

\section{Theory}

According to \eqref{fml:scmodel}, the SC data could be decomposed into a tonic component and a phasic component. In this work, we will model both components by different methods. The phasic component is decomposed by a series of state-space models sharing the same input, while the tonic component is described as a combination of different spline functions. Based on the theory-based multichannel forward modeling, the stimuli from the ANS could be solved by a joint optimization. In the meanwhile, the tonic component would be extracted from the data.

\subsection{Model the phasic component with a multichannel state-space representation}
According to the presumption in \cite{alexander2005separating,society2012publication,amin2019robust}, the phasic components detected in different regions of the body are regulated by the same ANS signal, the detected phasic component for channel $n$ could be formulated by
\begin{align}
p_n(t) = \alpha_n \zeta_n (t),
\end{align}
where $\alpha_n$ denotes the attenuation rate compared to the first channel. This attenuation is caused by the the different number and sizes of sweat glands in different location. In this configuration, $\alpha_1$ is fixed on 1.0. $\zeta_n(t)$ is an internal variable which is modeled by a smoothing filter applied to the neural stimuli $u(t-\beta_n)$. The conduction delay $\beta_n$ shows that the
same stimuli would be detected at different times at different locations. We follow the configurations in \cite{greco2015cvxeda,amin2019robust} and model the neural stimuli by a series of spikes with the sparse constraint, i.e. $u(t) = \sum_{i=1}^N q_i \delta(t - \Delta_i)$, where $q_i$ is the spike signal in time interval i, and $\Delta_i$ is the time for $i^{\mathrm{th}}$ time interval. Given $u(t)$, \textit{Alexander et al.}~\cite{alexander2005separating} provides a second-order ordinary differential equation to describe $\zeta_n(t)$,
\begin{align} \label{fml:the:ode}
\tau_r \tau_d \frac{\od^2 \zeta_n(t)}{\od t^2} + (\tau_r + \tau_d) \frac{\od \zeta_n(t)}{\od t} + \zeta_n(t) = u(t - \beta_n).
\end{align}
where $\tau_r$ and $\tau_d$ represent the rise and decay times respectively.

The difference between different channels in \eqref{fml:the:ode} is only the time delay. In other words, the solutions of $\zeta_n(t)$ for different $n$ are the same except the time delay. According to \eqref{fml:the:ode}, \textit{Faghih et al.}~\cite{faghih2015characterization} propose a state space model. Incorporating the time delay into the internal signal $\zeta_n(t)$, we define an internal state $x^{(n)}_2 (t)$ as $\zeta_n(t + \beta_n)$. The state-space model could be formulated as follows,
%\begin{subequations} \label{fml:the:state-raw}
%  \renewcommand{\theequation}
%  {\theparentequation-\arabic{equation}}
%  \begin{empheq}[left=\empheqlbrace]{align}
%  \dot{x}^{(n)}_1 (t) &= a x^{(n)}_1(t) + b u(t),\\
%  \tau_r \tau_d \dot{x}^{(n)}_2 (t) &= c x^{(n)}_1(t) + d x^{(n)}_2(t), \label{fml:the:state-raw-2}\\
%  y_n (t) &= \alpha_n x^{(n)}_2(t).
%  \end{empheq}
%\end{subequations}
%{\color{red} I have to say that I do not know why we design \eqref{fml:the:state-raw}.}
%
%where $x^{(n)}_1(t)$ is another internal state. To solve the coefficients $a, b, c, d$, we eliminate $x^{(n)}_1(t)$ in \eqref{fml:the:state-raw-2},
%\begin{equation} \label{fml:the:ode-from-state}
%\begin{aligned}
%\tau_r \tau_d \ddot{x}_2 (t) &= c \dot{x}^{(n)}_1(t) + d \dot{x}^{(n)}_2(t) \\
%&= c \left(a x^{(n)}_1(t) + b u(t) \right) + d \dot{x}^{(n)}_2(t) \\
%&= a ( \tau_r \tau_d \dot{x}^{(n)}_2 (t) - d x^{(n)}_2(t) ) + bc u(t) + d \dot{x}^{(n)}_2(t), \\
%&= (a \tau_r \tau_d + d) \dot{x}^{(n)}_2 (t) - a d x^{(n)}_2(t) + bc u(t).
%\end{aligned}
%\end{equation}
%
%To ensure the coherence between \eqref{fml:the:ode} and \eqref{fml:the:ode-from-state}, we have,
%\begin{subequations}
%  \renewcommand{\theequation}
%  {\theparentequation-\arabic{equation}}
%  \begin{empheq}[left=\empheqlbrace]{align}
%  a \tau_r \tau_d + d &= - \tau_r - \tau_d,\\
%  ad &= 1, \\
%  bc &= 1.
%  \end{empheq}
%\end{subequations}
%Given $a = -1/\tau_r$ and $b = 1/\tau_r$, we could solve the above equations, and substitute the solutions into \eqref{fml:the:state-raw},
\begin{subequations} \label{fml:the:state-sing}
  \renewcommand{\theequation}
  {\theparentequation-\arabic{equation}}
  \begin{empheq}[left=\empheqlbrace]{align}
    &\begin{aligned}
      \begin{bmatrix}
        \dot{x}^{(n)}_1 (t) \\ \dot{x}^{(n)}_2 (t)
      \end{bmatrix} &= \begin{bmatrix}
        -1/{\tau_r} & 0 \\ 1/{\tau_d} & -1/{\tau_d}
      \end{bmatrix} \begin{bmatrix}
        x^{(n)}_1(t) \\ x^{(n)}_2 (t)
      \end{bmatrix} \\ &+ \begin{bmatrix}
        1/{\tau_r} \\ 0
      \end{bmatrix} u(t),
    \end{aligned} \\
    &y_n (t) = \alpha_n x^{(n)}_2(t),
  \end{empheq}
\end{subequations}
where $x^{(n)}_1(t)$ is another internal state. This state is derived from the equivalency between the model \eqref{fml:the:state-sing} and the original derivative equation \eqref{fml:the:ode}.

Given the single-channel transmission matrix $\phi=\begin{bmatrix}
-1/{\tau_r} & 0 \\ 1/{\tau_d} & -1/{\tau_d}
\end{bmatrix}$, we could derive \eqref{fml:the:state-sing} into the multichannel form. When we have $\chi$ channels of data, the multichannel state-space model is
\begin{subequations} \label{fml:the:state-mul}
  \renewcommand{\theequation}
  {\theparentequation-\arabic{equation}}
  \begin{empheq}[left=\empheqlbrace]{align}
  \dot{\mathbf{x}} (t) &= \mathbf{A} \mathbf{x} (t) + \mathbf{B} u (t), \\
  \mathbf{y} (t) &= \mathbf{C} \mathbf{x} (t),
  \end{empheq}
\end{subequations}
where $\mathbf{x} (t) = \begin{bmatrix}
x^{(1)}_1 (t) \\ x^{(1)}_2 (t) \\ \vdots \\ x^{(\chi)}_1 (t) \\ x^{(\chi)}_2 (t)
\end{bmatrix}$, $\mathbf{y} (t) = \begin{bmatrix}
y_1 (t) \\ y_2 (t) \\ \vdots \\ y_{\chi} (t)
\end{bmatrix}$, $\mathbf{A} = \begin{bmatrix}
\phi & \mathbf{0} & \cdots & \mathbf{0} \\
\mathbf{0} & \phi & \cdots & \mathbf{0} \\
\vdots & \vdots & \ddots & \vdots \\
\mathbf{0} & \mathbf{0} & \cdots & \phi
\end{bmatrix}$, $\mathbf{B} = \begin{bmatrix}
1/{\tau_r} \\ 0 \\ \vdots \\ 1/{\tau_r} \\ 0
\end{bmatrix}$, and\\$\mathbf{C} = \begin{bmatrix}
0 & \alpha_1 & 0 & 0 & \cdots & 0 & 0 \\ 0 & 0 & 0 & \alpha_2 & \cdots & 0 & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & 0 & 0 & 0 & \cdots & 0 & \alpha_{\chi}
\end{bmatrix}$.

The above model could be rewritten as a discrete model. Denote the sampling rates of the observation and the neural stimuli as $T_y$ and $T_u$ respectively, we could formulate the discretion of the time as $t_k = k T_y$ and $\Delta_i = i T_u$ for both signals. When $T_u$ is small enough, we could approximate the discrete neural stimuli by $\mathbf{u} = \begin{bmatrix}
u_1 & u_2 & \cdots & u_N
\end{bmatrix}^T$, where we use $u_i=0$ to represent no neural impulse at the time step $\Delta_i$. Based on these configurations, we could derive the discrete model matrices by
\begin{subequations} \label{fml:the:state-dis-mat}
  \renewcommand{\theequation}
  {\theparentequation-\arabic{equation}}
  \begin{empheq}[left=\empheqlbrace]{align}
  \mathcal{A} &= \mathcal{L}^{-1}\{ \left(s \mathbf{I} - \mathbf{A} \right)^{-1} \}(T_u), \\
  \mathcal{B} &= \mathbf{A}^{-1}(\mathcal{A} - \mathbf{I})\mathbf{B}, \\
  \mathcal{C} &= \mathbf{C},
  \end{empheq}
\end{subequations}
where $\mathcal{L}^{-1}$ is the inverse Laplace transform.

The discrete form of \eqref{fml:the:state-mul} is
\begin{subequations}
  \renewcommand{\theequation}
  {\theparentequation-\arabic{equation}}
  \begin{empheq}[left=\empheqlbrace]{align}
  \mathbf{x} [k+1] &= \mathcal{A} \mathbf{x} [k] + \mathcal{B} \mathbf{u} [k], \\
  \mathbf{y} [k] &= \mathcal{C} \mathbf{x} [k],
  \end{empheq}
\end{subequations}

In practice, we assume that $T_y = L T_u$, where $L$ is an integer. In this case, we could denote the state vector based on the sampling rate $T_y$ by $\mathbf{z}[k] = \mathbf{x}[Lk]$. Then we have $\mathcal{A}_d = \mathcal{A}^L$, $\mathcal{B}_d = \begin{bmatrix}
\mathcal{A}^{L-1} \mathcal{B} & \mathcal{A}^{L-2} \mathcal{B} & \cdots \mathcal{B}
\end{bmatrix}$, $\mathbf{u}_d[k] = \begin{bmatrix}
\mathbf{u}[Lk] & \mathbf{u}[Lk+1] & \cdots \mathbf{u}[Lk+L-1]
\end{bmatrix}^T$. By this way, the discrete model is finally formulated as
\begin{subequations} \label{fml:the:state-dis}
  \renewcommand{\theequation}
  {\theparentequation-\arabic{equation}}
  \begin{empheq}[left=\empheqlbrace]{align}
  \mathbf{z} [k+1] &= \mathcal{A}_d \mathbf{z} [k] + \mathcal{B}_d \mathbf{u}_d [k], \\
  \mathbf{y} [k] &= \mathcal{C} \mathbf{z} [k],
  \end{empheq}
\end{subequations}

Since the system is casual, we have
\begin{align} \label{fml:the:out-dis}
\mathbf{y}[k] = \mathcal{F}[k] \mathbf{z}_0 + \mathcal{D}[k] \mathbf{u},
\end{align}
where $\mathcal{F}[k] = \mathcal{C}\mathcal{A}_d^k$, $\mathcal{D}[k] = \mathcal{C} \begin{bmatrix}
\mathcal{A}_d^{k-1} \mathcal{B}_d & \mathcal{A}_d^{k-2} \mathcal{B}_d & \cdots \mathcal{B}_d & \mathbf{0}_{N-kL}
\end{bmatrix}$ and \\$\mathbf{u} = \begin{bmatrix}
\mathbf{u}_d[0] & \mathbf{u}_d[1] & \cdots & \mathbf{u}_d[M-1]
\end{bmatrix}^T$. The initial condition is configured as \\$\mathbf{z}_{0} = \mathbf{z}[0] = \begin{bmatrix}
0 & y_1(0) & 0 & \frac{y_2(0)}{\alpha_2} & \cdots & 0 & \frac{y_{\chi}(0)}{\alpha_{\chi}}
\end{bmatrix}^T$.

Let $\mathbf{y} = \begin{bmatrix}
\mathbf{y}[1]^T & \mathbf{y}[2]^T & \cdots & \mathbf{y}[M]^T
\end{bmatrix}^T$, $\mathcal{F}_{\boldsymbol{\theta}} = \begin{bmatrix}
\mathcal{F}[0] & \mathcal{F}[1] & \cdots & \mathcal{F}[M-1]
\end{bmatrix}^T$ and \\$\mathcal{D}_{\boldsymbol{\theta}} = \begin{bmatrix}
\mathcal{D}[0] & \mathcal{D}[1] & \cdots & \mathcal{D}[M-1]
\end{bmatrix}^T$, we can represent the whole multichannel phasic component by
\begin{align} \label{fml:the:out-dis-all}
\mathbf{y} = \mathcal{F}_{\boldsymbol{\theta}} \mathbf{z}_{0} + \mathcal{D}_{\boldsymbol{\theta}} \mathbf{u},
\end{align}
where we use $\boldsymbol{\theta}$ to denote the learnable vector $\begin{bmatrix}
\tau_r & \tau_d & \alpha_1 & \alpha_2 & \cdots & \alpha_{\chi}
\end{bmatrix}^T$, where $\alpha_1$ is fixed on 1.0.

\subsection{Model the tonic component with adaptive spline functions}

The tonic signal of the $n^{\mathrm{th}}$ channel could be viewed as the coefficients $q_n (t)$ convolved with the cubic B-spline function $\psi(t)$,
\begin{align} \label{fml:tonic-conv}
s_n(t) =  \psi(t) \otimes q_n (t).
\end{align}

The B-spline function $\psi(t)$ is produced by the following convolution
\begin{subequations}
  \renewcommand{\theequation}
  {\theparentequation-\arabic{equation}}
  \begin{align}
    \psi(t) &= \psi_1 (t) \otimes \psi_1 (t), \\
    \psi_1(t) &= \left\{\begin{aligned}
      &\kappa - t, && 0 \leqslant t < \kappa  \\
      &\kappa + t, && -\kappa < t < 0 \\
      &0, && \text{otherwise}.
    \end{aligned}\right.
  \end{align}
\end{subequations}
where $\psi_1$ is the first-order function used for generating the B-spline function. $\kappa$ represents the knot size of the function. The knot size determines the shape of the function. The larger the knot size is, the smoother the function will be.

The coefficients could be viewed as a time-series signal composed of several spikes, i.e. $q_n (t) = \sum_{j=1}^K q_{nj} \delta(t^{(s)_j})$. Conventionally, $q_n(t)$ is modeled by a spikes with fixed intervals, i.e. $t^{(s)_j} = t - jT_s$, where $T_s$ is the sampling rate of the coefficients. With these configurations, the coefficients are discretized by $\tilde{\mathbf{q}}_n = \begin{bmatrix}
q_{n1} & q_{n2} & \cdots q_{nK}
\end{bmatrix}^T$.

In this work, we propose a new method for sampling $q_{n}(t)$, where the sampling rate of the bases is adaptive. We investigate the tonic components of the final results, and find that the fitness could be improved if the spikes of tonic components $q_n(t)$ appears a little bit later than the neural stimuli $u_j$. Therefore, we propose a new sampling method, where $t^{(s)}_j = t - t^{(u)}_j - T_{ds}$, where $t^{(u)}_j$ is the time point when the $j^{\mathrm{th}}$ non-zero impulse of the neural stimuli appears, and $T_{ds}$ is a pre-defined delay time related to the appearances of neural stimuli.

The convolution on $\psi(t)$ could be formulated by a $M \times K$ Toeplitz matrix $\tilde{\mathbf{S}} = \mathcal{S}(\mathbf{u})$, indicating that the bases are generated based on $\mathbf{u}_j$. Suppose that the signal is discretized as $M$ points, and we solve $K$ neural impulses for $\mathbf{u}$, the $m^{\mathrm{th}}$ row of the matrix could be formulated as
\begin{align} \label{fml:tonic-basis-1}
  \begin{aligned}
    \tilde{\mathbf{s}}_m = \big[ &\psi(mT_y - t^{(u)}_1 - T_{ds}) & \psi(mT_y - t^{(u)}_2 - T_{ds}) && \cdots \\
    & \psi(mT_y - t^{(u)}_K - T_{ds}) \big]. &&&
  \end{aligned}
\end{align}

We follows the work in~\cite{greco2015cvxeda}, where two linear trend parameters are introduced for solving the tonic component. Given the following linear trend matrix,
\begin{align} \label{fml:tonic-basis-2}
  \mathbf{R} = \begin{bmatrix}
    1/M & 2/M & 3/M & 4/M & \cdots & 1 \\
    1 & 1 & 1 & 1 & \cdots & 1
  \end{bmatrix}^T,
\end{align}
then we could formulate the tonic component as
\begin{align}
\mathbf{s}_n = \mathbf{S} \mathbf{q}_n = \begin{bmatrix}
  \tilde{\mathbf{S}} & \mathbf{R}
\end{bmatrix} \begin{bmatrix}
  \tilde{\mathbf{q}}_n \\ r_{1n} \\ r_{0n}
\end{bmatrix}.
\end{align}

\subsection{Preprocess the data}

Preprocessing aims at removing the outliers in the raw data. We follow the workflow in \cite{amin2019robust}. The whole process is shown in \Cref{alg:preprocessing}, where we use $\{\mathbf{y}_{\mathrm{SC}_n}\}_{n=1}^\chi$ to represents the set $\{\mathbf{y}_{\mathrm{SC}_1},~\mathbf{y}_{\mathrm{SC}_2},~\cdots,\mathbf{y}_{\mathrm{SC}_n}\}$. The basic idea of this workflow is to find the abnormal peaks of the differentiated raw data. To find those peaks, the searching method is based on comparing the 2 values near the center point. The data patches with outliers are all replaced by their spline approximations. Most of the outliers could be removed by this way.
\begin{algorithm}[tb]
  \caption{The preprocessing applied to the raw data.}
  \label{alg:preprocessing}
  \begin{algorithmic}[1]
    \REQUIRE The raw data $\{\mathbf{y}_{\mathrm{SC}_n}\}_{n=1}^\chi$ for $\chi$ channels.
    \ENSURE The preprocessed data $\{\hat{\mathbf{y}}_{\mathrm{SC}_n}\}_{n=1}^\chi$ for $\chi$ channels.
    % if-then-else
    \FOR{$n$ from $1$ to $\chi$}
    \STATE Find the local positive and negative peaks of the differential data $\dot{\mathbf{y}}_{\mathrm{SC}_n}$, where peaks means the data with a value larger than the around values by 0.1;
    \FORALL{founded peaks}
    \STATE Select 4 points near the peak, the range is limited in 100 points around the peak;
    \STATE Use the spline interpolation of the selected 4 points to replace the 100 points;
    \ENDFOR
    \STATE Perform the 64 order low-pass FIR filter, the cut-off frequency is 3 Hz;
    \ENDFOR
  \end{algorithmic}
\end{algorithm}

\subsection{Solve the deconvolution}

After modeling the data by the aforementioned two methods, we could formulate the $n^{\mathrm{th}}$ channel of the SC data as follows,
\begin{align}
\mathbf{y}_n = \mathcal{F}_{\boldsymbol{\theta}_n} \mathbf{z}_{0} + \mathcal{D}_{\boldsymbol{\theta}_n} \mathbf{u} + \mathbf{S} \mathbf{q}_n + \boldsymbol{\nu}_n,
\end{align}
where $\boldsymbol{\theta}_n = \begin{bmatrix}
\tau_r & \tau_d & \alpha_n
\end{bmatrix}^T$ is the subset of the learnable vector $\boldsymbol{\theta}$, $\mathbf{q}_n$ is the decomposition coefficients of the $n^{\mathrm{th}}$ tonic component and $\boldsymbol{\nu}_n$ is the noise vector for $n^{th}$ channel. Typically $\boldsymbol{\nu}_n$ could be approximated by the Gaussian random noise. For the phasic decomposition, the coefficients, i.e. the neural stimuli $\mathbf{u}$ is shared, and the model parameters are not totally shared across the channels. For the tonic decomposition, the modeling function $\mathbf{C}$ is shared crossing the channels, while the coefficients $\mathbf{q}_n$ are different among different channels.

\begin{algorithm}[!tb]
  \caption{The preprocessing applied to the raw data.}
  \label{alg:deconvolution}
  \begin{algorithmic}[1]
    \REQUIRE The preprocessed multichannel data $\{\hat{\mathbf{y}}_{\mathrm{SC}_n}\}_{n=1}^\chi$, the decomposition coefficients $\mathbf{u}, \{\mathbf{q}_{n}\}_{n=1}^\chi$, and the initialized parameters $\boldsymbol{\theta}$, where $\tau_d \sim U(0.10,~1.4)$, $\tau_r \sim U(1.5,~6.0)$, $\alpha_n \sim U(0.01,~1.0)$, $\mathbf{u} \sim 0$, and $\mathbf{q}_n \sim \mathcal{N}(0.1, 0.02)$.
    \ENSURE The solved coefficients $\mathbf{u}^{(N)}$, ${\{\mathbf{q}_{n}\}_{n=1}^\chi}^{(N)}$, $\boldsymbol{\theta}^{(N)}$.
    % if-then-else
    \FOR {$j$ from $1$ to $30$}
    \STATE Let $\boldsymbol{\theta},~\{\mathbf{q}_{n}\}_{n=1}^{\chi},~\lambda,~\{\mu_n\}_{n=1}^{\chi}$ fixed, use FOCUSS+ to solve $\tilde{\mathbf{u}}^{(j)}$ from \eqref{fml:the:optimization}. Set $\mathbf{u} = \tilde{\mathbf{u}}^{(j)}$;
    \STATE Let $\mathbf{u},~\{\mathbf{q}_{n}\}_{n=1}^{\chi},~\lambda,~\{\mu_n\}_{n=1}^{\chi}$ fixed, solve $\tilde{\boldsymbol{\theta}}^{(j)}$ from \eqref{fml:the:optimization} by the interior point method. Set $\boldsymbol{\theta} = \tilde{\boldsymbol{\theta}}^{(j)}$;
    \STATE Find the peaks of the solved $\mathbf{u}$, and generate the basis matrix $\mathbf{S}$ by \eqref{fml:tonic-basis-1} and \eqref{fml:tonic-basis-2}.
    \STATE Let $\boldsymbol{\theta},~\mathbf{u},~\lambda,~\{\mu_n\}_{n=1}^{\chi}$ fixed. For any $n$, use interior point method to solve $\{\mathbf{q}_{n}\}_{n=1}^{\chi}$ from \eqref{fml:the:optimization};
    \ENDFOR
    \STATE Initialize $\hat{\boldsymbol{\theta}}^{(0)}=\boldsymbol{\theta}$, $\hat{\mathbf{u}}^{(0)}=\mathbf{u}$, ${\{\hat{\mathbf{q}}_{n}\}_{n=1}^{\chi}}^{(0)}=\{\mathbf{q}_{n}\}_{n=1}^{\chi}$;
    \FOR{$i$ from $0$ until converge}
    \STATE Let $\hat{\lambda}^{(i)(0)} = 2 \times 10^{-3}$,
    \FOR{$m$ from $1$ until converge}
    \STATE Let $\boldsymbol{\theta},~\{\mathbf{q}_{n}\}_{n=1}^{\chi},\lambda = \hat{\lambda}^{(i)(m-1)},~\{\mu_n\}_{n=1}^{\chi}$ fixed, use FOCUSS+ to solve $\hat{\mathbf{u}}^{(i)(m)}$ from \eqref{fml:the:second-phasic}. Set $\mathbf{u} = \hat{\mathbf{u}}^{(i)(m)}$;
    \STATE Let $\mathbf{u},~\boldsymbol{\theta},~\{\mathbf{q}_{n}\}_{n=1}^{\chi},~\{\mu_n\}_{n=1}^{\chi}$ fixed, use GCV to solve $\hat{\lambda}^{(i)(m)}$. Set $\lambda = \hat{\lambda}^{(i)(m)}$;
    \ENDFOR
    \STATE Set $\hat{\mathbf{u}}^{(i)} = \hat{\mathbf{u}}^{(i)(m)}$.
    \STATE Let $\mathbf{u},~\{\mathbf{q}_{n}\}_{n=1}^{\chi},~\lambda,~\{\mu_n\}_{n=1}^{\chi}$ fixed, use the interior point method to solve $\hat{\boldsymbol{\theta}}^{(i)}$ from \eqref{fml:the:second-phasic}. Set $\boldsymbol{\theta} = \hat{\boldsymbol{\theta}}^{(i)}$;
    \STATE Let $\hat{\mu}^{(i)(0)} = 2 \times 10^{-3}$,
    \STATE Find the peaks of the solved $\hat{\mathbf{u}}^{(i)}$, and generate the basis matrix $\mathbf{S}$ by \eqref{fml:tonic-basis-1} and \eqref{fml:tonic-basis-2}.
    \FOR{$m$ from $1$ until converge}
    \STATE Let $\mathbf{u},~\boldsymbol{\theta},~\lambda,~\mu = \hat{\mu}^{(i)(m-1)}$ fixed, use interior point method to solve ${\{\hat{\mathbf{q}}_{n}\}_{n=1}^{\chi}}^{(i)(m)}$ from \eqref{fml:the:second-tonic}. Set $\{\mathbf{q}_{n}\}_{n=1}^{\chi} = {\{\hat{\mathbf{q}}_{n}\}_{n=1}^{\chi}}^{(i)(m)}$;
    \STATE Let $\mathbf{u},~\boldsymbol{\theta},~\{\mathbf{q}_{n}\}_{n=1}^{\chi},~\lambda$ fixed, use GCV to solve $\hat{\mu}^{(i)(m)}$. Set $\mu = \hat{\mu}^{(i)(m)}$;
    \ENDFOR
    \STATE Set ${\{\hat{\mathbf{q}}_{n}\}_{n=1}^{\chi}}^{(i)} =  {\{\hat{\mathbf{q}}_{n}\}_{n=1}^{\chi}}^{(i)(m)}$.
    \ENDFOR
  \end{algorithmic}
\end{algorithm}

The multichannel deconvolution with the tonic separation could be formulated by the following constrained joint optimization problem,
\begin{subequations} \label{fml:the:optimization}
  \renewcommand{\theequation}
  {\theparentequation-\arabic{equation}}
  \begin{align}
  \argmin\limits_{\substack{\boldsymbol{\theta},~\mathbf{u}, \\ \{\mathbf{q}_n\}_{n=1}^{\chi}, \\ \lambda,~\{\mu_n\}_{n=1}^{\chi}}} & \frac{1}{\chi} \left( \sum_{n=1}^{\chi} \mathcal{J}_n + \mu_n \lVert \mathbf{q}_n \rVert_2^2 \right) + \lambda \lVert \mathbf{u} \rVert^p_p, \\
  \mathrm{s.t.}~&\mathcal{J}_n = \lVert \mathbf{y}_n - \mathcal{F}_{\boldsymbol{\theta}_n} \mathbf{z}_{0} - \mathcal{D}_{\boldsymbol{\theta}_n} \mathbf{u} - \mathbf{S} \mathbf{q}_n \rVert^2_2, \label{fml:loss-channel}\\
  & \boldsymbol{\Gamma} \boldsymbol{\theta} \preccurlyeq \mathbf{b},~ \mathbf{u} \succcurlyeq \mathbf{0} \label{fml:cons-phasic}\\
  & \forall~n,~\mathbf{q}_n \succcurlyeq \mathbf{0},~\mathbf{S}\mathbf{q}_n \preccurlyeq \mathbf{y}_n, \label{fml:cons-tonic}
  \end{align}
\end{subequations}
where \eqref{fml:loss-channel} represents the loss function for the $n^{\mathrm{th}}$ channel, \eqref{fml:cons-phasic} is the constraint of the phasic decomposition, and \eqref{fml:cons-tonic} is the constraint of the tonic extraction. $\{\mathbf{q}_n\}_{n=1}^{\chi}$ and $\{\mu_n\}_{n=1}^{\chi}$ denote the set of all channels of tonic components and their corresponding regularization ratios respectively. In \eqref{fml:cons-phasic}, the Tikhonov matrix $\boldsymbol{\Gamma}$ and the boundary vector $\mathbf{b}$ are defined as
\begin{subequations} \label{fml:the:constraint}
  \renewcommand{\theequation}
  {\theparentequation-\arabic{equation}}
  \begin{align}
  \boldsymbol{\Gamma} &= \begin{bmatrix}
  1 & -1 & 0 & 0 & \cdots & 0 & 0 \\
  0 & 0 & 1 & -1 & \cdots & 0 & 0 \\
  \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\
  0 & 0 & 0 & 0 & \cdots & 1 & -1
  \end{bmatrix}, \\
  \mathbf{b} &=
    \begin{array}{ccccc}
      \big[1.4 & -0.1 & 6.0 & -1.5 & 100.0 \\
      & -0.01 & \cdots & 100.0 & -0.01 \big]^T
    \end{array}
  \end{align}
\end{subequations}

\begin{figure}[!tb]
  \centering
  \subfigure[]{\includegraphics[width=0.99\columnwidth]{p11_middle_phalanx_3} \label{fig:middle_phalanx}}
  \subfigure[]{\includegraphics[width=0.99\columnwidth]{p11_medial_plantar_3} \label{fig:medial_plantar}}
  \subfigure[]{\includegraphics[width=0.99\columnwidth]{p11_thenar_3} \label{fig:thenar}}
  \subfigure[]{\includegraphics[width=0.99\columnwidth]{p11_stimuli_3} \label{fig:stimuli}}
  \DeclareGraphicsExtensions.
  \caption{The reconstructed signals for the data collected from the $11^{\mathrm{th}}$ participant. (a) The volar middle phalanx data. (b) The medial plantar surface data. (c) The thenar/hypothenar data. (d) The solved neural stimuli.} \label{fig:results}
\end{figure}

Conventionally, it is difficult to decide the proper regularization coefficients $\lambda,~\mu_n$. In our work, we take the optimization of the coefficients into the consideration. The optimization is based on Focal Underdetermined System Solver (FOCUSS+) algorithm~\cite{murray2005visual}, while the estimation of the tunable regularization coefficients is based on  generalized cross-validation (GCV) method~\cite{zdunek2008improved}.

To solve the joint problem in \eqref{fml:the:optimization}, we need to divide the problem into two sub-problems and solve them alternatively. This technique is called concurrent coordinate descent method. The whole algorithm is discussed in \Cref{alg:deconvolution}. The algorithm has 2 phases. In both phases, the tonic component and the phasic component are solved alternatively, the only difference is the regularization method. In the first phase, we make the initialization ratio fixed to simplify the algorithm. This phase is used for finding a good initialization. In the beginning of the initialization phase, we use conventional method to generate the tonic component bases, where the intervals between two basis are fixed.

\begin{figure}[!tb]
  \centering
  \subfigure[]{\includegraphics[width=0.99\columnwidth]{p12_middle_phalanx_3} \label{fig:12middle_phalanx}}
  \subfigure[]{\includegraphics[width=0.99\columnwidth]{p12_medial_plantar_3} \label{fig:12medial_plantar}}
  \subfigure[]{\includegraphics[width=0.99\columnwidth]{p12_thenar_3} \label{fig:12thenar}}
  \subfigure[]{\includegraphics[width=0.99\columnwidth]{p12_stimuli_3} \label{fig:12stimuli}}
  \DeclareGraphicsExtensions.
  \caption{The reconstructed signals for the data collected from the $12^{\mathrm{th}}$ participant. (a) The volar middle phalanx data. (b) The medial plantar surface data. (c) The thenar/hypothenar data. (d) The solved neural stimuli.} \label{fig:12results}
\end{figure}

In the second phase, the optimization could be divided into an outer loop and an inner loop. In the outer loop, we solve the modeling parameters $\boldsymbol{\theta}$. The deconvolution and the tonic extraction is performed in the inner loop. Particularly, given the solution of $\{\mathbf{q}_n\}_{n=1}^{\chi}$, when solving the tonic component, for any $n$, we need to solve the following constrained quadratic problem,
\begin{subequations} \label{fml:the:second-tonic}
  \renewcommand{\theequation}
  {\theparentequation-\arabic{equation}}
  \begin{align}
    \argmin\limits_{\mathbf{q}_n,~\mu_n} & \lVert \mathbf{s}_n - \mathbf{S} \mathbf{q}_n \rVert^2_2 + \mu_n \lVert \mathbf{q}_n \rVert_2^2, \\
    \mathrm{s.t.}~& \mathbf{s}_n = \mathbf{y}_n - \mathcal{F}_{\boldsymbol{\theta}_n} \mathbf{z}_{0} - \mathcal{D}_{\boldsymbol{\theta}_n} \mathbf{u}, \label{fml:the:second-tonic-obj}\\
    & \forall~n,~\mathbf{q}_n \succcurlyeq \mathbf{0},~\mathbf{S}\mathbf{q}_n \preccurlyeq \mathbf{y}_n,
  \end{align}
\end{subequations}
%
where $\mathbf{S}\mathbf{q}_n$ is upper bounded by the signal $\mathbf{y}_n$. We do not set the upper bound by $\mathbf{s}_n$, because we do not require the positive residual for the solution of the phasic component.

When solving the phasic component, we make the tonic component fixed, and solve the following problem,
\begin{subequations} \label{fml:the:second-phasic}
  \renewcommand{\theequation}
  {\theparentequation-\arabic{equation}}
  \begin{align}
  \argmin\limits_{\boldsymbol{\theta},~\mathbf{u},~\lambda} & \frac{1}{\chi}  \left( \sum_{n=1}^{\chi} \lVert \mathbf{p}_n - \mathcal{F}_{\boldsymbol{\theta}_n} \mathbf{z}_{0} - \mathcal{D}_{\boldsymbol{\theta}_n} \rVert^2_2 \right) + \lambda \lVert \mathbf{u} \rVert_p^p, \\
  \mathrm{s.t.}~& \mathbf{p}_n = \mathbf{y}_n -  \mathbf{S} \mathbf{q}_n, \\
  & \boldsymbol{\Gamma} \boldsymbol{\theta} \preccurlyeq \mathbf{b},~ \mathbf{u} \succcurlyeq \mathbf{0}.
  \end{align}
\end{subequations}

\section{Results}

The dataset used in this work is called \texttt{PsPM-SCRV10}\cite{bach2014pspm}, where ``\texttt{PsPM}'' is the abbreviation of the organization ``Psycho-Physiological Modelling'', and ``\texttt{SCRV\_10}'' is the symbolic name of the dataset. This dataset is collected from an experiment. 26 volunteers (12 males and 14 females) are invited for participating the test. The researchers produce a series white noise bursts as auditory stimulations. In response, the participants are required to press a food pedal every time they hear the burst. The data is recorded by Cambridge Electronic Design (CED) spike software.

According to the type of the data, the dataset could be divided into two parts. The first part is called ``cogent''. It contains the record about the participants' reactions including which pedal they press and how long the pedal is pressed. This part is not related to our work. The second part is called ``spikes''. It is a collection of skin conductance response (SCR) measurements. We only use the following parts of the data:

\begin{enumerate}
  \item Marker denoting timings of the auditory stimulations;
  \item Skin conductance on the thenar/hypothenar of the non-dominant hand;
  \item Skin conductance on the volar middle phalanx of the dominant 2nd/3rd finger;
  \item Skin conductance on the medial plantar surface of the non-dominant foot;
\end{enumerate}

Since each volunteer could provides 3 SC data, we construct our model as a 3-channel one. The marker denoting the auditory stimulations are used as the ground truth for estimating our results.

To simplify the calculation, we downsample the data by setting $T_y=1\rm{s}$. The bin sizes of the stimuli $T_u$ are configured as 4s. For the tonic components, the knot size $\kappa$ is configured as $4s$, and the relatively delay time of the $T_{ds}$ is 25s. We take the results of the participant 11 and participant 12 as examples, where we extract the data from 200s to 600s. The results are shown in \Cref{fig:results} and \Cref{fig:12results} respectively. In \Cref{fig:stimuli} and \Cref{fig:12stimuli}, we draw the ground truth of the stimulation as gray bars and the solved neural stimuli as purple pins. The results show that the stimuli are always a little bit later than the stimulation. This phenomenon is reasonable, because there may be a latency caused by the physiological reactions. The solved parameters $\boldsymbol{\theta}$ are shown in \Cref{tab:theta}, where $\tau_r$ and $\tau_d$ are the rise and decay times, and $\alpha_2$ and $\alpha_3$ represent the relatively attenuation rates of the medial plantar surface data and the thenar/hypothenar data compared to the volar middle phalax data.

\begin{table}[h]
  \centering
  \caption{The solve paramters $\boldsymbol{\theta}$ for the shown examples.} \label{tab:theta}
  \begin{tabular}{|c|c|c|c|c|}
    \hline
    Participant & $\tau_r$ & $\tau_d$ & $\alpha_{2}$ & $\alpha_{3}$ \\
    \hline
    11 & 0.2799s & 3.2583s & 0.4540 & 0.3844\\ \hline
    12 & 0.6752s & 4.8733s & 0.4780 & 0.5492\\
    \hline
  \end{tabular}
\end{table}

\section{Discussion}

The solutions in \Cref{fig:results} and \Cref{fig:12results} show a high fitness between the reconstructed signals and the raw data. By utilizing the multi-channel data, the influence of the noise is eliminated. For example, although there is an unfitness near 520s in \Cref{fig:middle_phalanx}, the other two channels could still achieve good fitness and suggest the correct impulse in the neural stimuli.

\begin{figure}[!tb]
  \centering
  \subfigure[]{\includegraphics[width=0.6\columnwidth]{basis-c0} \label{fig:bases-c0}}
  \subfigure[]{\includegraphics[width=0.6\columnwidth]{basis-c} \label{fig:bases-c}}
  \DeclareGraphicsExtensions.
  \caption{A comparison between the bases generated by different methods. This experiment is performed on participant 11. We use the same knot size for both methods. (a) Conentional method, where the interval between bases is fixed. (b) Our method, where the positions of bases are relative to the solve neural stimuli.} \label{fig:bases}
\end{figure}

\begin{figure}[!tb]
  \centering
  \subfigure[]{\includegraphics[width=0.99\columnwidth]{thenar_bad}}
  \subfigure[]{\includegraphics[width=0.99\columnwidth]{thenar_3_good}}
  \DeclareGraphicsExtensions.
  \caption{Reconstructed signal for the thenar/hypothenar data of the participant 11 with different tonic extraction methods. (a) Conventional tonic extraction. (b) Our tonic extraction.} \label{fig:comptonic}
\end{figure}

Experiments show that the fitness of the final solution is highly dependent on the quality of the solved tonic component. In previous works~\cite{greco2015cvxeda,amin2019tonic}, the B-spline bases are sampled by a fixed interval. In these methods, it is very tricky for determining how long the interval should be. If the interval is too small, there would be a deformation for the solved phasic component. If too large, the solved tonic component would be over-smoothed. According to \eqref{fml:the:ode}, since the neural stimuli is a series of sparse impulses, the phasic component is modeled by a series of peaks. According to \eqref{fml:tonic-conv}, the tonic component could be also viewed as a combination of B-spline peaks. Empirically, we find that when each peak of the tonic component appears a little bit later than the impulses of neural stimuli, the solution could reach the highest fitness. Inspired by this inspection, we propose the adaptive B-spline bases in \eqref{fml:tonic-basis-1}. In \Cref{fig:bases}, we compare the bases generated by the conventional method and our method when the shape of the bases are configured same. In \Cref{fig:bases-c}, we use gray lines to represent the location of the solved neural stimuli. We could find that the bases generated by our method is more sparse, and the position of each basis is related to the appearance of neural stimuli.

In \Cref{fig:comptonic}, we take the thenar/hypothenar data as an example. We perform two experiments with all the same configurations except for the tonic extraction methods. The results are produced by the fixed bases in \Cref{fig:bases-c0} and the adaptive bases in \Cref{fig:bases-c} respectively. With our method, the fitness is significantly improved in the region marked by red circles. In the conventional method, because the interval between bases is fixed, the marked tonic component peak is almost near the end of one phasic component peak. In comparison, our method ensures that the tonic component peak is nearly centered in the range of one phasic component peak.

Currently, we are using a fixed delay $T_{ds}$ for determining the relative position of the tonic bases, this may not be a best configuration. In the future work, we could make the delay for each basis learnable, and find the optimal relative delay by the algorithm.

\section{Conclusion}

In this work, we try to combine the tonic component extraction and the multi-channel phasic component deconvolution together. The problem is formulated as a joint problem for solving both components. To solve this problem, we use the concurrent coordinate descent method. For the phasic component, we formulate the model by a state-space representation, and use FOCUSS+ to solve the neural stimuli with sparsity regularization. In comparison, the tonic component is modeled by the decomposition of B-spline functions, where the positions of the bases are adaptive. We use the interior point method to solve the constrained quadratic problem. The regularization ratio for both components are estimated by the GCV method. The final results show that the phasic component and tonic component are separated, and the neural stimuli is well-regularized by the sparsity.

\bibliographystyle{ieeetr}
\bibliography{ref}

\end{document}


\documentclass[12pt,letterpaper]{article}

\usepackage{amsmath, amsthm, amsfonts, amssymb}
\usepackage{microtype, parskip, graphicx}
\usepackage[comma,numbers,sort&compress]{natbib}
\usepackage{lineno}
\usepackage{longtable}
\usepackage{docmute}
\usepackage{caption, subcaption, multirow, morefloats, rotating}
\usepackage{wrapfig}
\usepackage{hyperref}

\frenchspacing

\begin{document}

\section{Results}

\subsection{Posterior predictive results}
% why do i want to show all these graphs?
%   demonstrate how well the model recapitulates the observed data
%   if the model simulates datasets like the one we observed
%     then we can be more confident in our inferences
%   PPCs by group reveal even more about the fit of our model
%     are some time bins better predicted by others?
%     well predicted time bins indicate congruence with model assumptions
%     poorly predicted time bins indicate incongruence with model assumptions
%       something different is happening in these bins that requires explanation

Overall, the expected taxonomic diversity of geological units for each taxonomic group is adequately described by the fitted models, where adequacy means that the posterior predictive distribution of our models resemble the empirical data. While there are aspects of misfit when considering their entire distribution, the aspects of the distribution which are critical to our analysis are well fit by our models.

A point comparison between the observed mean geologic unit diversities and the posterior predictive distributions for each taxonomic group indicate that our fitted models are able to recapitulate this aspect of the observed data (Fig. \ref{fig:ppc_mean}). This result is reassuring because our model is specifically a model of expected geologic unit diversity, and a good fit to mean diversity means our model fits are at least capturing this basic aspect of the data.
% overall ppc-s
% mean
\begin{figure}[ht]
  \centering
  \includegraphics[width=\textwidth,height=0.5\textheight,keepaspectratio=true]{figure/ppc_mean_diversity}
  \caption{Posterior predictive results comparing the observed mean diversity of a geological unit for each of the studied taxonomic groups to a distribution of 1000 estimates from datasets simulated from the posterior predictive distribution of our models. Model adequacy is determined by how similar the posterior predictive distribution is to the observed value. In all cases, our models appear able to reproduce to observed means.}
  \label{fig:ppc_mean}
\end{figure}

Comparison of the observed standard deviation estimates for each of the taxonomic datasets to the posterior predictive distributions of our model fits show that our model is slightly over estimating the scale of our data, though not to a necessarily concerning degree (Fig. \ref{fig:ppc_sd}). Count data can frequently be over dispersed and have a standard deviation to mean ratio greater than 1; this reality is the reason we chose to use a truncated Negative-Binomial as opposed to a Poisson distribution because the addition of a second parameter allows us to model this overdispersion \citep{ARM,BDA3}. While our model is not too different from the data, there is room for improvement in modeling the actual dispersion of geologic unit diversity.
% sd
\begin{figure}[ht]
  \centering
  \includegraphics[width=\textwidth,height=0.5\textheight,keepaspectratio=true]{figure/ppc_sd_diversity}
  \caption{Posterior predictive results comparing the observed standard deviation diversity of a geological unit for each of the studied taxonomic groups to a distribution of 1000 estimates from datasets simulated from the posterior predictive distribution of our models. Model adequacy is determined by how similar the posterior predictive distribution is to the observed value. In all cases, our models appear able to reproduce to observed standard deviations.}
  \label{fig:ppc_sd}
\end{figure}

Comparisons of the empirical probability density functions for each of the taxonomic groups to the posterior predictive distribution of density functions generated by our model fits indicate that our model is very capable of recapitulating the observed data for nearly its entire range (Fig. \ref{fig:ppc_dens}). There are, however, minor but noticable differences between the posterior predictive distribution and the empirical data. For example, the posterior predictive distribution for Anthozoa slightly underestimates the number of units with diversity approximately 8. A similar underestimate is observable when comparing the posterior predictive distibution to the empirical data at unit diversity of approximately 11, and Brachiopoda at unit diversity of approximately 11. However, the posterior predictive distributions of our models fit the data well in nearly all cases, indicating that our model is potentially capturing some aspects of the data generating process.
\begin{figure}[ht]
  \centering
  \includegraphics[width=\textwidth,height=0.5\textheight,keepaspectratio=true]{figure/ppc_dens_zoom_diversity}
  \caption{Posterior predictive results comparing the empirical probability density of a geological unit for each of the studied taxonomic groups to a distribution of 1000 probability densities from datasets simulated from the posterior predictive distribution of our models. Model adequacy is determined by how similar the posterior predictive distribution is to the observed value. In all cases, our models appear able to almost reproduce to observed ecdf-s.}
  \label{fig:ppc_dens}
\end{figure}

%The cummulative distribution function is description of the normalized rank order accumulation of the data.
%\begin{figure}[ht]
%  \centering
%  \includegraphics[width=\textwidth,height=0.5\textheight,keepaspectratio=true]{figure/ppc_ecdf_diversity}
%  \caption{Posterior predictive results comparing the empirical cummulative distribution function (ecdf) of a geological unit for each of the studied taxonomic groups to a distribution of 1000 estimates from datasets simulated from the posterior predictive distribution of our models. Model adequacy is determined by how similar the posterior predictive distribution is to the observed value. In all cases, our models appear able to almost reproduce to observed ecdf-s.}
%  \label{fig:ppc_ecdf}
%\end{figure}

\subsection{Estimated versus observed unit diversity}
% what is the point of this section?
%   graph unit diversity by time bin
%   compare to estimate from model
%   this is like the ppc for group mean, but inside-out
%   if model has poor estimates for time bin
%     that bin is not like what we would expect based on our model
%   if we have good estimates for all bins
%     then we need to look at covariates to see if there are any switch patterns

% time series graph
% what does this graph represent?
%   geological unit diversity counts at time bins
%   comparison to posterior predictive mean est with 80CI
Comparison between observed unit diversity over time and our models' estimates of mean unit diversity for those time steps reveals broad congruence (Fig. \ref{fig:time_div}). At no point does our model have a spurious or unrealistic estimate of mean geologic unit taxonomic diversity. This congruence gives us confidence in estimating the probability of the Hirnantian (1) having lower average unit diversity than the times directly before and after, and (2) having lower average unit diversity than the average unit diversity of the late Ordovician and the Silurian.
\begin{figure}[ht]
  \centering
  \includegraphics[width=\textwidth,height=0.5\textheight,keepaspectratio=true]{figure/unitdiv_time_diversity}
  \caption{Geological unit diversity though time and the expected diversity (with 80\% credible interval) as estimated from our models. Unit diversity is presented as partially transparent points and our jittered in the y-axis to improve readability. Point size is proportional to the number of units in that interval that have identical unit diversity. The dashed grey line corresponds to the onset of the Hirnantian geological stage, while the dashed-dotted grey line corresponds to the end of the Ordovician epoch and the start of the Silurian epoch.}
  \label{fig:time_div}
\end{figure}

We tested the probability that geologic units during the Hirnantian have lower expected unit diversity than the times immediately before and after by testing all advacent time bins (Fig. \ref{fig:diff_div}). For each adjacent time bins, we estimated the probability that the earlier time bin (time \(t\)) has a greater expected duration than the later time bin (time \(t + 1\)). Our analysis demonstrates that, for most comparisons, the expected unit diversity of Hirnantian time bin is not expected with high probability to be different from the time units immediately preceeding and following it.

\begin{figure}[ht]
  \centering
  \includegraphics[width=\textwidth,height=0.5\textheight,keepaspectratio=true]{figure/unitdiv_diff_diversity}
  \caption{Probability that our estimate of mean unit diversity at time \(t\) is greater than the estimate at time \(t + 1\). The dashed grey horizontal lines correspond to probability of 0.9 and 0.1; these are the thresholds we chose as indicating if a pair-wise difference is potentially larger (or smaller) than no-difference (\(P = 0.5\)), and worthy of further inspection.}
  \label{fig:diff_div}
\end{figure}

% covariance of effect change?

\subsection{Effects of geological covariates on estimated diversity}
% what is the point of this section?
%   we've estimated the effect of multiple covariates
%   these estimates are allowed to vary through time
%     temporal structure is taken into account
%   what is the general relationship? what happens during hirnantian?
%     if all units have the same relationship
%       and all good bin estimates
%         there is no change in unit diversity though time
%         there is no change in geol properties that affect unit diversity
%       and some bad bin estimates
%         our model can not explain these bins; something is going on here
%     if some units have diff relationships (sign change, effect loss)
%       and all good bin estimates
%         we capture the changing relationship between preservational context and observed diversity
%       and some bad bin estimates
%         our model can not explain these bins; something is going on here

% time series graph
% what does this graph represent?
%   effect of covariate on expected count over time
%   violin shows full posterior estimate
%   pointrange shows mean with 80CI
As stated earlier, for all taxonomic groups the intercept term is an estimate of the expected (log) diversity of geologic unit diversity with mean thickness, area, latitude, and a purely non-dolomitic carbonate lithology. The effects of thickness, area, and latitude correspond to the expected change in (log) geologic unit diversity per change of the covariate value in units of standard deviations. The effects of dolomite, fine and coarse siliciclastic correspond to the change associated with unit change to the logratio representing the lithological composition of interest \citep{Hron2012}.

Estimates of covariates effects over time demonstrate a gradual shift in effects over time and not a sudden shift during the Hirnantian or between the Ordovician and the Silurian (Fig. \ref{fig:time_cov}). Interestingly, the covariate that may demonstrate the biggest pattern associated with the Hirnantian is the effect of geologic unit areal extent which appears to decrease in effect for Bivalvia, Gastropoda, and Mollusca.
\begin{figure}[ht]
  \centering
  \includegraphics[width=\textwidth,height=0.5\textheight,keepaspectratio=true]{figure/cov_time_diversity}
  \caption{Estimates of all estimated covariate effect time series for each of the analyzed taxonomic groups, including intercept estimates. Points represent mean estimate along with a 80\% credible interval. The black horizontal line corresponds to no effect. Points are plotted at the mid-point of the discrete time interval.}
  \label{fig:time_cov}
\end{figure}

Similar to our earlier comparison of expected geologic unit diversity, we tested if any of the covariate effects estimated for one time bin (time \(t\)) were greater than the estimates from the following time bin (time \(t + 1\)). We find no strong evidence that the Hirnantian time bin is significantly different from the time bins immediately before and after (Fig. \ref{fig:diff_cov}). %There is only one example of a time unit comparison where we might expect a significant difference and that is for the estimated effect of areal extent on (log) geologic unit diversity between the 7th and 8th time bins. This is further evidence of gradual changes to the relationship between the properties of a geologic unit and its expected taxonomic diversity.
There are very few examples of one time bin having significantly different expected unit diversity than the one proceeding it. The intercept of our model fit to the Bivalvia dataset is expected to be greater during the second time unit than the third, though this difference is not associated with the Hirnantian time unit and thus is not very relevant to the thrust of our analysis. Similarly, we estimated a potentially significant difference the estimated effect of area in Mollusca, where the estimate for the seventh time unit is greater than the estimate for the eighth time unit.
\begin{figure}[ht]
  \centering
  \includegraphics[width=\textwidth,height=0.5\textheight,keepaspectratio=true]{figure/cov_diff_diversity}
  \caption{Probability that a parameter estimate at time \(t\) is greater than the estimate at time \(t + 1\). The dashed grey horizontal lines correspond to probability of 0.9 and 0.1; these are the thresholds we chose as indicating if a pair-wise difference is potentially larger (or smaller) than no-difference (\(P = 0.5\)), and worthy of further inspection.}
  \label{fig:diff_cov}
\end{figure}

Of particular interest is if the Hirnantian has a lower expected unit diversity than the average of the Ordovician or Silurian. Additionally, we are interested in if the covariate estimates for the Hirnantian are different than those estimated for the rest of the Ordovician or Silurian. Here we calculate the probability that the expected unit diversity of the Hirnantian is less than the averages for the Ordovician or Silurian and compare those estimates to the probabilities that the covariate effects are less than the averages for the Ordovician or Silurian (Fig. \ref{fig:pvalue}). We find that in most cases there is no strong evidence (probability \(> 0.9\)) for the Hirnantian being significantly different from the averages of either the Ordovician or the Silurian. However, there is weak evidence (probability \(> 0.75\)) for some differences between the Hirnantian and the Ordovician or Silurian.

All of the following results are supported with only weak evidence and are thus of interest for future study of lithological and diversity differences associated with the Hirnantian.

For Brachiopoda, Gastropoda, and Mollusca the average diversity of geologic units in the Orodovician is estimated with weak support to be greater than the expected unit diversity of the Hirnantian. We find weak support for a lower effect of dolomite composition on Anthozoan unit diversity during the Hirnantian than the average of the Ordovician. Similarly, we find weak support for a lower intercept term and the effect of coarse siliciclastics for Brachiopoda, Gastropoda, Mollusca, and Trilobita in the Hirnantian than the average of the Ordovician. The intercept is also the effect of being a purely non-dolomitic carbonate unit. We also find marginal support for the effect of area on unit diversity of Bivalvia, Gastropoda, and Mollusca being greater for the average of the Ordovician than those units from the Hirnantian.

We do find evidence that Brachiopods are expected to have greater unit diversity during the Silurian than during the Hirnantian. This result is one of the strongest support results from this study. Bivalvia and Mollusca intercept weak evidence expected to be greater in Hirnantian than average Silurian. This is also the effect of being a purely non-dolomitic carbonate unit. Bivalvia and Mollusca are estimated to have a lower effect of geologic unit area during the Hirnantian than the average of the Silurian. Finally, there is marginal evidence for the intercept estimate for Brachiopoda during the Hirnantian is expected to be lower than the average estimated intercept for the Silurian.
\begin{figure}[ht]
  \centering
  \includegraphics[width=\textwidth,height=0.5\textheight,keepaspectratio=true]{figure/compare_pval_diversity}
  \caption{Scatterplot of the estimated probability that geological unit diversity is lower during the Hirnantian than either the Ordovician (left facet) or the Silurian (right facet) vs the estimated probability that a covariate estimate is lower during the Hirnantian than either the Ordovician or the Silurian. For each of the taxonomic groups there is only one estimate for the probability of difference in diversity, but there are six probability estimates for each of the covariate effect parameters. }
  \label{fig:pvalue}
\end{figure}

\end{document}


\documentclass{article}
\usepackage{fullpage}
\usepackage{geometry}
\usepackage{fancyhdr}
\usepackage{amsmath, amsthm, amssymb}
\usepackage{graphicx}
\usepackage{hyperref}
\usepackage{parskip}
\newcommand{\var}[1]{\emph{#1}}

\title{Abstract Syntax Trees\\ for Khan Exercises}
\begin{document}
\maketitle
\begin{verbatim}
Authors: Jeff Dyer <jeffdyer@acm.org>
         Mathias Hiron <mathias@khanacademy.org>

History: 04-Jan-2012, second draft, reviewed by Mathias
         03-Jan-2012, first draft, unreviewed
\end{verbatim}

\section{Overview}

In this note we define an abstract syntax tree (AST) form for encoding information used in Khan
exercises. We also define a mapping from ASTs to LaTeX formatted strings. ASTs and LaTeX have wide
applicability in the Khan exercise framework, and so by defining and implementing a core library
for handling them we avoid duplication of effort and gain the benefits normally associated by
software reuse.

\section{Abstract Syntax Trees}

We use trees to represent expressions, equations, and other structured data that underlies Khan
exercises. The trees consist of nodes containing an operator string and zero or more child nodes.

The nodes are encoded as JavaScript objects of the form:

\begin{quote}
\verb'{'{\bf op}{\tt :} {\em opstr}{\tt ,} {\bf args}{\tt :} {\tt [}$nd1${\tt ,}\dots $ndN${\tt ]}$\verb'}'$
\end{quote}

where $opstr$ is an \emph{operator string} and $nd1, \dots ndN$ is a sequence of child nodes.

An \emph{operator string} is a string value that, along with the number of arguments, distinguishes
one node kind from another. AST nodes get their meaning from the parsing, formatting and evaluating
procedures that construct and interpret them, and otherwise have no special meaning within the AST
definition and implementation.

Primitive (non-object) JavaScript values are used to represent literal numbers and strings
in the AST. Variables are encoded as nodes with the unary ``var'' operator with a literal string
operand.

We define a core set of operators here, but this set is not fixed. As new exercises require new
operators, additional strings to be contrived to represent those operators, and interpreters can be
defined to format and evaluate nodes with those operators.

\section{Predefined Operators}

The following set of operators are implemented by the AST module (see below). Additional operators
may be added to this core set over time, and others may be added by problem specific utility
modules.

\vspace{20pt}

\begin{tabular}{ | l | l | p{10cm} |}
\hline
{\bf Operator}& {\bf Sample}   & {\bf Comment} \\ \hline
\verb'+'      & $x+y$          & addition, unary plus\\ \hline
\verb'-'      & $x-y$          & subtraction, unary minus\\ \hline
\verb'times'  & $x \times y$   & multiplication\\ \hline
\verb'div'    & $x \div y$     & division\\ \hline
\verb'frac'   & $\frac{x}{y}$  & fraction\\ \hline
\verb'^'      & $x^y$          & exponent\\ \hline
\verb'sqrt'   & $\sqrt{x}$     & unary square root\\ \hline
\verb'sqrt'   & $\sqrt[n]{x}$  & binary square root\\ \hline
\verb'pm'     & $x \pm y$      & plus or minus, unary plus or minus\\ \hline
\verb'sin'    & $\sin{\theta}$ & sine\\ \hline
\verb'cos'    & $\cos{\theta}$ & cosine\\ \hline
\verb'tan'    & $\tan{\theta}$ & tangent\\ \hline
\verb'sec'    & $\sec{\theta}$ & secant\\ \hline
\verb'ln'     & $\ln{x}$       & natural logarithm\\ \hline
\verb'()'     & $(expr)$       & parenthesis\\ \hline
\verb'var'    & $\var{x}$      & variable (unknown)\\ \hline
\end{tabular}

\pagebreak

\section{LaTeX Translation}

In addition to the internal AST form, we define a mapping from ASTs to LaTeX. The following table
shows the mapping between the AST nodes and LaTeX.

\vspace{20pt}

\begin{tabular}{ | l | l |}
\hline
{\bf AST}                           & {\bf LaTeX} \\ \hline
\verb'{op: "+", args: [1, 2]}'      & \verb'1+2' \\ \hline
\verb'{op: "-", args: [1, 2]}'      & \verb'1-2' \\ \hline
\verb'{op: "times", args: [1, 2]}'  & \verb'1 \times 2' \\ \hline
\verb'{op: "div", args: [1, 2]}'    & \verb'1 \div 2' \\ \hline
\verb'{op: "frac", args: [1, 2]}'   & \verb'\frac{1}{2}' \\ \hline
\verb'{op: "^", args: [1, 2]}'      & \verb'1^2' \\ \hline
\verb'{op: "sqrt", args: [9]}'      & \verb'\sqrt{9}' \\ \hline
\verb'{op: "sqrt", args: [3, 8]}'   & \verb'\sqrt[3]{8}' \\ \hline
\verb'{op: "pm", args: [2, 1]}'     & \verb'2 \pm 1' \\ \hline
\verb'{op: "sin", args: [0]}'       & \verb'\sin{0}' \\ \hline
\verb'{op: "cos", args: [0]}'       & \verb'\cos{0}' \\ \hline
\verb'{op: "tan", args: [0]}'       & \verb'\tan{0}' \\ \hline
\verb'{op: "sec", args: [0]}'       & \verb'\sec{0}' \\ \hline
\verb'{op: "ln", args: [1]}'        & \verb'\ln{1}' \\ \hline
\verb'{op: "()", args: [10]}'       & \verb'(10)' \\ \hline
\verb'{op: "var", args: ["x"]}'     & \verb'\var{x}' \\ \hline
\end{tabular}

\vspace{20pt}

The AST library provides functions for translating between ASTs and LaTeX formatted strings.
These formatting and parsing algorithms conform to common practice including:

\begin{itemize}
\item Insert \verb'\left' and \verb'\right' to auto size brackets.
\item Rewrite addition of a negative number as subtraction.
\item Elide \verb'\times' when it occurs before a non-numeric symbol during formatting, and construct
      a \verb'\times' node when a non-numeric symbol occurs after a non-operator symbol.
\item Insert parenthesis when formatting a binary expression of lower precedence that occurs in an
      expression of higher precedence.
\item (more here)
\end{itemize}

\pagebreak
\section{AST API}

%We provide an API for constructing, formatting, evaluating, and interning ASTs. Some of these
%functions take a optional \emph{model} object which interprets the meaning of operators that are not
%defined by the AST library, or which override those that are. When the \emph{model} parameter is
%given, the AST function delegates the call to that object, which may call back to the AST function
%to handle operators it does not distinguish.

The AST API is implemented as a module that can be loaded into an exercise. All functions are
instance methods of the {\bf ast} property added to {\bf KhanUtil}, and therefore are called through
that property (e.g. {\bf KhanUtil.ast.fromLaTeX}(``1+2''), or {\bf ast.fromLaTeX}(``1+2'') when
called from exercise HTML that loads the AST module).

Below is the list of API methods implemented by {\tt ast.js}.

\begin{tabular}{ | l | p{10cm} |}
\hline
{\bf Name} & {\bf Description} \\ \hline
{\bf fromLaTeX}(\emph{str}) & Return an AST node constructed from a LaTeX formatted string \\ \hline
{\bf toLaTeX}(\emph{node})  & Return the LaTeX formatted string correponding to an AST node \\ \hline
{\bf eval}(\emph{node})     & Return the result of evaluating an AST \\ \hline
{\bf intern}(\emph{node})   & Add an AST node and its child nodes to the node pool and return the root node id (nid) \\ \hline
{\bf node}(\emph{nid})      & Reconstitute an AST from its node id \\ \hline
\end{tabular}

\end{document}


\documentclass[preprint]{sigplanconf}
\usepackage{url}
\usepackage{proof}
\usepackage{amssymb}
\usepackage{stmaryrd}
\usepackage{listings}
\usepackage{graphicx}
\usepackage{comment}
\usepackage{multicol}

\newcommand{\dgm}[2][1.5]{
\begin{center}
\scalebox{#1}{
\includegraphics{diagrams/#2.pdf}
}
\end{center}
}

\newtheorem{definition}{Definition}
\newcommand{\bra}[1]{\langle #1 |}
\newcommand{\singleton}[1]{\textsc{Sing}~#1}

%subcode-inline{bnf-inline} name Pi
%! swap+ = \mathit{swap}^+
%! swap* = \mathit{swap}^*
%! dagger =  ^{\dagger}
%! assocl+ = \mathit{assocl}^+
%! assocr+ = \mathit{assocr}^+
%! assocl* = \mathit{assocl}^*
%! assocr* = \mathit{assocr}^*
%! identr* = \mathit{uniti}
%! identl* = \mathit{unite}
%! dist = \mathit{distrib}
%! factor = \mathit{factor}
%! (o) = \fatsemi
%! (;) = \fatsemi
%! (*) = \times
%! (+) = +
%! foldB = fold_B
%! unfoldB = unfold_B
%! foldN = fold_N
%! unfoldN = unfold_N
%! trace+ = \mathit{trace}^{+}
%! trace* = \mathit{trace}^{\times}
%! :-* = \multimap
%! :-+ = \multimap^{+}
%! emptyset = \emptyset

%subcode-inline{bnf-inline} regex \{\{(((\}[^\}])|[^\}])*)\}\} name main include Pi
%! [^ = \ulcorner
%! ^] = \urcorner
%! [v = \llcorner
%! v] = \lrcorner
%! [[ = \llbracket
%! ]] = \rrbracket
%! ^^^ = ^{\dagger}
%! eta* = \eta
%! eps* = \epsilon
%! Union = \bigcup
%! in = \in
%! |-->* = \mapsto^{*}
%! |-->> = \mapsto_{\ggg}
%! |-->let = \mapsto_{let}
%! |--> = \mapsto
%! <--| = \mapsfrom
%! |- = \vdash
%! bra = \bra
%! singleton = \singleton
%! <=> = \Longleftrightarrow
%! <-> = \leftrightarrow
%! -> = \rightarrow
%! ~> = \leadsto
%! ::= = ::=
%! /= = \neq
%! vi = v_i
%! di = d_i
%! si = s_i
%! sj = s_j
%! F = \texttt{F}
%! T = \texttt{T}
%! forall = \forall
%! exists = \exists
%! empty = \emptyset
%! Sigma = \Sigma
%! eta = \eta
%! where = \textbf{where}
%! epsilon = \varepsilon
%! least = \phi
%! loop+ = loop_{+}
%! loop* = loop_{\times}
%! CatC = {\mathcal C}
%! CatA = {\mathcal A}
%! gamma = \gamma
%! {[ = \{
%! ]} = \}
%! elem = \in
%! dagger = ^\dagger
%! alpha = \alpha
%! beta = \beta
%! rho = \rho
%! @@ = \mu
%! @ = \,@\,
%! Pow = \mathcal{P}
%! Pi = \Pi
%! PiT = \Pi^{o}
%! PiEE = \Pi^{\eta\epsilon}_{+}
%! PiT = \Pi^{o}
%! PiTF = \Pi^{/}
%! bullet = \bullet
%! * = \times

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{document}

\conferenceinfo{ICFP '13}{date, City.}
\copyrightyear{2013}
\copyrightdata{[to be supplied]}

\titlebanner{}
\preprintfooter{}

\title{Reversible Higher-Order Programs \\ (Functional Pearl)}
\authorinfo{Jacques Carette \and Amr Sabry}
           {McMaster University \and Indiana University}
           {carette@mcmaster.ca \and sabry@indiana.edu}

\maketitle

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{abstract}
  In previous work, we developed a \emph{first-order},
  information-preserving, and reversible programming language {{Pi}} founded
  on type isomorphisms. In subsequent work, we extended this language with
  \emph{fractional types}, the dual of conventional product types which, from
  an information-theoretic point of view, corresponds to the type of values
  with \emph{negative information}. The extended language was given an
  operational interpretation that hinted at two facts: fractional types can
  be used to define first-class pattern-matching clauses as well as
  higher-order reversible functions. Neither of these observations was backed
  by an appropriate denotational model, however. In this paper, we develop
  such a model by interpreting fractional types using functions mapping
  singleton types to the unit type. This interpretation requires that nearly
  all of the type constructs in {{Pi}} be re-interpreted as \emph{dependent
    types}. The framework, formalized in Agda, takes steps towards developing
  a computational interpretation of homotopy type theory using higher-order
  reversible languages.
\end{abstract}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{X}

For every value {{v : b}}, we introduce the \emph{singleton} type
{{ singleton{v} }} whose only element is the value {{v}}.

For every value {{v : b}}, we define the \emph{pattern} {{ bra{v} :
    singleton{v} 1}} which matches exactly {{v}} and nothing else. The type
of this pattern is a function which

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\bibliographystyle{abbrvnat}
\bibliography{cites}

\end{document}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%