% !TEX root =main.tex

\section{Multi-instance  Time-lock Puzzle}

\vspace{-2mm}

\subsection{Strawman Solution}\label{C-TLP-overview}

In the following, we elaborate on the  problems that would arise if an existing time-lock puzzle is used directly to handle  multiple puzzles at once.  Without loss of generality, to illustrate the problems, we use the well-known TLP scheme presented in Section \ref{Time-lock-Encryption}.

Consider the case where a client wants a server to learn a vector of messages: $\vv{\bm{m}}=[m_{\scriptscriptstyle 1},...,m_{\scriptscriptstyle z}]$ at times  $[f_{\scriptscriptstyle 1},...,f_{\scriptscriptstyle z}]$ respectively, where the client is available and online only at an earlier time $f_{\scriptscriptstyle 0}< f_{\scriptscriptstyle 1}$.  For the sake of simplicity, let $\Delta=f_{\scriptscriptstyle 1}-f_{\scriptscriptstyle 0}$ and $\Delta=f_{\scriptscriptstyle j+1}-f_{\scriptscriptstyle j}$, where $1\leq j \leq z$. A naive way to address the problem is that the client uses the TLP  to encrypt each message $m_{\scriptscriptstyle j}$ separately, such that it can be decrypted at time $f_{\scriptscriptstyle j}$ if  all ciphertexts and public keys are passed on to the server at time $t_{\scriptscriptstyle 0}$.  For the server to decrypt the messages  on time, it needs to start decrypting \emph{all of them} as soon as the ciphertexts and public keys are given to it.

\noindent\textit{\textbf{Parallel Composition Problem}}. The above naive approach yields two serious issues: (a) imposing a high computation cost, as  the server has to perform $S\Delta \sum\limits_{\scriptscriptstyle j=1}^{\scriptscriptstyle z}j$ squaring to decrypt all   messages, and (b) demanding a high level of parallelisation, as each puzzle has to be dealt with separately in parallel to the rest.  The  issues can be cast  as  ``\emph{parallel composition problem}'', where $z$ instances of a puzzle scheme are given at once to a server whose only option, to find solutions on time, is to solve them in parallel\footnote{It should not be confused with the ``universally composable'' notion put forth in \cite{Canetti01}.}. Also, for the client  to efficiently compute $a_{\scriptscriptstyle j}$  for each  message $m_{\scriptscriptstyle j}$,  where $j>1$, it has to perform at least one modular multiplication, i.e. $a_{\scriptscriptstyle j}=a_{\scriptscriptstyle 1} a_{\scriptscriptstyle j-1}=2^{\scriptscriptstyle j  T}$, where $a_{\scriptscriptstyle 1}=2^{\scriptscriptstyle T}$. In this step, in total $z-1$ modular multiplications are required  to compute all $a_{\scriptscriptstyle j}$ values, for $z$ messages (which is not optimal). Note, we do not see the above issues as  previous schemes' flaws, because they were not initially designed for the multi-puzzle setting.

\vspace{-3mm}

%

 \subsection{An Overview of our Solutions}\label{Overview-of-our-Solutions}
 Our key observation is, in the naive approach, the process of decrypting  messages has many overlaps  leading to a high  computation cost. So,  by removing the overlaps, we can considerably lower the overall cost both in \emph{puzzle solving} and \emph{puzzle creating} phases.  One of our core ideas  is to chain the puzzles. While chaining different puzzles may seem a relatively obvious approach to tackle the  issues, designing a secure protocol that also can make black-box use of a standard time-lock puzzle scheme, supports public verifiability, and has low costs is challenging. In our solution, a client  first encrypts the message  that is supposed to be decrypted after the rest and embeds the information needed for decrypting it into the ciphertext of the message that will be decrypted before that message. In other words, the client integrates the information (i.e. a part of public keys) needed to decrypt message $m_{\scriptscriptstyle j}$ into the ciphertext related to message $m_{\scriptscriptstyle j-1}$. In this case, the server after learning message $m_{\scriptscriptstyle j-1}$ at time $f_{\scriptscriptstyle j-1}$ learns the public key needed to perform the sequential squaring to decrypt the next message: $m_{\scriptscriptstyle j}$. This means after fully decrypting $m_{\scriptscriptstyle j-1}$, the server starts  squaring sequentially to decrypt $m_{\scriptscriptstyle j}$

  \noindent\textit{\textbf{Addressing Parallel Composition Problem}}.  The above approach solves the parallel composition problem for two main reasons.  First, the total  number of squaring required to decrypt all $z$ messages is now much lower, i.e. $S \Delta z$, and is equivalent to the number of squaring needed to solve only the last puzzle,  i.e. $z\text{-th}$ one. Second, it does not call for  high parallelisation. Because now the server does not need to deal with all of the puzzles in parallel; instead, it solves them sequentially one after another.

  \noindent\textit{\textbf{Adding Efficient Publicly Verifiable Algorithm}}. To let the  scheme  support  efficient public verifiability, we use the following novel trick. The client uses a commitment scheme to commit to every message: $m_{\scriptscriptstyle i}$ and publishes the  commitment. Then, it uses the time-lock encryption to encrypt the commitment's opening, i.e. a combination of $m_{\scriptscriptstyle i}$ and a random value. But, unlike the traditional commitment, the client does not open the commitment itself. Instead, the server does that, after it discovers the puzzle's solution.  When it finds a solution, it decodes the solution to find the opening and sends it to the public who can check the solution correctness. So, to verify  a solution's correctness,   a verifier  only needs to run the commitment's verification algorithm that is: (a)  publicly verifiable, and (b)   efficient. It can be built in the random oracle  or  the standard model.

    The approach also allows  the client at the setup to compute only a single $a=2^{\scriptscriptstyle T}$  reusable for all $z$ puzzles, imposing only $O(1)$  cost.

\vspace{-3mm}

\subsection{Multi-instance   Time-lock Puzzle Definition}\label{Section::Multi-instance-Time-lock Puzzle-Definition}
In this section, we provide a formal definition of a multi-instance time-lock puzzle. Our starting point is the  time-lock puzzle definition, i.e. Definition \ref{Def::Time-lock-Puzzle}, but we extend it from several  perspectives, so it can: (a) handle multiple  solutions/messages in setup, (b)  produce multiple puzzles for the messages,   (c) solve the puzzles given the puzzles and public parameters, and (d) support public verifiability. In the following, we provide the formal definition of a multi-instance  time-lock puzzle.
\begin{definition}[Multi-instance Time-lock Puzzle] A multi-instance time-lock puzzle has the following  five algorithms and satisfies completeness and efficiency properties.
\begin{itemize}[leftmargin=.43cm]
\item \textbf{Algorithms}:
\begin{itemize}
\item[$\bullet$]$\mathtt{Setup}(1^{\scriptscriptstyle\lambda},\Delta,z)\rightarrow (pk,sk,\vv{\bm{d}})$:  a probabilistic algorithm that takes as input  security: $1^{\scriptscriptstyle\lambda}$ and time:  $\Delta$ parameters and the total number of solutions/puzzles: $z$. Let     $j \Delta$ be a time period after which $j\text{\small{-th}}$ solution is found.   It outputs public-private key pair: $(pk,sk)$ and a vector of fixed size  secret witnesses: $\vv{\bm{d}}$

%\vv{\bm{s}}
\item[$\bullet$]$\mathtt {GenPuz}(\vv{\bm{m}}, pk, sk,\vv{\bm{d}})\rightarrow \ddot{o}$:  a probabilistic algorithm that takes as an input  a  message vector: $\vv{\bm{m}}=[m_{\scriptscriptstyle 1},...,m_{\scriptscriptstyle z}]$,  the public-private key pair: $(pk,sk)$, and the witness vector: $\vv{\bm{d}}$. It  outputs $\ddot{o}:(\vv{\bm{o}},\vv{\bm{h}})$, where $\vv{\bm{o}}$ is a puzzle vector, and $\vv{\bm{h}}$ is a commitment vector. Each $j\text{\small{-th}}$ element in  vectors $\vv{\bm{o}}$ and $\vv{\bm{h}}$ corresponds to a solution $s_{\scriptscriptstyle j}$ of the form: $s_{\scriptscriptstyle j}=m_{\scriptscriptstyle j}||d_{\scriptscriptstyle j}$ %Given $s_{\scriptscriptstyle j}$ and $b$, a public decoding function, $\mathtt{Decode}()$ returns $m_{\scriptscriptstyle j}$, i.e. $\mathtt{Decode}(s_{\scriptscriptstyle j},b)\rightarrow m_{\scriptscriptstyle j}$.

\item[$\bullet$]$\mathtt {SolvPuz}(pk,\vv{\bm{o}})\rightarrow \vv{\bm{s}}$:   a deterministic algorithm that takes as input  the public key: $pk$ and  puzzle vector: $\vv{\bm{o}}$. It outputs a solution vector: $\vv{\bm{s}}$

\item[$\bullet$]$\mathtt {Prove}(pk,s_{\scriptscriptstyle j})\rightarrow \ddot{p}_{\scriptscriptstyle j}$:  a deterministic algorithm that takes the public key: $pk$ and a solution: $s_{\scriptscriptstyle j}\in\vv{\bm{s}}$. It outputs a proof, $\ddot{p}_{\scriptscriptstyle j}:(m_{\scriptscriptstyle j},d_{\scriptscriptstyle j})$

\item[$\bullet$]$\mathtt {Verify}(pk,\ddot{p}_{\scriptscriptstyle j},h_{\scriptscriptstyle j})\rightarrow \{0,1\}$:  a deterministic algorithm that takes  public key: $pk$,  proof: $\ddot{p}_{\scriptscriptstyle j}$ and commitment: $h_{\scriptscriptstyle j}\in \vv{\bm{h}}$. It outputs  $0$ if it rejects, or $1$ if it accepts.
\end{itemize}
\item \textbf{Completeness}: for any honest prover and verifier, it always holds that:
\begin{itemize}
\item$\mathtt{SolvPuz}(pk,[o_{\scriptscriptstyle 1},...,o_{\scriptscriptstyle j}])=[s_{\scriptscriptstyle1},...,s_{\scriptscriptstyle j}]$, for every $j$, $1\leq j\leq z$

\item $\mathtt {Verify}(pk,\mathtt {Prove}(pk,s_{\scriptscriptstyle j}),h_{\scriptscriptstyle j})\rightarrow 1$
\end{itemize}
\item \textbf{Efficiency}: the run-time of algorithm $\mathtt {SolvPuz}(pk,[o_{\scriptscriptstyle 1},...,o_{\scriptscriptstyle j}])=[s_{\scriptscriptstyle1},...s_{\scriptscriptstyle j}]$ is bounded by:  $ poly(j\Delta,\lambda)$, where $poly(.)$ is a fixed polynomial and  $1\leq j\leq z$
\end{itemize}
\end{definition}

Informally, a multi-instance time-lock puzzle is secure if it satisfies two properties:  a solution's \emph{privacy} and  \emph{validity}. The former  requires  its $j\text{\small{-th}}$ solution   to remain hidden from all adversaries running in parallel within  time period: $j \Delta$, while the latter one requires that it is  infeasible for  a PPT adversary to come up with an invalid solution  and passes the verification. The two properties are formally defined in Definitions \ref{Def::Solution-Privacy} and \ref{Def::Solution-Validity}.

% \begin{definition}[Chained Time-lock Puzzle's Sequentiality] For functions  $\pi(t)$ and $\delta(t)$, a  chained time-lock puzzle is $(\pi,\delta)$-sequential if for any pair of randomised algorithm $\mathcal{A} : (\mathcal{A}_{\scriptscriptstyle 1},\mathcal{A}_{\scriptscriptstyle 2})$, where $\mathcal{A}_{\scriptscriptstyle 1}$ runs in total time $O(poly(t,\lambda))$ and $\mathcal{A}_{\scriptscriptstyle 2}$ runs in  time $\delta(t)$ using at most $\pi(t)$ parallel processors, there exists a negligible function $\mu(.)$ such that:
%

% $$ Pr\left[    \begin{array}{l}  \mathcal{A}_{\scriptscriptstyle 2}(pk, \ddot{o},state)\rightarrow s \\
% s.t.\\
% s=\mathtt {SolvPuz}(pk,\theta)
%
%   \end{array}
%   \middle |
%    \begin{array}{l}
%\mathtt{Setup}(1^{\scriptscriptstyle\lambda},\Delta,1)\rightarrow (pk,sk,\vv{\bm{d}})\\
%\mathcal{A}_{\scriptscriptstyle 1}(1^{\scriptscriptstyle\lambda},pk, \Delta,1)\rightarrow state\\
%m\stackrel{\scriptscriptstyle\$}\leftarrow \mathcal{M}\\
%\mathtt {GenPuz}(m, pk, sk)\rightarrow \ddot{o}\\
%\end{array}    \right]\leq \mu(\lambda)$$
%  \end{definition}
%  where $\theta\in \ddot{o}$.

%   $$ Pr\left[    \begin{array}{l}  \mathcal{A}_{\scriptscriptstyle 2}(pk, \ddot{o},state)\rightarrow a \\
% s.t.\\
%m'=\mathtt{Decode}(\mathtt {SolvPuz}(pk,\theta),b)\\
% a=m'
%
%   \end{array}
%   \middle |
%    \begin{array}{l}
%\mathtt{Setup}(1^{\scriptscriptstyle\lambda},\Delta,1)\rightarrow (pk,sk,\vv{\bm{d}})\\
%\mathcal{A}_{\scriptscriptstyle 1}(1^{\scriptscriptstyle\lambda},pk, \Delta,1)\rightarrow state\\
%m\stackrel{\scriptscriptstyle\$}\leftarrow \mathcal{M}\\
%\mathtt {GenPuz}(m, pk, sk)\rightarrow \ddot{o}\\
%\end{array}    \right]\leq \mu(\lambda)$$
%  \end{definition}
%  where $\theta\in \ddot{o}$.
%
%
%
%  xxx The above definition also captures the sequentiality for a single solution as well that means the adversary cannot find a single solution significantly less than required steps.
%

\begin{definition}[Multi-instance Time-lock Puzzle's Solution-Privacy]\label{Def::Solution-Privacy} A multi-instance time-lock puzzle  is privacy-preserving  if for all $\lambda$ and  $\Delta$,  any number of puzzle: $z\geq1$, any pair of randomised algorithm $\mathcal{A} : (\mathcal{A}_{\scriptscriptstyle 1},\mathcal{A}_{\scriptscriptstyle 2})$, where $\mathcal{A}_{\scriptscriptstyle 1}$ runs in  time $O(poly(j\Delta,\lambda))$ and $\mathcal{A}_{\scriptscriptstyle 2}$ runs in  time $\delta(j\Delta)<j\Delta$ using at most $\pi(\Delta)$ parallel processors, there exists a negligible function $\mu(.)$, such that:
\small{
$$ Pr\left[  \begin{array}{l}
 \mathcal{A}_{\scriptscriptstyle 2}(pk,\ddot{o},\text{state})\rightarrow \ddot{a}\\
 \text{s.t.}\\
\ddot{a}:(b_{\scriptscriptstyle i},i)\\
  m_{\scriptscriptstyle b_{\scriptscriptstyle i},i}=m_{\scriptscriptstyle b_{\scriptscriptstyle j},j}
  \end{array}
 \middle |
    \begin{array}{l}
\mathtt{Setup}(1^{\scriptscriptstyle\lambda},\Delta,z)\rightarrow (pk,sk,\vv{\bm{d}})\\
%\mathcal{A}_{\scriptscriptstyle 1}(1^{\scriptscriptstyle\lambda},pk,z)\rightarrow ([(m_{\scriptscriptstyle 0,1},m_{\scriptscriptstyle 1,1}),...,(m_{\scriptscriptstyle 0,z},m_{\scriptscriptstyle 1,z})],state)\\
\mathcal{A}_{\scriptscriptstyle 1}(1^{\scriptscriptstyle\lambda},pk,z)\rightarrow (\vv{\bm{m}},\text{state})\\
\left[b_{\scriptscriptstyle 1},...,b_{\scriptscriptstyle z}\right], b_{\scriptscriptstyle j}\stackrel{\scriptscriptstyle\$}\leftarrow \{0,1\}\\
%\mathtt {GenPuz}((m_{\scriptscriptstyle b_{\scriptscriptstyle 1},\scriptscriptstyle 1},..., m_{\scriptscriptstyle b_{\scriptscriptstyle z},\scriptscriptstyle z}), pk, sk,\vv{\bm{d}})\rightarrow \ddot{o}\\
\mathtt {GenPuz}(\vv{\bm{m}}', pk, sk,\vv{\bm{d}})\rightarrow \ddot{o}\\
\end{array}    \right]\leq \frac{1}{2}+\mu(\lambda)$$
}
where  $\vv{\bm{m}}: [(m_{\scriptscriptstyle 0,1},m_{\scriptscriptstyle 1,1}),...,(m_{\scriptscriptstyle 0,z},m_{\scriptscriptstyle 1,z})]$, $\vv{\bm{m}}':(m_{\scriptscriptstyle b_{\scriptscriptstyle 1},\scriptscriptstyle 1},..., m_{\scriptscriptstyle b_{\scriptscriptstyle z},\scriptscriptstyle z})$, $1\leq j\leq z$ and $1\leq i\leq z$
%$b_{\scriptscriptstyle j'}\in \left[b_{\scriptscriptstyle 1},...,b_{\scriptscriptstyle z}\right]$.
\end{definition}

%all probabilistic polynomial time adversaries $\mathcal{A}=(\mathcal{A}_{\scriptscriptstyle 1},\mathcal{A}_{\scriptscriptstyle 2})$ whose run-time is  bounded by  $T_{\scriptscriptstyle j}=j\cdot poly(\lambda,\Delta)$, where  $j\in [ 1,z]$,

The  definition above also ensures  the  solutions to appear after $j\text{\small{-th}}$ one,  remain hidden from the adversary with a high probability, as well. Similar to \cite{BonehBBF18,MalavoltaT19,garay2019}, it captures that even if     $\mathcal{A}_{\scriptscriptstyle 1}$ computes on the public parameters for a polynomial time,  $\mathcal{A}_{\scriptscriptstyle 2}$  cannot find $j\text{\small{-th}}$  solution in time $\delta(j\Delta)<j\Delta$ utilising $\pi(\Delta)$ parallel processors, with a probability significantly greater than $\frac{1}{2}$. As highlighted in  \cite{BonehBBF18}, we can set $\delta(\Delta)=(1-\epsilon)\Delta$ for a small  $\epsilon$, where $0<\epsilon<1$
\begin{definition}[Multi-instance Time-lock Puzzle's Solution-Validity]\label{Def::Solution-Validity}
A multi-instance time-lock puzzle preserves a   solution validity,   if  for all $\lambda$ and  $\Delta$,  any number of puzzles: $z\geq1$, all probabilistic polynomial-time adversaries $\mathcal{A}=(\mathcal{A}_{\scriptscriptstyle 1},\mathcal{A}_{\scriptscriptstyle 2})$ that run in  time $O(poly(\Delta,\lambda))$ there is  negligible function $\mu(.)$, such that:
\small{
$$ Pr\left[
    \begin{array}{l}
 \mathcal{A}_{\scriptscriptstyle 2}(pk,\vv{\bm{s}}, \ddot{o},\text{state})\rightarrow a\\

 \text{s.t.}\\
a:(j,\ddot{p}_{\scriptscriptstyle j} ,\ddot{p}')\\
 \ddot{p}_{\scriptscriptstyle j}: (m_{\scriptscriptstyle j},d_{\scriptscriptstyle j}),
\ddot{p}':(m',d') \\
 m_{\scriptscriptstyle j}\in \vv{{\bm{m}}}, d_{\scriptscriptstyle j}\in\vv{{\bm{d}}},
m\neq m'\\
\mathtt {Verify}(pk,\ddot{p},h_{\scriptscriptstyle j})= 1\\
\mathtt {Verify}(pk,\ddot{p}',h_{\scriptscriptstyle j})= 1\\
\end{array}
\middle |
\begin{array}{l}

\mathtt{Setup}(1^{\scriptscriptstyle\lambda},\Delta,z)\rightarrow (pk,sk,\vv{\bm{d}})\\
\mathcal{A}_{\scriptscriptstyle 1}(1^{\scriptscriptstyle\lambda},pk, \Delta,z)\rightarrow (\vv{{\bm{m}}},\text{state})\\

\mathtt {GenPuz}(\vv{{\bm{m}}}, pk, sk,\vv{\bm{d}})\rightarrow \ddot{o} \\
\mathtt {SolvPuz}(pk,\vv{\bm{o}})\rightarrow \vv{\bm{s}}

\end{array}
   \right]\leq  \mu(\lambda)$$
   }
where $\vv{{\bm{m}}}=[m_{\scriptscriptstyle 1},...,m_{\scriptscriptstyle z}]$, and $h_{\scriptscriptstyle j}\in \vv{\bm{h}}\in \ddot{o}$
\end{definition}

%In Definition \ref{Def::Solution-Validity}, we do not need to bound the adversaries' parallel computation power, as it does not need to solve any puzzles, in fact  puzzles' solutions are provided to them. Therefore, they can run in polynomial time $O(poly(\Delta,\lambda))$.
%
\begin{definition}[Multi-instance Time-lock Puzzle Security]\label{def::C-TLP-security} A multi-instance time-lock puzzle scheme  is secure if it meets solution-privacy and solution-validity properties.
\end{definition}

\vspace{-4mm}

\subsection{Chained  Time-lock Puzzle (C-TLP) Protocol}\label{Section::C-TLP-protocol}

In this section, we present the chained  time-lock puzzle (C-TLP), an instantiation of the multi-instance time lock puzzle. Since we have already presented an outline of C-TLP (in Section \ref{Overview-of-our-Solutions}), in this section we present C-TLP protocol in detail.  Recall, a client wants a server to learn a vector of messages: $\vv{\bm{m}}=[m_{\scriptscriptstyle 1},...,m_{\scriptscriptstyle z}]$ at times  $[f_{\scriptscriptstyle 1},...,f_{\scriptscriptstyle z}]$ respectively, where the client is available and online only at an earlier time $f_{\scriptscriptstyle 0}< f_{\scriptscriptstyle 1}$.  Also, the client wants to ensure that anyone can validate a solution found by the  server, i.e. supports public verifiability. For the sake of simplicity, let $\Delta=f_{\scriptscriptstyle 1}-f_{\scriptscriptstyle 0}$ and $\Delta=f_{\scriptscriptstyle j+1}-f_{\scriptscriptstyle j}$, where $1\leq j \leq z$ and $T=S \Delta$. Below, we provide C-TLP protocol. We refer readers to Appendix \ref{discussion-C-TLP} for further remarks on the protocol.

\input{CR-TLP-protocol-v2.tex}

   \begin{theorem}[C-TLP Security]\label{C-TLP-Sec}  C-TLP  is a secure multi-instance time-lock puzzle.
   \end{theorem}

\begin{proof}[Outline]
The proof of Theorem \ref{C-TLP-Sec} relies on the security of the TLP, symmetric key encryption, and commitment schemes. It is also based on the fact that the probability to  find a certain random generator is negligible. It shows both C-TLP's solution privacy (due to security of the above three schemes) and validity (due to the security of the commitment) are satisfied.  We refer readers to Appendix \ref{CR-TLP-Proof} for  detailed proof.
\end{proof}

\vspace{-2.5mm}
%
%\begin{remark} Recall, to make each  puzzle instance, a distinct random generator: $r_{\scriptscriptstyle j}$, is used. This is the reason, in  Fig. \ref{fig:CTE}, before a puzzle  is  generated   in step \ref{call-RTLP-GenPuz},  a new public key is set in step \ref{set-pk-in-loop}.  Also, at the beginning of the protocol only $r_{\scriptscriptstyle 1}$ is public and the rest of the generators are kept secret. They are found and used sequentially after their related puzzle is solved.
%\end{remark}
%
%
%\begin{remark} The commitments opening, including the commitment random values, are not known to other verifiers (than the puzzle generator) at the beginning of the protocol. At this point,  only the committed values are public. Once a solver solves each puzzle,  it  extracts one of the commitments' opening, and sends it to a public verifier who can check if the opening matches the commitment.
%\end{remark}
%
%\begin{remark}
% In Fig. \ref{fig:CTE}, we use the folklore hash-based commitment scheme, in the random oracle model, only to achieve more computation improvement than that can be achieved in the standard model. But C-TLP can utilise any efficient non-interactive commitment scheme in the \emph{standard model} as well, e.g. Pedersen Commitment.
%\end{remark}
%
%
%\begin{remark}
%The efficiency of  C-TLP scheme stems from three crucial factors: (a) removing computation overlaps when solving different puzzles: even though solving $j\text{\small{-th}}$ puzzle, where $j>1$, requires $jT$ squaring, $(j-1) T$ of the squaring is used to solve previous puzzles that leads to $\frac{z+1}{2}$ times computation cost reduction at the server-side,  (b)  supporting reusable single  public parameter: $a=2^{\scriptscriptstyle T}$, generated only once that costs $O(1)$, as opposed to the RSA TLP whose cost is linear: $O(z)$, and (c) supporting efficient verification: due to the way each message is encoded (i.e. embedding the opening in a solution).
%\end{remark}
%
%\begin{remark}
%C-TLP also can efficiently  be used in a multi-server setting,  where there are $z$ servers: $\{S_{\scriptscriptstyle 1},...,S_{\scriptscriptstyle z}\}$,  each $S_{\scriptscriptstyle j}$ needs to solve puzzle $\ddot{o}_{\scriptscriptstyle j}$ at time $f_{\scriptscriptstyle j}$ and passes on the solution to the next server $S_{\scriptscriptstyle j+1}$ to solve the next puzzle by time $f_{\scriptscriptstyle j+1}>f_{\scriptscriptstyle j}$. In this setting,  due to the scalability property of C-TLP (and unlike using the existing time-lock puzzles naively), other servers do not need to start solving the puzzle   as soon as the client releases puzzles public parameters. Instead, they can wait until the previous solution is issued that saves them significant cost. Furthermore, a  server can first  verify the correctness  of the solution found by the previous server (due to the public verifiability of C-TLP),  if accepted then   it starts finding the next solution.
%\end{remark}
%
%
%\begin{remark}
%In the following, we outline an  approach that looks an option to construct an efficient C-TLP; however, as we will show it would not be secure. In particular,  one uses the TLP to generate $z$ public and secret key pairs. Then, it uses the TLP to compute $z\text{\small{-th}}$ puzzle as $\mathtt{TLP.GenPuZ}(m_{\scriptscriptstyle z},pk_{\scriptscriptstyle z},sk_{\scriptscriptstyle z})\rightarrow \ddot{o}_{\scriptscriptstyle z}$.  Then, it embeds $\ddot{o}_{\scriptscriptstyle z}$ into $(z-1)\text{\small{-th}}$ one, i.e. $\mathtt{TLP.GenPuZ}(m_{\scriptscriptstyle z-1}||\ddot{o}_{\scriptscriptstyle z},pk_{\scriptscriptstyle z-1},sk_{\scriptscriptstyle z-1})\rightarrow \ddot{o}_{\scriptscriptstyle z-1}$. This process goes on until $\ddot{o}_{\scriptscriptstyle 1}$  is created. It sends the combined puzzles and public key (including all random generators) to the server; with the hope that puzzles can be solved sequentially and the time gap between finding two solutions will be $\Delta$.  This approach is not secure, because as soon as the server accesses $\ddot{o}_{\scriptscriptstyle 1}$ and public parameters, it can in parallel perform $T$ squaring on every generator, i.e. $r^{\scriptscriptstyle 2^{\scriptscriptstyle T}}_{\scriptscriptstyle i}$, for all $i, 1\leq i\leq z$. In this case, as soon as $\ddot{o}_{\scriptscriptstyle 1}$ is solved and  $\ddot{o}_{\scriptscriptstyle 2}$  is extracted, it has enough information to  immediately solve $\ddot{o}_{\scriptscriptstyle 2}$ and accordingly the rest of the puzzles without doing any further exponentiation.
 %\end{remark}

 \input{CR-TLP-cost-summary}
 % \input{CR-TLP-proof}


\chapter{Related work}
\label{sec:related-work}
In the Related work section, we introduce selected literature relevant in the scope of our thesis. In particular, we present two general-purpose survival methods, the Cox Proportional Hazards (\emph{Cox PH}) model and the \glsxtrfull{rsf}. Beyond general-purpose survival models, we also establish literature methods used for multi-omics integration in survival models. Lastly, we survey relevant miscellaneous literature related to our study, chiefly on \glsxtrfull{sgl} regularization and supervised autoencoders.

Please note that some of this section might seem like background but was deliberately kept in the Related work section. We had to define specific algorithms (for example, survival trees) to do justice to specific related work. Since these topics are not background needed for \emph{our} work, we chose to keep them in the Related work section, allowing users only interested in our work to skip over them.

\section{General purpose survival models}
\label{sec:gp-surv-models}
\subsection{Regression-based methods}
Some of the earliest work in survival analysis goes back to the Cox Proportional Hazards (Cox PH) model \citep{cox1972regression, breslow1975analysis}, which was derived by assuming that the \glsxtrlong{hf} $\lambda(t)$ can be decomposed multiplicatively into a time-dependent baseline hazard $\lambda_0(t)$ and a patient (and thus covariate) specific partial hazard (Equation \ref{eq:cox}).

\begin{equation}\label{eq:cox}
    \lambda(t\mid X_i) = \underbrace{\lambda_0(t)}_{\text{baseline hazard}} \ \underbrace{\text{exp}(X_i \beta)}_{\text{partial hazard}}
\end{equation}

Where $X_i$ is the covariate vector for patient $i$ and $\beta$ are the parameters to be estimated. We also call the $X_i \beta$ term the log-partial hazard.

Assume we have our data given as a triplet, $(U_j, \delta_j, X_j), \ \ j = 1, ..., n$. $U_j = \text{min}(T_j, C_j)$ where $T_j$ is the time to event for patient $j$ and $C_j$ is the time to right-censoring for patient $j$. Further, let us assume noninformative censoring as defined in Chapter \ref{sec:surv-intro}, that is the event is independent of the censoring time conditional on the covariates. Denote the ordered event times by $t_1 < ... < t_D$ and let $X_{(i)k}$ be the k-th feature for the patient who experienced the event at $t_i$. Lastly, define the risk set $R(t_i)$, as all patients who had not been censored and for whom the event had not occured immediately prior to $t_i$ (\emph{i.e.,} who were "at risk" of experiencing the event at $t_i$) \citep[p. 253]{klein2003survival}. Assuming for simplicity that there are no tied event times, we can then write the partial likelihood (Equation \ref{eq:orig-pl}) based on the \gls{hf}:

\begin{equation}
\label{eq:orig-pl}
    L(\beta) = \prod_{i=1}^D \frac{\text{exp}(\sum_{k=1}^p\beta_k X_{(i)k})}{\sum_{j \in R(t_i)} \text{exp}(\sum_{k=1}^p\beta_k X_{jk})}
\end{equation}

We can treat this partial-likelihood as a "usual likelihood" \citep[p. 253]{klein2003survival} resorting to the standard technique of maximum likelihood to find the parameters which maximize it. In particular, most often the partial \emph{log}-likelihood is maximized for numerical reasons (Equation \ref{eq:pl-efron}):

\begin{equation}\label{eq:pl-efron}
    \ell(\beta) = \text{log}(L(\beta)) = \sum_{i=1}^D \sum_{k=1}^p \beta_k X_{(i)k} - \sum_{i=1}^D\text{log}\left(\sum_{j \in R(t_i)} \text{exp}\left(\sum_{k=1}^p \beta_k X_{jk}\right)\right)
\end{equation}

Which follows by basic algebra \citep[p. 253]{klein2003survival}.

Up to this point, we know only the partial hazard $\text{exp}(X_i \beta)$, but have not specified the baseline hazard $\lambda_0$. While the baseline hazard is generally treated non-parametrically \citep[p. 244]{klein2003survival}, at times it is not even specified since clinicians or researchers might be interested "only" in a direct interpretation of the regression coefficients found by maximizing the partial (log-)likelihood. In particular, we can investigate two patients with differing feature vectors $X$ and $X^*$ and compare their hazard functions according to the Cox PH model \citep[p. 245]{klein2003survival}:

\begin{equation}
    \frac{\lambda(t \mid X)}{\lambda(t \mid X^*)} = \frac{\text{exp}(\sum_{k=1}^p \beta_k X_k)}{\text{exp}(\sum_{k=1}^p \beta_k X^*_k)} = \text{exp}\left(\sum_{k=1}^p \beta_k (X_k - X_k^*)\right)
\end{equation}

Since this is a constant, this also directly gives the reason why the Cox PH model is called "proportional hazards" - because the hazard functions for any two patients are proportional to each other \citep[p. 245]{klein2003survival}. A further interpretation of each coefficient $\beta_i \in \beta$ is easiest to expose for binary covariates: Suppose we have two patients with covariate vectors $X = [1, x_2, ..., x_p]$ and $X^* = [0, x_2, ..., x_p]$. Then we have:

\begin{equation}
    \frac{\lambda(t \mid X)}{\lambda(t \mid X^*)} = \frac{\text{exp}(\sum_{k=2}^p \beta_k X_k) \text{exp}(\beta_1)}{\text{exp}(\sum_{k=2}^p \beta_k X^*_k)} = \text{exp}(\beta_1)
\end{equation}

Thus, the natural interpretation of $\beta_1$ is the risk of experiencing the event of a patient who had $x_1=1$ (whatever this may be) relative to another patient who had identical covariates except for $x_0 = 0$. While the exact interpretation of risk is not straightforward in this context, this interpretation of coefficients is nevertheless popular among clinicians and researchers. Reporting the effect of covariates on the hazard function like this is often deemed a \emph{hazard ratio}. Equivalently, "The hazard ratio is an estimate of the ratio of the hazard rate in the treated versus the control group" \citep{spruance2004hazard} (in the case when $X_0$ is a binary treatment).

Above, we assumed no ties were present (that is, no two patients experienced the event simultaneously). In real data, there are, however, usually ties present. The two most common strategies to handle these are the Breslow (Equation \ref{eq:breslow-approx}) and Efron approximations (Equation \ref{eq:efron-approx}).\footnote{Note that all methods for tie handling collapse to the partial likelihood previously introduced in the presence of no ties \citep[p. 260]{klein2003survival}.}

Let $t_1 < ... < t_D$ denote $D$ \emph{distinct} and ordered event times. Further, let $D_i$ be set number of patients who experienced the event at $t_i$, and $R_j$ be the set of patients who were at risk immediately prior to $t_j$ \citep[p. 259]{klein2003survival}. Then, Breslow's approximation is \citep[p. 259]{klein2003survival}:

\begin{equation}\label{eq:breslow-approx}
\begin{split}
        L_\text{Breslow}(\beta) = \prod_{i=1}^D \frac{\text{exp}(\beta \sum_{j \in D_i} X_j)}{\left(\sum_{j_ \in R_i} \text{exp}(\beta X_j)\right)}
\end{split}
\end{equation}

And Efron's approximation is \citep[p. 259]{klein2003survival}:

\begin{equation}\label{eq:efron-approx}
\begin{split}
        L_\text{Efron}(\beta) = \prod_{i=1}^D \frac{\text{exp}(\beta \sum_{j \in D_i} X_j)}{\prod_{j=1}^{|D_i|}\left(\sum_{k \in R_i} \text{exp}(\beta X_k) - \frac{j-1}{d_i}\sum_{k \in D_i} \text{exp}(\beta X_k)\right)}
\end{split}
\end{equation}

Where $\beta$ are the coefficients to be estimated and $X_j$ is the covariate vector corresponding to patient $j$. Recall that in this case, the $i$s we multiply over are distinct events. It can be seen quite easily (Equation \ref{eq:breslow-equivalence}) that the Breslow approximation is in effect the same thing as applying the original partial log-likelihood despite the presence of ties. Although the Efron approximation is slightly more accurate, it is also more challenging to program.

\begin{equation}\label{eq:breslow-equivalence}
    \begin{split}
            L_\text{Breslow}(\beta) = \prod_{i=1}^D \frac{\text{exp}(\beta \sum_{j \in D_i} X_j)}{\left(\sum_{j_ \in R_i} \text{exp}(\beta X_j)\right)} = \prod_{i=1}^D \prod_{j \in D_i}\frac{\text{exp}(\beta X_j)}{\sum_{k \in R_i} \text{exp}(\beta X_k)} = L(\beta)
    \end{split}
\end{equation}

Depending on the number of ties in a dataset, it thus makes sense to use the Efron approximation when available, especially since it is implemented in most common software packages for survival analysis today \citep{polsterl2020scikit, survival-package-2020}.

Due to colinearity leading to non-unique solutions, the unregularized Cox PH model can effectively only be used in $n \geq p$ settings, similar to linear and logistic regression. However, there have been proposals to regularize the Cox model to handle high-dimensional data. \citet{tibshirani1997lasso} proposed to use the Lasso \citep{tibshirani1996regression} for regularization of Cox PH models (Equation \ref{eq:cox-lasso}) and showed in simulations that the Lasso regularized Cox PH could outperform stepwise selection in estimating the true coefficients under the presence of noise variables while simultaneously providing a more stable model with respect to data changes. \citet{simon2011regularization} later made an \gls{en} regularized Cox PH model (Equation \ref{eq:cox-en}) available as part of their \emph{glmnet} \citep{friedman2010regularization} software implementation, which is still one (if not the) most popular implementations of regularized Cox PH models used by researchers today.

\begin{equation}\label{eq:cox-lasso}
    \beta \in \argmin_\beta - \ell(\beta) + \lambda ||\beta||_1
\end{equation}

\begin{equation}\label{eq:cox-en}
    \beta \in \argmin_\beta - \ell(\beta) + \alpha \lambda ||\beta||_1 + (1-\alpha) \lambda ||\beta||^2_2, \ \ \ \alpha \in [0, 1]
\end{equation}

Where $\beta$ are the parameters to be estimated, $\ell(\beta)$ is a partial log-likelihood (depending on which tie approximation is used), $\lambda$ is a regularization hyper-parameter and $\alpha$ is a tradeoff hyper-parameter between the Lasso (for $\alpha = 1$) and Ridge (for $\alpha = 0$).

Regularized Cox PH models are among the most used models for survival analysis today, both in studies predicting survival from a single input group (mostly gene expression) \citep{ching2018cox} as well as in multi-omics work \citep{herrmann2021large, hornung2019block, huang2019salmon}. Beyond preventing convergence issues seen in highly correlated or generally high-dimensional datasets, regularized Cox PH (especially the Lasso) is, of course, also popular for feature selection in survival analysis \citep{tibshirani1997lasso}. Furthermore, regularized Cox PH has been extended in various ways for multi-omics data. We will cover these methods in Section \ref{sec:multi-omics-stat}.

\subsection{Tree-based survival methods}
An alternative method for the prediction of survival times is the method of survival trees. Survival trees are a natural extension of regression and classification decision trees to survival data. Since the general way of fitting survival trees is identical to that of regression or classification trees, we first briefly introduce decision trees more broadly and then highlight what distinguishes survival trees.

\subsubsection{Decision trees}
A decision tree "aims to partition the covariate space recursively to form groups (nodes in the tree) of subjects which are similar according to the outcome of interest" \citep{bou2011review}. The partitioning is performed by optimizing a measure of node impurity specific to the task being performed. Depending on the particular algorithm used, trees are generally built starting with all observations. An exhaustive search is performed to find the best binary split (according to the split point criterion). The algorithm runs until some stopping criterion is met, \emph{e.g.,} the minimum size of a leaf node (Algorithm \ref{alg:decision-tree}) \citep{bou2011review, friedman2001elements}. All stopping criteria are designed to prevent trees from overfitting the data too much, as would be possible by assigning each sample its own leaf node.

\begin{algorithm}
\centering
    \caption{Decision tree training algorithm \citep{bou2011review, friedman2001elements}.}\label{alg:decision-tree}
        \begin{algorithmic}
        \Require $X, y, \text{split point criteria}, \text{stopping criteria}, m$
        \State Initialize tree
        \For{each terminal node}
            \While{none of the stopping criteria is met}
                \State Select $m$ variables at random from all $p$ available variables (a split is performed on only $1$ variable out of the $m$)
                \State Pick the binary split maximizing the split-point criterion
                \State Split the terminal node into two daughter nodes
            \EndWhile
        \EndFor
        \State \Return tree
    \end{algorithmic}
\end{algorithm}

Where $m$ is the number of variables to be considered at each split (where $m \leq p$). After training, a decision tree \emph{may} have overfitted the data, depending on the stopping criteria. The tree can then be pruned (\emph{i.e.,} have some of its splits removed) if desired to enhance generalizability \citep{bou2011review}. Alternatively, ensembles of overfit decision trees may be used to improve generalization capabilities. Since pruning is less used today and most implementations rely on tree ensembles, we neglect to develop pruning further.

Once the tree has been fitted, the prediction for a new sample that ends in leaf node \(i\) is made using a suitable summary statistic of the target of all training samples in the same leaf node, \emph{e.g.,} the majority class within the leaf node in classification (Figure \ref{fig:rpart-iris-class}).

\begin{figure}
    \centering
        \includegraphics[width=15.7463cm,height=7.87315cm]{./content/figures/fig_plot_iris_class_tree.png} \caption{Example classification tree grown on the \emph{iris} dataset \citep{fisher1936use, anderson1936species}.}\label{fig:rpart-iris-class}
\end{figure}

\subsubsection{Survival trees}
Survival trees differ from decision trees primarily in the split point criterion used, their stopping criteria, and how the final prediction is made, all of which \emph{may} be specific to survival data.

While there are various possibilities for split-point criteria in survival trees (we refer to \citet{bou2011review} for a review), we focus on the log-rank splitting rule, as this is both the most common and also the one employed by \emph{ranger} \citep{wright2015ranger} and \emph{BlockForest} \citep{hornung2019block}, two tree-based survival methods which we included in our study for purposes of benchmarking.

To define the log-rank statistic used for survival trees, we follow the exposition in \citet{segal1988regression} and draw the readers attention to Figure \ref{fig:surv-table}. Let $A_i$ be the random variable denoting the number of deaths in population $1$ for survival table $i$. Let $\{1, ..., k\}$ be all distinct, uncensored observations (for each of which there will be a table). Our null hypothesis is that "the death rates for the two populations are equal" \citep{segal1988regression}. Under $H_0$ and for all marginal totals fixed, $A_i$ can be shown to be hypergeometric, with the following expected value and variance \citep{segal1988regression, fisher1935logic}:

\begin{equation}
    \mathbbm{E}_0(A_i) = \frac{m_{i1} n_{i1}}{n_i}
\end{equation}

and

\begin{equation}
    \mathbbm{V}_0(A_i) = \frac{m_{i1} (n_i - m_{i1})}{n_i -1 } \cdot \left(\frac{n_{i1}}{n_i} \left(1- \frac{n_{i1}}{n_i} \right)\right)
\end{equation}

where $n_i = n_{i1} + n_{i2}$. The log-rank statistic $T_{\text{LR}}$ is then (Equation \ref{eq:log-rank}) \citep{segal1988regression}:

\begin{equation}
    T_{\text{LR}} = \frac{\sum_{i=1}^k (a_i - \mathbbm{E}_0(A_i))}{\sqrt{\sum_{i=1}^k \mathbbm{V}_0(A_i)}}
    \label{eq:log-rank}
\end{equation}

\begin{figure}
\centering
\begin{tikzpicture}[
box/.style={draw,rectangle,minimum size=2cm,text width=1.5cm,align=center}]
\matrix (conmat) [row sep=.1cm,column sep=.1cm] {
\node (tpos) [box,
    label=left:\( \textbf{Population 1} \),
    label=above:\( \textbf{Dead} \),
    ] {$a_i$};
&
\node (fneg) [box,
    label=above:\textbf{Alive},
    label=above right:,
    label=right:\( n_{i1} \)] {};
\\
\node (fpos) [box,
    label=left:\textbf{Population 2},
    label=below left:,
    label=below:\(m_{i1}\)] {};
&
\node (tneg) [box,
    label=right:\(\),
    label=below:] {};
\\
};
\end{tikzpicture}
\caption{Survival table to define the log-rank statistic. Reproduced from \citet{segal1988regression}.}
\label{fig:surv-table}
\end{figure}

On a more practical note, one may imagine our training samples being stratified by the variable under consideration (for example, if one was to split on gender, population one might be female and population two male). Further, recall that since we consider only \emph{distinct} censored observations, it is possible that multiple samples from each population died at a specific time $t^* \in \{1, ..., k\}$.

For stopping criteria, the standard \emph{Python} implementation of survival trees, \emph{sksurv} \citep{polsterl2020scikit}, uses a specific number of minimum samples to be contained in a leaf for a split to be attempted, combined with a minimum number of samples to be included in each leaf node after the split. Another option is setting a max depth (\emph{e.g.,} a max depth of $x$ means the tree can perform at most $x$ splits in total). In survival trees, another possible stopping criterion is requiring each leaf node after a split to contain at last $d_0 > 0$ unique event times \citep{ishwaran2008random}.

For the final prediction for a new individual $i$, survival trees most often return either an estimate of the \gls{chf} (Definition \ref{def:cumulative-hazard}) or an estimate of the \gls{sf} (Definition \ref{def:surv-funct}), both of which are estimated based on all training samples falling into the same leaf node as $i$. The \gls{chf} function is usually estimated using the \emph{Nelson-Aalon} estimator \citep[p. 93-95]{klein2003survival} (Equation \ref{eq:nelson-aalon}). Meanwhile, the survival function is usually estimated using the \emph{Kaplan-Meier} estimator \citep[p. 92]{klein2003survival} (Equation \ref{eq:km}).

\begin{equation}
    \hat \Lambda(t) = \left\{\begin{array}{lr}
        0, & \text{if }  t \leq t_1\\
        \sum_{t_i \leq t} \frac{|D_i|}{|R_i|}, & \text{if } t_1 \leq t
        \end{array}\right.
    \label{eq:nelson-aalon}
\end{equation}

\begin{equation}
    \hat S(t) = \left\{\begin{array}{lr}
        1, & \text{if }  t < t_1\\
        \prod_{t_i \leq t} \left(1 - \frac{|D_i|}{|R_i|}\right), & \text{if } t_1 \leq t
        \end{array}\right.
    \label{eq:km}
\end{equation}

Where $D_i$ is the set of patients who experienced an event at $t_i$ and $R_i$ is the set of patients which were at risk at time $t_i$ (\emph{i.e.,} patients who had not been censored or experienced the event immediately prior to $t_i$) \citep[p. 91-92]{klein2003survival}. Both the Kaplan-Meier and the Nelson-Aalen estimator are not defined for $t > t_\text{max}$, where $t_\text{max} = \text{max}\{t_1, ..., t_n\}$ \citep[p. 91-94]{klein2003survival}.

Figure \ref{fig:ctree-surv} shows an example survival tree built using \emph{partykit} \citep{hothorn2015partykit} on the veteran \citep{kalbfleisch2011statistical} dataset contained in the \emph{R} \emph{survival} package \citep{survival-package-2020}. In addition to the splits, Figure \ref{fig:ctree-surv} also shows the p-value of the log-rank test (middle) and the Kaplan-Meier estimations of the survival curves for each leaf node.

\begin{figure}
    \centering
        \includegraphics[width=15.7463cm,height=7.87315cm]{./content/figures/fig_plot_veteran_surv_tree.png} \caption{Example survival tree grown on the \emph{veteran} dataset \citep{kalbfleisch2011statistical}.}\label{fig:ctree-surv}
\end{figure}

We now introduce random (survival) forests, the most common ensemble of decision trees (hence the name forest), and, more broadly, the most common usage of decision trees today.

\subsubsection{Random (survival) forests}
In random forests \citep{breiman2001random}, a few adjustments are made to decision trees to de-correlate the trees from each other while keeping the variance of each tree as small as possible \citep{friedman2001elements}.\footnote{This analysis can be made much more rigorous, essentially showing that the bias of a random forest is equal to that of any individual bootstrapped tree within the forest. Furthermore, it can be shown that as $B \to \infty$, the variance of the prediction (in the case of regression trees; since this is a mean) is equal to the pairwise correlation between trees times the variance of each tree. Thus, the goal is to reduce the correlation between trees as much as possible while keeping variance moderate.}

First, each split within each tree is made using only a subset of input features.\footnote{While this is also possible with singular decision trees, it is much more common in random forests.} Standard options include only considering the square root or logarithm of all input features (randomly sampled) at each split in each tree. Secondly, random forests randomly sample input samples (with replacement) until the number of samples equals a certain fraction of the original number of input samples. Both of these adjustments decrease the pairwise correlation among decision trees in the random forest, which can improve overall performance \citep{breiman2001random}. Algorithm \ref{alg:random-forest} gives a full overview \citep{friedman2001elements}.

\begin{algorithm}[H]
\centering
    \caption{Random forest training algorithm \citep{friedman2001elements}.}\label{alg:random-forest}
        \begin{algorithmic}
        \Require $X, y, \text{split point criterion}, \text{stopping criteria}, m, B, u$
        \State Initialize forest
        \For{$i = 1, ..., {B}$}
            \State Draw a bootstrap sample $X^*$ of size $u * n$ from $X$
            \State Initialize $T_i$
            \State $T_i \gets \text{train decision tree}(X^*, y, \text{split point criterion}, \text{stopping criteria}, m)$ (Algorithm \ref{alg:decision-tree})
            \State forest[i] $\gets T_i$
        \EndFor
        \State \Return forest
    \end{algorithmic}
\end{algorithm}

Where $B$ is the number of trees in the random forest, $u$ is the size of each bootstrapped dataset relative to the original dataset (which is of size $n$), and $m$ is the number of variables to consider at each split (where $m \leq p$).

To get a final prediction using a random forest, the predictions from all decision trees in the forest are ensembled using a summary statistic. This can be done using the majority class for classification problems, while the mean may be used for regression problems.

\citet{ishwaran2008random} introduced random survival forests, an extension of random forests to survival trees. The researchers grew a forest of survival trees, using a split point criterion and stopping criteria of choice. Afterward, the ensemble prediction (for the \gls{chf} or the \gls{sf}) is made by taking the mean of the respective predictions (for either the \gls{chf} or the \gls{sf}) of each tree in the forest \citep{ishwaran2008random}. \citet{ishwaran2008random} showed that their new methods of random survival forests could outperform the Cox PH model across various survival datasets in terms of Harrell's concordance index \citep{harrell1982evaluating}. Since survival random forests have shown to perform well without excessive tuning, they are often included when performing benchmarks on survival data \citep{herrmann2021large, hornung2019block, huang2019salmon}.

Having covered the general-purpose survival models needed in the scope of our thesis, we now move on to survival models specifically designed for multi-omics integration.

\section{Multi-omics survival models}
As explained in Chapter \ref{sec:multi-omics}, multi-omics data needs specific methods to properly integrate it due to factors such as strong correlations between and within groups of variables and vastly differing dimensions of input variable groups \citep{boulesteix2017ipf, herrmann2021large}. We start by surveying statistical methods for multi-omics integration in survival models before moving on to neural networks.

\subsection{Statistical methods}
\label{sec:multi-omics-stat}
\citet{hornung2019block} proposed five variations of the random survival forest algorithm tailored for multi-omics integration. All of their models change the way variables are sampled in the random forest for split point selection by considering that the input variables belong to different blocks (\emph{i.e.,} groups). The basic idea is thus to account for the fact that some variable groups (\emph{e.g.,} clinical data) may be of much smaller dimension but contain a lot of prognostic information. Two of their five proposed variants (in particular, \glsxtrfull{bf} and \emph{RandomBlock}) performed significantly better than \glsxtrfull{rsf} across 20 \gls{tcga} datasets as measured by Harrell's concordance \citep{harrell1982evaluating}. Furthermore, \gls{bf}, their best performing method, has been shown to perform state-of-the-art among statistical models for multi-omics survival analysis in both their work and a large-scale benchmarking study considering only non-neural methods \citep{hornung2019block, herrmann2021large}. \emph{BlockForest} and its variants are non-sparse. That is, they use all variables initially selected for prediction, which may make the transfer to clinical settings more difficult. Some variants, in particular \emph{RandomBlock}, expose block selection probabilities, however, which allow for an assessment of the importance of each modality \citep{hornung2019block}.

\citet{boulesteix2017ipf} proposed a Lasso model which scales the Lasso penalty \(\lambda\) with a group-specific penalty factor that can be chosen either through \emph{a priori} knowledge or determined using cross-validation. Their new model, termed \emph{ipflasso}, performed better than Lasso regularized Cox PH, separate Lasso fits to each modality combined using logistic regression and \glsxtrlong{sgl} regularized Cox PH on \gls{tcga} as measured by the integrated brier score \citep{brier1950verification} when integrating clinical data, gene expression and CNV for \gls{tcga}-\gls{laml}. Furthermore, \emph{ipflasso} yielded among the sparsest models of all methods considered in their benchmark, selecting only $7.3$ variables on average on the \gls{tcga}-\gls{laml} dataset.

\citet{klau2018priority} introduced a sequential regression approach based on so-called offsetting, \emph{prioritylasso}, which considers input modalities one at a time in a specific order and uses the previous model prediction as an offset. The authors showed that their model could perform as well as or better than Lasso regularized Cox PH on an acute myeloid leukemia dataset. Furthermore, \emph{prioritylasso} on average selected sparser models relative to Lasso regularized Cox PH and, in particular, also allows clinicians to give preference to certain blocks by assigning them a higher priority order (that is, considering them first, or at least earlier in the sequential modeling process). Both \emph{prioritylasso} and \emph{ipflasso} might thus be more suitable for clinics, as they emphasize their \emph{transportability}, that is the fact that they can be used by simply communicating the selected features, preprocessing strategy, and coefficients. This can be a big advantage relative to other models, which might require clinics to run their code, which might become deprecated over the years \citep{klau2018priority, boulesteix2017ipf}.

\subsection{Neural networks}
Especially starting with \citet{ching2018cox}, there has been much interest in applying neural networks to estimating the log-partial hazard term, $X_i\beta$, in the Cox PH model by minimizing the partial log-likelihood. \citet{ching2018cox} used a neural network architecture with one hidden layer, dropout, and L2 regularization to predict survival from only gene expression.\footnote{The work of \citet{ching2018cox} is thus not a multi-omics model, but we felt compelled to include it anyway since it was formative for a lot of other work in the literature.} Their architecture outperformed \gls{rsf}, Ridge regularized Cox PH (Equation \ref{eq:cox-en} with $\alpha=0$), and \emph{coxboost} \citep{binder2015package} on ten \gls{tcga} cancers in terms of Harrell's concordance and Inverse Probability of Censoring Weighting concordance \citep{uno2011c}.

\citet{xie2019group} proposed an architecture similar to \citet{ching2018cox} for multi-omics data including gene expression, CNV, RPPA, mutation and clinical data. Their model uses Group-Lasso regularization on the edges going from the input to the first hidden layer. The authors showed that Group-Lasso regularization prevented overfitting relative to Lasso and using no regularization and statistically significantly outperformed the same architecture with Lasso regularization as well as no regularization on $3$ out of $14$ considered \gls{tcga} cancers.

 \citet{cheerla2019deep} proposed a novel neural network architecture that integrates gene expression, miRNA, clinical data, and whole slide images to cancer survival on 20 \gls{tcga} cancer types by optimizing the partial log-likelihood. Their model benefitted (\emph{i.e.,} exhibited an increased Harrell's concordance index) from pan-cancer training relative to training on each cancer for most considered cancers. Although \citet{cheerla2019deep} did not consider direct benchmarks in their study since they were able to handle missing modalities (which none of the benchmarks would have been able to), they note that in direct comparisons, they generally performed as well as or better than comparable studies.

 \citet{kim2020improved} proposed an architecture based on a \gls{vae} \citep{kingma2013auto} on which they applied transfer learning. The authors trained their \gls{vae} on 20 \gls{tcga} cancers using gene expression only and transferred the weights of the first two layers to a neural survival model with an additional hidden layer, which was fine-tuned on each of the same ten \gls{tcga} cancers on which \emph{cox-nnet} \citep{ching2018cox} was benchmarked.\footnote{The work of \citet{kim2020improved} was similarly not a multi-omics method. Still, we once again felt compelled to include it as it was one of the few examples of transfer learning in neural models for survival analysis.} Their model outperformed both \emph{cox-nnet} and regularized Cox PH in terms of Harrell's concordance on seven out of the ten cancers considered. The models of \citet{tong2020deep} were trained sequentially, meaning that not everything was trained jointly as would be the case with supervised autoencoders.

\citet{tong2020deep} explored multi-modal autoencoders for the integration of multi-omics data in \gls{tcga}-\gls{brca} survival. The authors proposed two architectures, each of which uses a dedicated autoencoder per input modality. The first, termed \emph{CrossAE}, tries to reconstruct both its own input and all other input variable groups. Afterward, \emph{CrossAE} mean pools the latent representations and uses the result to predict the log-partial hazard. In their second architecture, \emph{ConcatAE}, each autoencoder only reconstructs its input. Afterward, the model concatenates all latent representations and uses them to predict the log-partial hazard. When comparing different combined modalities and their different architectures, \emph{ConcatAE} performed the best in terms of Uno's concordance index \citep{uno2011c} when integrating methylation and miRNA and using \gls{pca} for dimensionality reduction.

\citet{huang2019salmon} modelled \gls{tcga}-\gls{brca} survival by integrating \gls{mirna}, \gls{mrna}, \gls{cnv}, and mutation data. They proposed an architecture that takes the \gls{mrna}-seq eigengene and the \gls{mirna}-seq eigengene matrices, both passed through a hidden layer individually (\emph{i.e.,} \gls{mirna} does not interact with \gls{mrna}). Afterward, a final layer predicts the log-partial hazard from the output of the hidden layer of \gls{mirna}, the hidden layer of \gls{mrna}, \gls{cnv}, mutation, and selected clinical variables. Their new model outperformed \gls{rsf}, regularized Cox PH and \emph{deepsurv} \citep{katzman2018deepsurv} as measured by Harrell's concordance.

\citet{jiang2020tlsurv} performed survival analysis of TCGA-LUAD by integrating a combination of CNV, DNA methylation, miRNA, and mRNA.\footnote{Please note that the authors always combined a maximum of two input modalities at a time.} In particular, they proposed a "super-hybrid network" based on the idea of sequentially training each part of the network. They integrated each multi-omics modality by first fitting a one-hidden layer neural network (\emph{i.e.,} not an autoencoder) on each block to predict survival. Afterward, the hidden nodes from each modality's network were fused by a multi-modal autoencoder using mean-pooling or concatenation. The fused view of all modalities was then passed into a survival network which mirrors the architecture of \emph{coxnnet} \citep{ching2018cox} to predict the log-partial hazard for each patient. \citet{jiang2020tlsurv} showed that their mean-pooling approach, \emph{TLSurv (MAE)}, was able to outperform the best \emph{coxnnet} model in terms of time-dependent concordance \citep{antolini2005time} when integrating methylation and miRNA, methylation and mRNA as well as mRNA and miRNA.\footnote{The authors fit \emph{coxnnet} on each modality individually, the best performing of which was mRNA.} \emph{TLSurv (MAE)} was not able to outperform the best \emph{coxnnet} model when integrating any of the other two modalities combined. Similarly, \emph{TLSurv (VAE)}, their model leveraging concatenation based on a \gls{vae}, was not able to outperform the best performing \emph{coxnnet} model using any combination of modalities.

Having covered the work we deemed relevant in terms of general-purpose and multi-omics survival methods, we come to miscellaneous related work, chiefly on the topic of supervised autoencoders and \gls{sgl} regularization.

\section{Miscellaneous}
\subsection{Supervised autoencoders}
\citet{tan2020multi} presented an application of supervised autoencoders on \gls{tcga} data. The authors predicted binarized clinical endpoints (\emph{e.g.,} overall survival, disease-free survival) from \gls{tcga} data using methylation, \gls{mirna}, \gls{mrna}, and \gls{rppa}. They trained one supervised autoencoder for each omics modality and fused their latent spaces using mean pooling, from which they then predicted the endpoint. Their new architecture outperformed other machine learning models such as random forests and support vector machines on binary endpoint classification in terms of \emph{ROC AUC} \citep{bradley1997use}.

\subsection{Sparse-Group-Lasso (SGL) regularization}
\gls{sgl} \citep{friedman2010note, simon2013sparse} regularization is a convex combination of the Lasso \citep{tibshirani1996regression} and the Group-Lasso \citep{yuan2006model} (Equation \ref{eq:sgl}):

{
\begin{equation}
  \beta \in \argmin_{\beta} \sum_{i=1}^n L(y_i, \hat y_i) + (1-\alpha)\lambda \sum_{l=1}^m \sqrt{p_l}||\beta^{(l)}||_2 + \alpha \lambda ||\beta||_1, \ \ \ \alpha \in [0, 1]
  \label{eq:sgl}
\end{equation}
}

Where $L$ is a loss function appropriate for the task we are interested in,  $\beta$ are the parameters to be estimated, $\lambda$ is a regularization hyper-parameter, and $\alpha$ is a trade-off hyper-parameter between the Lasso (for $\alpha = 1$) and the Group-Lasso (for $\alpha = 0$). $\beta$ is the full parameter vector to be estimated, $\beta^{(l)}$ are the parameters corresponding to variable group $l$, and $p_l = |\beta^{(l)}|$.

Initially introduced by \citet{friedman2010note}, \gls{sgl} regularization tries to promote sparsity within groups (through the Lasso) and between groups (through the Group-Lasso). The hope is that incorporating the group structure of the input variables can increase accuracy and sparsity \citep{simon2013sparse}. \citet{simon2013sparse} showed that \gls{sgl} could perform as well as or better than the Lasso and the Group-Lasso when grouped inputs (\emph{e.g.,} pathways) were available, depending on the exact problem at hand. In addition, \gls{sgl} was generally able to achieve at least as high group-level sparsity as the Group-Lasso while achieving higher overall sparsity (since \gls{sgl} was able to include only some variables from a specific group). \citet{simon2013sparse} made their code available in an \emph{R} package, \emph{SGL} \citep{sgl2019}, in which they set $\alpha=0.95$ as the default.

\gls{sgl} regularization can also be applied to neural networks, as done for example by \citet{scardapane2017group}, who treated "all outgoing weights from a neuron as a single group" \citep{scardapane2017group} and showed that this regularization combined with clipping weights below a certain threshold to zero produced much sparser networks at comparable accuracy relative to L1, L2, and Group-Lasso regularization. \gls{sgl} regularization has also been used in more applied settings. \citet{lemsara2020pathme} proposed a multi-modal autoencoder, deemed \emph{PathME}, which leverages \gls{sgl} regularization between different multi-omics feature groups to perform, in combination with sparse non-negative matrix factorization, clustering of \gls{tcga} patients.

There is also work on the weighted \gls{sgl}, that is, using different penalty factors for the variables (or group of variables) in the Group-Lasso or Lasso terms, similar to what \emph{ipflasso} \citep{boulesteix2017ipf} proposed to do for the Lasso. \citet{che2020genetic} used the weighted Sparse-Group-Lasso for genetic variant detection and placed a prior information weight on each element of the vector \(\beta\) in the Lasso term based on prior biological information which they obtained from their input data  (Equation \ref{eq:weighted-sgl}).

{
\begin{equation}
  \beta \in \argmin_\beta \sum_{i=1}^n L(y_i, \hat y_i) + (1-\alpha)\lambda \sum_{l=1}^m \sqrt{p_l}||\beta^{(l)}||_2 + \alpha \lambda ||\omega \beta||_1, \ \ \ \alpha \in [0, 1]
  \label{eq:weighted-sgl}
\end{equation}
}

Where $\omega$ is a weight vector adjusting the weight of the Lasso penalty for each variable. $\beta$ is the full parameter vector to be estimated, $\beta^{(l)}$ are the parameters corresponding to variable group $l$, and $p_l = |\beta^{(l)}|$. Although \citet{che2020genetic} chose to only weight the coefficients within the Lasso penalty, the same is possible for the Group-Lasso term (Equation \ref{eq:weighted-sgl-both})

{

\begin{equation}
  \beta \in \argmin_\beta \sum_{i=1}^n L(y_i, \hat y_i) + (1-\alpha)\lambda \sum_{l=1}^m \sqrt{p_l \zeta^{(l)}}||\beta^{(l)}||_2 + \alpha \lambda ||\omega \beta||_1, \ \ \ \alpha \in [0, 1]
  \label{eq:weighted-sgl-both}
\end{equation}
}

Where $\zeta$ is a weight vector adjusting the weight of the Group-Lasso penalty for each input variable group.


\documentstyle[11pt,reduce]{article}
\date{}
\title{NUMERIC}
\author{Herbert Melenk \\
Konrad--Zuse--Zentrum f\"ur Informationstechnik Berlin \\
Takustra\"se 7 \\
D--14195 Berlin -- Dahlem \\
Federal Republic of Germany \\[0.05in]
E--mail: melenk@zib.de}
\begin{document}
\maketitle

\index{NUMERIC package}
The {\small NUMERIC} package implements some numerical (approximative)
algorithms for \REDUCE, based on the \REDUCE\ rounded mode
arithmetic. These algorithms are implemented for standard cases.
They should not be called for ill-conditioned problems;
please use standard mathematical libraries for these.

\section{Syntax}

\subsection{Intervals, Starting Points}

Intervals are generally coded as lower bound and
upper bound connected by the operator \verb+`..'+, usually
associated to a variable in an
equation. E.g.

\begin{verbatim}
     x= (2.5 .. 3.5)
\end{verbatim}

means that the variable x is taken in the range from 2.5 up to
3.5. Note, that the bounds can be algebraic
expressions, which, however, must evaluate to numeric results.
In cases where an interval is returned as the result, the lower
and upper bounds can be extracted by the \verb+PART+ operator
as the first and second part respectively.
A starting point is specified by an equation with a numeric
righthand side, e.g.

\begin{verbatim}
     x=3.0
\end{verbatim}

If for multivariate applications several coordinates must be
specified by intervals or as a starting point, these
specifications can be collected in one parameter (which is then
a list) or they can be given as separate parameters
alternatively. The list form is more appropriate when the
parameters are built from other REDUCE calculations in an
automatic style, while the flat form is more convenient
for direct interactive input.

\subsection{Accuracy Control}

The keyword parameters $accuracy=a$ and $iterations=i$, where
$a$ and $i$ must be positive integer numbers, control the
iterative algorithms: the iteration is continued until
the local error is below $10^{-a}$; if that is impossible
within $i$ steps, the iteration is terminated with an
error message. The values reached so far are then returned
as the result.

\subsection{tracing}

Normally the algorithms produce only a minimum of printed
output during their operation. In cases of an unsuccessful
or unexpected long operation a trace of the iteration can be
printed by setting

\begin{verbatim}
    on trnumeric;
\end{verbatim}

\section{Minima}

The Fletcher Reeves version of the $steepest\ descent$
algorithms is used to find the minimum of a
function of one or more variables. The
function must have continuous partial derivatives with respect to all
variables. The starting point of the search can be
specified; if not, random values are taken instead.
The steepest descent algorithms in general find only local
minima.

Syntax:

\begin{description}
\item[NUM\_MIN] $(exp, var_1[=val_1] [,var_2[=val_2] \ldots]$

$             [,accuracy=a][,iterations=i]) $

or

\item[NUM\_MIN] $(exp, \{ var_1[=val_1] [,var_2[=val_2] \ldots] \}$

$             [,accuracy=a][,iterations=i]) $

where $exp$ is a function expression,

$var_1, var_2, \ldots$ are the variables in $exp$ and
$val_1,val_2, \ldots$ are the (optional) start values.

NUM\_MIN tries to find the next local minimum along the descending
path starting at the given point. The result is a list
with the minimum function value as first element followed by a list
of equations, where the variables are equated to the coordinates
of the result point.
\end{description}

Examples:

\begin{verbatim}
   num_min(sin(x)+x/5, x);

   {4.9489585606,{X=29.643767785}}

   num_min(sin(x)+x/5, x=0);

   { - 1.3342267466,{X= - 1.7721582671}}

   % Rosenbrock function (well known as hard to minimize).
   fktn := 100*(x1**2-x2)**2 + (1-x1)**2;
   num_min(fktn, x1=-1.2, x2=1, iterations=200);

   {0.00000021870228295,{X1=0.99953284494,X2=0.99906807238}}

\end{verbatim}

\section{Roots of Functions/ Solutions of Equations}

An adaptively damped Newton iteration is used to find
an approximative zero of a function, a function vector or the solution
of an equation or an equation system. Equations are
internally converted to a difference of lhs and rhs such
that the Newton method (=zero detection) can be applied. The expressions
must have continuous derivatives for all variables.
A starting point for the iteration can be given. If not given,
random values are taken instead. If the number of
forms is not equal to the number of variables, the
Newton method cannot be applied. Then the minimum
of the sum of absolute squares is located instead.

With ON COMPLEX solutions with imaginary parts can be
found, if either the expression(s) or the starting point
contain a nonzero imaginary part.

Syntax:

\begin{description}
\item[NUM\_SOLVE]  $(exp_1, var_1[=val_1][,accuracy=a][,iterations=i])$

or

\item[NUM\_SOLVE]  $(\{exp_1,\ldots,exp_n\},
   var_1[=val_1],\ldots,var_n[=val_n]$
\item[\ \ \ \ \ \ \ \ ]$[,accuracy=a][,iterations=i])$

or

\item[NUM\_SOLVE]  $(\{exp_1,\ldots,exp_n\},
   \{var_1[=val_1],\ldots,var_n[=val_n]\}$
\item[\ \ \ \ \ \ \ \ ]$[,accuracy=a][,iterations=i])$

where $exp_1, \ldots,exp_n$ are function expressions,

      $var_1, \ldots, var_n$ are the variables,

      $val_1, \ldots, val_n$ are optional start values.

NUM\_SOLVE tries to find a zero/solution of the expression(s).
Result is a list of equations, where the variables are
equated to the coordinates of the result point.

The Jacobian matrix is stored as a side effect in the shared
variable JACOBIAN.

\end{description}

Example:

\begin{verbatim}
    num_solve({sin x=cos y, x + y = 1},{x=1,y=2});

    {X= - 1.8561957251,Y=2.856195584}

    jacobian;

    [COS(X)  SIN(Y)]
    [              ]
    [  1       1   ]
\end{verbatim}

\section{Integrals}

For the numerical evaluation of univariate integrals over a finite
interval the following strategy is used:
\begin{enumerate}
\item If the function has an antiderivative in close form
    which is bounded in the integration interval, this
    is used.
\item Otherwise a Chebyshev approximation is computed,
    starting with order 20, eventually up to order 80.
    If that is recognized as sufficiently convergent
    it is used for computing the integral by directly
    integrating the coefficient sequence.
\item If none of these methods is successful, an
    adaptive multilevel quadrature algorithm is used.
\end{enumerate}
For multivariate integrals only the adaptive quadrature is used.
This algorithm tolerates isolated singularities.
The value $iterations$ here limits the number of
local interval intersection levels.
$Accuracy$ is a measure for the relative total discretization
error (comparison of order 1 and order 2 approximations).

Syntax:

\begin{description}
\item[NUM\_INT] $(exp,var_1=(l_1 .. u_1)[,var_2=(l_2 .. u_2)\ldots]$
\item[\ \ \ \ \ \ ]$[,accuracy=a][,iterations=i])$

where $exp$ is the function to be integrated,

$var_1, var_2 , \ldots$ are the integration variables,

$l_1, l_2 , \ldots$ are the lower bounds,

$u_1, u_2 , \ldots$ are the upper bounds.

Result is the value of the integral.

\end{description}

Example:

\begin{verbatim}
    num_int(sin x,x=(0 .. pi));

    2.0000010334
\end{verbatim}

\section{Ordinary Differential Equations}

A Runge-Kutta method of order 3 finds an approximate graph for
the solution of a ordinary differential equation
real initial value problem.

Syntax:
\begin{description}
\item[NUM\_ODESOLVE]($exp$,$depvar=dv$,$indepvar$=$(from .. to)$

$                   [,accuracy=a][,iterations=i]) $

where

$exp$ is the differential expression/equation,

$depvar$ is an identifier representing the dependent variable
(function to be found),

$indepvar$ is an identifier representing the independent variable,

$exp$ is an equation (or an expression implicitly set to zero) which
contains the first derivative of $depvar$ wrt $indepvar$,

$from$ is the starting point of integration,

$to$ is the endpoint of integration (allowed to be below $from$),

$dv$ is the initial value of $depvar$ in the point $indepvar=from$.

The ODE $exp$ is converted into an explicit form, which then is
used for a Runge Kutta iteration over the given range. The
number of steps is controlled by the value of $i$
(default: 20).
If the steps are too coarse to reach the desired
accuracy in the neighborhood of the starting point, the number is
increased automatically.

Result is a list of pairs, each representing a point of the
approximate solution of the ODE problem.
\end{description}

Example:

\begin{verbatim}

    num_odesolve(df(y,x)=y,y=1,x=(0 .. 1), iterations=5);

 {{0.0,1.0},{0.2,1.2214},{0.4,1.49181796},{0.6,1.8221064563},

  {0.8,2.2255208258},{1.0,2.7182511366}}

\end{verbatim}

Remarks:

\begin{enumerate}

\item[--] If in $exp$ the differential is not isolated on the lefthand side,
please ensure that the dependent variable is explicitly declared
using a \verb+DEPEND+ statement, e.g.

\begin{verbatim}
    depend y,x;
\end{verbatim}

otherwise the formal derivative will be computed to zero by REDUCE.

\item[--] The REDUCE package SOLVE is used to convert the form into
an explicit ODE. If that process fails or has no unique result,
the evaluation is stopped with an error message.

\end{enumerate}

\section{Bounds of a Function}

Upper and lower bounds of a real valued function over an
interval or a rectangular multivariate domain are computed
by the operator BOUNDS. The algorithmic basis is the computation
with inequalities: starting from the interval(s) of the
variables, the bounds are propagated in the expression
using the rules for inequality computation. Some knowledge
about the behavior of special functions like ABS, SIN, COS, EXP, LOG,
fractional exponentials etc. is integrated and can be evaluated
if the operator BOUNDS is called with rounded mode on
(otherwise only algebraic evaluation rules are available).

If BOUNDS finds a singularity within an interval, the evaluation
is stopped with an error message indicating the problem part
of the expression.

Syntax:

\begin{description}
\item[BOUNDS]$(exp,var_1=(l_1 .. u_1) [,var_2=(l_2 .. u_2) \ldots])$

\item[{\it BOUNDS}]$(exp,\{var_1=(l_1 .. u_1) [,var_2=(l_2 .. u_2)\ldots]\})$

where $exp$ is the function to be investigated,

$var_1, var_2 , \ldots$ are the variables of exp,

$l_1, l_2 , \ldots$  and  $u_1, u_2 , \ldots$ specify the area (intervals).

$BOUNDS$ computes upper and lower bounds for the expression in the
given area. An interval is returned.

\end{description}

Example:

\begin{verbatim}

    bounds(sin x,x=(1 .. 2));

    {-1,1}

    on rounded;
    bounds(sin x,x=(1 .. 2));

    0.84147098481 .. 1

    bounds(x**2+x,x=(-0.5 .. 0.5));

     - 0.25 .. 0.75

\end{verbatim}

\section{Chebyshev Curve Fitting}

The operator family $Chebyshev\_\ldots$ implements approximation
and evaluation of functions by the Chebyshev method.
Let $T_n^{(a,b)}(x)$ be the Chebyshev polynomial of order $n$
transformed to the interval $(a,b)$. Then a function $f(x)$ can be
approximated in $(a,b)$ by a series

$f(x) \approx \sum_{i=0}^N c_i T_i^{(a,b)}(x)$

The operator $Chebyshev\_fit$ computes this approximation and
returns a list, which has as first element the sum expressed
as a polynomial and as second element the sequence
of Chebyshev coefficients ${c_i}$.
$Chebyshev\_df$ and $Chebyshev\_int$ transform a Chebyshev
coefficient list into the coefficients of the corresponding
derivative or integral respectively. For evaluating a Chebyshev
approximation at a given point in the basic interval the
operator $Chebyshev\_eval$ can be used. Note that
$Chebyshev\_eval$ is based on a recurrence relation which is
in general more stable than a direct evaluation of the
complete polynomial.

\begin{description}
\item[CHEBYSHEV\_FIT] $(fcn,var=(lo .. hi),n)$

\item[CHEBYSHEV\_EVAL] $(coeffs,var=(lo .. hi),var=pt)$

\item[CHEBYSHEV\_DF] $(coeffs,var=(lo .. hi))$

\item[CHEBYSHEV\_INT] $(coeffs,var=(lo .. hi))$

where $fcn$ is an algebraic expression (the function to be
fitted), $var$ is the variable of $fcn$, $lo$ and $hi$ are
numerical real values which describe an interval ($lo < hi$),
$n$ is the approximation order,an integer $>0$, set to 20 if missing,
$pt$ is a numerical value in the interval and $coeffs$ is
a series of Chebyshev coefficients, computed by one of
$CHEBYSHEV\_COEFF$, $\_DF$ or $\_INT$.
\end{description}

Example:

\begin{verbatim}

on rounded;

w:=chebyshev_fit(sin x/x,x=(1 .. 3),5);

               3           2
w := {0.03824*x  - 0.2398*x  + 0.06514*x + 0.9778,

      {0.8991,-0.4066,-0.005198,0.009464,-0.00009511}}

chebyshev_eval(second w, x=(1 .. 3), x=2.1);

0.4111

\end{verbatim}

\section{General Curve Fitting}

The operator $NUM\_FIT$ finds for a set of
points the linear combination of a given set of
functions (function basis) which approximates the
points best under the objective of the least squares
criterion (minimum of the sum of the squares of the deviation).
The solution is found as zero of the
gradient vector of the sum of squared errors.

Syntax:

\begin{description}
\item[NUM\_FIT] $(vals,basis,var=pts)$

where $vals$ is a list of numeric values,

$var$ is a variable used for the approximation,

$pts$ is a list of coordinate values which correspond to $var$,

$basis$ is a set of functions varying in $var$ which is used
  for the approximation.

\end{description}

The result is a list containing as first element the
function which approximates the given values, and as
second element a list of coefficients which were used
to build this function from the basis.

Example:

\begin{verbatim}

     % approximate a set of factorials by a polynomial
    pts:=for i:=1 step 1 until 5 collect i$
    vals:=for i:=1 step 1 until 5 collect
            for j:=1:i product j$

    num_fit(vals,{1,x,x**2},x=pts);

                   2
    {14.571428571*X  - 61.428571429*X + 54.6,{54.6,

         - 61.428571429,14.571428571}}

    num_fit(vals,{1,x,x**2,x**3,x**4},x=pts);

                   4                 3
    {2.2083333234*X  - 20.249999879*X

                      2
      + 67.791666154*X  - 93.749999133*X

      + 44.999999525,

     {44.999999525, - 93.749999133,67.791666154,

       - 20.249999879,2.2083333234}}

\end{verbatim}

\section{Function Bases}

The following procedures compute sets of functions
e.g. to be used for approximation.
All procedures have
two parameters, the expression to be used as $variable$
(an identifier in most cases) and the
order of the desired system.
The functions are not scaled to a specific interval, but
the $variable$ can be accompanied by a scale factor
and/or a translation
in order to map the generic interval of orthogonality to another
(e.g. $(x- 1/2 ) * 2 pi$).
The result is a function list with ascending order, such that
the first element is the function of order zero and (for
the polynomial systems) the function of order $n$ is the $n+1$-th
element.

\begin{verbatim}

     monomial_base(x,n)       {1,x,...,x**n}
     trigonometric_base(x,n)  {1,sin x,cos x,sin(2x),cos(2x)...}
     Bernstein_base(x,n)      Bernstein polynomials
     Legendre_base(x,n)       Legendre polynomials
     Laguerre_base(x,n)       Laguerre polynomials
     Hermite_base(x,n)        Hermite polynomials
     Chebyshev_base_T(x,n)    Chebyshev polynomials first kind
     Chebyshev_base_U(x,n)    Chebyshev polynomials second kind

\end{verbatim}

Example:

\begin{verbatim}
 Bernstein_base(x,5);

         5      4       3       2
    { - X  + 5*X  - 10*X  + 10*X  - 5*X + 1,

           4      3      2
     5*X*(X  - 4*X  + 6*X  - 4*X + 1),

         2      3      2
     10*X *( - X  + 3*X  - 3*X + 1),

         3   2
     10*X *(X  - 2*X + 1),

        4
     5*X *( - X + 1),

      5
     X }

\end{verbatim}

\end{document}


\section{Isostatic Adjustment}
The ice sheet model includes simple approximations for calculating isostatic adjustment. These approximations depend on how the lithosphere and the mantle are treated. For each subsystem there are two models. The lithosphere can be described as a
\begin{description}
\item[\textbf{local lithosphere:}] the flexural rigidity of the lithosphere is ignored, i.e. this is equivalent to ice floating directly on the asthenosphere;
\item[\textbf{elastic lithosphere:}] the flexural rigidity is taken into account;
\end{description}
while the mantle is treated as a
\begin{description}
\item [\textbf{fluid mantle:}] the mantle behaves like a non-viscous fluid, isostatic equilibrium is reached instantaneously;
\item [\textbf{relaxing mantle:}] the flow within the mantle is approximated by an exponentially decaying hydrostatic response function, i.e. the mantle is treated as a viscous half space.
\end{description}

\subsection{Calculation of ice-water load}
At each isostasy time-step, the load of ice and water is calculated, as an
equivalent mantle-depth ($L$). If the basal elevation is above sea-level, then the
load is simply due to the ice:
\begin{equation}
L=\frac{\rho_i}{\rho_m}H,
\label{load_land_ice}
\end{equation}
where $H$ is the ice thickness, with $\rho_i$ and $\rho_m$ being the densities
of the ice and mantle respectively. In the case where the bedrock is below
sea-level, the load is calculated is that due to a change in sea-level rise and/or
the presence of non-floating ice. When the ice is floating ($\rho_i
H<\rho_o(z_0-h)$), the load is only due to sea-level changes
\begin{equation}
L=\frac{\rho_o}{\rho_m}z_0,
\label{load_sea_float}
\end{equation}
whereas when the ice is grounded, it displaces the water, and adds an
additional load:
\begin{equation}
L=\frac{\rho_i H+\rho_o h}{\rho_m}.
\label{load_sea_grounded}
\end{equation}
here, $\rho_o$ is the density of sea water, $z_0$ is the change in sea-level
relative to a reference level and $h$ is the bedrock elevation relative to the
same reference level. The value of $h$ will be negative for submerged bedrock,
hence the plus sign in (\ref{load_sea_grounded}).

\subsection{Elastic lithosphere model}
This is model is selected by setting \texttt{lithosphere = 1} in the
configuration file. By simulatuing the deformation of the lithosphere, the
deformation seen by the aesthenosphere beneath is calculated. In the absence of this
model, the deformation is that due to Archimedes' Principle, as though the
load were floating on the aesthenosphere.

The elastic lithosphere model is based on work by \cite{Lambeck1980}, and its
implementation is fully described in \cite{Hagdorn2003}. The lithosphere
model only affects the geometry of the deformation --- the timescale for
isostatic adjustment is controlled by the aesthenosphere model.

The load due to a single (rectangular) grid point is approximated as being
applied to a disc of the same area. The deformation due to a disc of ice of
radius $A$ and thickness $H$ is given by these expressions. For $r<A$:
\begin{equation}
w(r)=\frac{\rho_i H}{\rho_m}\left[1+C_1\,\mathrm{Ber}\left(\frac{r}{L_r}\right)+C_2\,\mathrm{Bei}\left(\frac{r}{L_r}\right)\right],
\end{equation}
and for $r\geq A$:
\begin{equation}
w(r)=\frac{\rho_i
  H}{\rho_m}\left[D_1\,\mathrm{Ber}\left(\frac{r}{L_r}\right)+D_2\,\mathrm{Bei}\left(\frac{r}{L_r}\right)
+D_3\,\mathrm{Ker}\left(\frac{r}{L_r}\right)+D_4\,\mathrm{Kei}\left(\frac{r}{L_r}\right)\right],
\end{equation}
where $\mathrm{Ber}(x)$, $\mathrm{Bei}(x)$, $\mathrm{Ker}(x)$ and
$\mathrm{Kei}(x)$ are Kelvin functions of zero order, $L_r=(D/\rho_m
g))^{1/4}$ is the radius of relative stiffness, and $D$ is the flexural
rigidity. The constants $C_i$ and $D_i$ are given by
\begin{equation}
\begin{array}{rcl}
C_1&=&a\,\mathrm{Ker}'(a)\\
C_2&=&-a\,\mathrm{Ker}'(a)\\
D_1&=&0\\
D_2&=&0\\
D_3&=&a\,\mathrm{Ber}'(a)\\
D_4&=&-a\,\mathrm{Ber}'(a).
\end{array}
\end{equation}
Here, the prime indicates the first spatial derivative of the Kelvin functions.

\subsection{Relaxing aesthenosphere model}
If a fluid mantle is selected, it adjusts instantly to changes in lithospheric
loading. However, a relaxing mantle is also available.

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "isos"
%%% End: