\label{sec:swaption}
To the best of our knowledge, Liu proposed the only implementation of atomic swaption which does not require either blockchain to support smart contracts \cite{liu2018atomic}. Afterward, Tefagh~\etal~ designed atomic bonded cross-chain debt (ABCD) as the first practical cross-chain bond platform in the form of atomic swaptions \cite{tefagh2020atomic}. In this paper, we have designed a more general model of atomic swaption named \emph{\MetaSwaption} which is abstractly shown in Fig~\ref{fig:moc-swaption}.

In what follows, we are going to briefly explain every module of the \MetaSwaption, and through the rest of this paper, we will use this general form as the building block for making some new application-specific swaptions.

\begin{figure}
    \centering
    \includegraphics[width=\textwidth,height=\textheight,keepaspectratio]{figures/meta-swaption.png}
    \caption{General overview of the \MetaSwaption}
    \label{fig:moc-swaption}
\end{figure}

Similar to atomic swap and atomic swaption, HTLCs are used so that a party can make decisions by revealing a secret before a locktime or letting the locktime expire. In \MetaSwaption also there are four types of such secrets:
\begin{itemize}
    \item \Aone key
    \item \keyone key
    \item \Atwo key
    \item \Delegation key
\end{itemize}

Note that in HTLC contracts, generaly the locktime of the holder of the secret has to be greater than the other party.
%\ahC{It has to be greater}
% We call the one who buys the swaption, the swpation \SwaptionOwner. In our cases it is mostly Alice.

In the Fig~\ref{fig:moc-swaption}, a \MetaSwaption is
divided into five different parts, each representing a particular stage in the swaption process.

Note that in every module introduced in this paper, all transactions in the execution process are exchanged and signed by their corresponding parties before anything goes on chain. During this not-yet-confirmed transactions sharing, all parties are assured that no one can steal their money.

\begin{itemize}

    \item \textbf{Contract funding}: The funding for the swaption buyer (swaption owner) consists of premium and for the seller only margin. Depending on application, the \SwaptionOwner may include margin in her funding.
    The \SwaptionOwner has a relatively small amount of time to reveal \Aone key to buy the option. If she buys the option, premium goes to the seller and margins go to the principal deposition contracts.
    %  Each party sends his margin to a HTLC contract. The swaption buyer also pays a premium to buy the option from the seller. After the exchange of funding transactions, it is time for the swaption buyer to decide weather she buys the option or not. If she reveals the \Aone key, premium goes to the seller and margins go to the margin deposit contracts.

    \item \textbf{Principal deposition}: After buying the swpation, each party has to deposit his principal within a specified time interval. If both parties cooperate, the next stage begins.
    The principal deposition transactions might have sighash type of any-one-can-pay\footnote{ANYONECANPAY} since nobody knows all of its inputs in the first place. In this case, since we do not know all the inputs of this transaction at the time of signing, we would not have the correct transaction ID \cite{bip143}. Therefore, we can not create the transactions that get their inputs form this transaction until we discover what all the inputs of the this transaction are going to be. These inputs are only known at the time of broadcasting, not at the time of creating the transactions. So, at the time of creating the transactions in the next stage, the ID of the principal deposition transaction is unknown.

    \item \textbf{Leader lock}: Depending on the application, the swaption \SwaptionOwner may decide to use a portion of her option duration as the allowed period of her late principal deposition in order to make money for her principal. In this stage, the party who deposits his principal earlier, \keyone key holder, locks his principal. After assurance of the owner's principal deposition, he is expected to reveal \keyone key determining if the swaption goes to the next stage or not.
    As mentioned in the last stage, the ID of the input of the transactions of this stage is not known at the time of creating. Thus, we have to use the sighash type of no-input\footnote{NOINPUT}\cite{bip118} that allows us to create a transaction where none of its inputs are known. So, by this sighash we can create the transactions in the leader lock stage without any inputs. Later, at the time of broadcast, since the inputs of the principal deposition transaction are known and consequently we have its ID, we can use this transaction as an input for the transactions of the leader lock stage and broadcast them.

    \item \textbf{Master lock}: In this stage, the \SwaptionOwner (\Atwo key holder) chooses weather she exercises the option or not using \Atwo key. By exercising the option, principals are exchanged and the swaption ends. By not revealing \Atwo key, letting the locktime expire, next stage begins. Like the last stage, since the inputs of the parent transaction of the transactions of this stage are not clear, we have to use the no-input sighash.

    \item \textbf{Option delegation}: This stage might be utilized in certain use-cases such as futures arbitrage discussed later. For fixing the problem of cyclic locktimes in master lock stage, we develop a method to delegate the option from \SwaptionOwner to another desired party. The newly promoted party can later decide the execution of the swaption by the \Delegation key. In this stage, like last two stages we have to use the no-input sighash.

    % \item \textbf{Trust box}: In some cases, i.e. futures arbitrage, the \SwaptionOwner needs to transfer the right of signing some transactions to a party other than swaption buyer or seller. The extents of trust box may differ for each use-cases of \MetaSwaption instances.
    % \ahC{Is this paragraph needed anymore?}
\end{itemize}

When using the no-input sighash, the party signs the transaction without specifying its inputs, so a malicious party has the ability to give any UTXO that belongs to the public key of this party. Hence, each party has to create a separate public key for each transaction that she makes.
Using the stages described above, we can generate new instances of \MetaSwaption targeting swaptions or bonds. Based on different goals, we can choose which of these stages appear in our instance. In the following subsections, we describe different swaptions devised for a variety of use-cases:
\begin{itemize}
    \item Early deposition swaption
    \item Late deposition swaption
    \item Margin-free limited swaption
\end{itemize}
Notice that in Fig~\ref{fig:swaption-early-deposition}, Fig~\ref{fig:swaption-late-deposition}, and Fig~\ref{fig:swaption-margin-free-limited} the locktimes $P$,$T$,$M$,$E$, and $T'$ are calculated with respect to  a common origin of time. In other words, the current time of the swaption establishment has to be added to all the locktimes shown in these stages, because they are not relative but absolute times.
% Each of these components has different procedure. For further analysing the effectiveness of each component, we design a time elapsed experiment aiming to analyse the worst case running time of them.

% \begin{experiment}
% {Time Elapsed}\\
% Given $p_{buyer}$ and $p_{seller}$ the buyer and the seller of the component $\mathcal{C}$, the $\mathcal{T}[\mathcal{C}]$ is the minimum spend locktime during the execution runtime of $\mathcal{C}$ if $p_{seller}$ adversarially waits in every steps until the last moments.
% \end{experiment}

% We discus the second type in detail. The other types are discussed in \Apn{\ref{app:conv-swaption}} and \Apn{\ref{app:margin-free-swaption}}.

% \ahC{Somewhere we have to mention that we split the option time into two parts, one is time for revealing \keyone key and the other one is \Atwo key. Then we can say that Alice is settling other swaptions in the case of arbitrage during option time.}

\subsection{Early Deposition Swaption}
\label{app:conv-swaption}

This type of swaption is the conventional form that is first introduced in \cite{liu2018atomic}. Alice wants to exchange her ACoins with Bob's BCoins. We rebuild this type with our \MetaSwaption extended form in the way depicted in Fig~\ref{fig:swaption-early-deposition}. We begin analysing each stage of this type by explaining every possible scenarios as follows:

\begin{itemize}
    \item \textbf{Contract funding}: Alice and Bob broadcast their funding transactions. Alice's includes margin and premium and Bob's includes margin worth equal to Alice's margin.

    \item \textbf{Principal deposition}: In this stage, Bob is waiting for Alice to deposit her principal. There are two possible scenarios:
    \begin{itemize}
        \item Alice does not deposit her principal. In this case, Bob also defaults and their margins are exchanged.
        \item Alice deposits her principal but Bob does not. In this case, Alice is in master lock stage. So, she reveals \Atwo and takes Bob's margin besides her own principal.
        % Alice is already in option contract stage and she can take the ownership of both her principal and Bob's margin by revealing the \Atwo key and broadcasting Bob's default transaction.
    \end{itemize}
     But if Bob fails to deposit his principal,

    \item \textbf{Master lock}: If both parties go to this stage, Alice can then use her option as mentioned earlier.
\end{itemize}

\begin{figure}
    \centering
    \includegraphics[width=\textwidth]{figures/early dep.png}
    \caption{Early deposition swaption where the buyer (Alice) deposits her principal earlier than the seller (Bob). The pink-bordered transactions are broadcasted by Alice and the blue-bordered ones by Bob.}
    \label{fig:swaption-early-deposition}
\end{figure}
% \ahC{Shall we mention that 546 Satoshis can not be counted as margin?}

\subsection{Late Deposition Swaption}
In this type of swaption the swaption \SwaptionOwner, Alice, needs the seller, Bob, to deposit his principal before her, so that she can settle other deals on other contracts before depositing her principal in this swaption. Alice can exploit this type of swaption in the situations where she does not own enough budget for her principal and she is going to make the required amount of capital using what she has takes from Bob. Since Bob has to wait a longer time for Alice to deposit, Alice has to pay more amount of premium compared to the early deposition swaption.
One example use-case can be the futures arbitrages which will be discussed later. The overview of this swaption is shown on Fig.~\ref{fig:swaption-late-deposition}.

\begin{itemize}
    \item \textbf{Contract funding:} The two parties broadcast their funding transactions. Alice includes margin in her funding and an amount of guarantee is added to Bob's funding besides his margin. We can use this guarantee amount to punish Bob in case of cheating. If he behaves normally, the guarantee will return back to him. The amount of guarantee is negotiable depending on the use-case.

    \item \textbf{Principal deposition:} When this stage begins Alice is waiting for Bob to deposit his principal. If Bob defaults, then Alice has two options to punish him:
    \begin{itemize}
        \item She defaults, loses her margin and takes Bob's guarantee and margin.
        \item She deposits her principal and enters the next stage.
    \end{itemize}

    \item \textbf{Leader lock}:
    There are four possible scenarios:
    \begin{itemize}
        \item In the last stage, Bob deposited his principal but Alice did not. Now Bob does not reveal \keyone key, so that Alice's margin is exchanged with Bob's margin.
        \item Both have deposited their principals and Bob does not reveal \keyone key. Broadcasting the Alice's leader lock transactions, Alice gets Bob's guarantee.
        \item In the last stage, Alice deposited her principal but Bob did not (Second way to punish Bob). Now Alice broadcasts the Alice's leader lock expiration transaction and takes all of her principal back in addition to Bob's margin and guarantee that she has previously taken in the last stage.
        \item Both have deposited their principals. Bob reveals \keyone key and take back his guarantee. Afterward, they go to the next stage.
    \end{itemize}

    \item \textbf{Master lock}: It is the time for Alice to use her option. She either exercises and principals are exchanged or reveals nothing and each party gets his principal back.
\end{itemize}

\begin{figure}
    \centering
    \includegraphics[width=\textwidth,height=0.93\textheight,keepaspectratio]{figures/late dep.png}
    \caption{Late deposition swaption where the seller (Bob) deposits his principal earlier than the buyer (Alice). Pink-bordered transactions are broadcast by Alice and blue-bordered ones by Bob.}
    \label{fig:swaption-late-deposition}
\end{figure}

% Note that since Bob's only incentive is the premium (and not the margin), it is not possible that Alice uses this swaption alongside with other swaptions. For instance, in the case that Alice uses Bob's principal as her principal in another swaption, Bob can cheat on Alice by pretending to be a different person and act as the other party in the second swaption as well as the first one. Then he can steal Alice's premium in the second swaption by not exposing the \keyone key.
% Later we will prove that Alice can only use one premium guarantee in a set of swaptions which are expected to be exercised in one run simultaneously. Because every run must have only one \keyone and \Atwo and \Aone key which is impossible when we use multiple premium guarantee boxes.
% It is not possible to allow Alice to send the premium of other swaption in directly with usage of another Premium Guarantee
%\ahC{This section is changed and it is no longer standalone. Still move to appendix?}
% The detailed process of this swaption is written in \Apn{\ref{app:margin-free-swaption}}
% \ref{app:margin-free-swaption}.

% \subsection{Atomic Non-collateralized Loan}
% A modification of the {\it Late-Deposition swaption} in which Alice does not have to deposit any margin before she gets Bob's principal has the functionality of a loan with no collateral needed. As mentioned in \cite{liu2018atomic} it is not possible to have no margin for the buyer if the parties are on different blockchains. \fatemeC{explain more} Hence, in order to have this loan, the parties have to be on the same chain.

\subsection{Margin-Free Limited Swaption}

\label{app:margin-free-swaption}
Until now, it was believed that Alice's margin deposit is necessary due to the limitations of HTLCs \cite{liu2018atomic}. In this work, we propose a novel approach which allows Alice to participate in a swaption without depositing any margin.
In the beginning of contract, Bob deposits an amount of BCoin as guarantee which is sent to Alice directly by revealing \Aone key. Alice also adds an amount of ACoin equal to Bob's guarantee to her premium, though none of them will be directly sent to Bob. Later, the premium and guarantee go to Bob in all possible situations except where Bob refuses to reveal the \keyone key when Alice does not default, then he will be punished by not getting back his guarantee.
% In Fig~ \ref{fig:swaption-margin-free-limited} we use our getting a guarantee money from Bob to build the {\it Margin Free} swaption.
% In this type besides premium, Alice deposits an amount of ACoiun equal to Bob's guarantee. Her premium is not paid directly after revealing of \Aone key, but instead will be locked until Bob acts honestly up to end of the Leader Lock stage.
% Later the premium goes to Bob in all possible situations except where Bob refuses to reveal the \keyone key and he will be punished by not paying back his guarantee.
The amount of guarantee can vary depending on the Alice's need. In next section, we will explain the limitation imposed on the amount of guarantee in details. The execution procedure of the margin-free swaption is as follows:
\begin{itemize}
    \item \textbf{Contract funding}: Bob pays his margin plus an amount of guarantee which prevents him from cheating in later stages. Alice also pays extra premium which in the case of Bob's honest behaviour pays back the guarantee to Bob. Guarantee in the Bob's section is directly sent to Alice after revealing the \Aone key.

    \item \textbf{Principal deposition}: This stage is the same as previous principal deposition stages for Bob. He has M locktime to deposit his principal. If he does not, Alice takes his margin. In Alice's section, either she defaults and gives Bob the premium or she deposits her principal and goes to the next stage waiting for Bob to reveal the \keyone key.
    % \item \textbf{Premium Guarantee}: Either Alice defaults and gives Bob the premium or she deposits her principal and goes to the next stage waiting for Bob to reveal the \keyone key.

    \item \textbf{Leader key}: If Bob has not deposited his principal until M locktime, Alice will also avoid depositing her principal and gives Bob the premium and ACoin guarantee while getting his margin from him.
    % and Alice has two options ahead: 1)She deposits her principal before M + T, then broadcasts the \keyone key expiration transaction, in which her premium is refunded. Therefore, if Bob defaults and Alice does not, there would be no premium for Bob. In any other situation he gets the premium.
    If both parties have deposited their principals when this stage begins, Bob's decision whether to reveal the \keyone key or not, determines the future of the swaption. If he refuses to reveal, he takes his own money back and Alice takes her own money including premium back. In this case, Bob loses his guarantee as a punishment. Otherwise, if he reveals the \keyone key, they both go to the next stage waiting for Alice to exercise her option. Additionally, by revealing the \keyone key, Bob finishes his task and it is the time to send back his guarantee. Hence at the end of this stage, Bob's guarantee will be paid back to himself.

    \item \textbf{Option funding}: This stage is similar to the last versions of swpation.
\end{itemize}

\begin{figure}
    \centering
    \includegraphics[width=\textwidth,height=0.93\textheight]{figures/swaption-margin-free.png}
    \caption{Margin-free limited swpation where the buyer (Alice) is not supposed to deposit any margin. Pink-bordered transactions are broadcasted by Alice and blue-bordered ones by Bob.}
    \label{fig:swaption-margin-free-limited}
\end{figure}
% \fatemeC{mention auction as an application for the margin-free swaption. you can use the aucion section primarily written}

Locktimes in all of these swaptions have to stick to some general rules:
\begin{itemize}
    \item $T$ is the minimum amount of time or number of blocks needed for a mined transaction to be confirmed. In bitcoin it is $6$ blocks which is approximately achieved in 1 hour.

    \item $M$ is the time for Bob to deposit principal. Hence, can be equal or greater than $T$.

    \item $T'$ can be relatively large, since it gives Alice the time she needs to deposit her principal.

    \item $P$ is the locktime that Bob has to reveal the \keyone key. This has to be larger than $T'$ so that Alice has to deposit before Bob's revealing time elapses.

    \item $E$ is the time for Alice's option exercise which can be relatively large, since it is an option.
\end{itemize}
Note that we use $T$ to make two locktimes differ in a reasonable amount of time to prevent cheating. However, this is the minimum difference needed. Hence, everywhere $T$ is used, we can replace it with different larger values. We use $T$ for simplicity though.


\providecommand{\tightlist}{%
  \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}

\subsection{Leabra Algorithm
Equations}\label{leabra-algorithm-equations}

The pseudocode for Leabra is given here, showing exactly how the pieces
of the algorithm fit together, using the equations and variables from
the actual code. The implementation contains a number of optimizations
(including vectorization and GPU code), but this provides the core math
in simple form.

See the \texttt{Matlab} directory in the emergent \url{svn} source
directory for a complete implementation of these equations in Matlab,
coded by Sergio Verduzco-Flores --- this can be a lot simpler to read
than the highly optimized C++ source code.

\subsubsection{Timing}\label{timing}

Leabra is organized around the following timing, based on an
internally-generated alpha-frequency (10 Hz, 100 msec periods) cycle of
expectation followed by outcome, supported by neocortical circuitry in
the deep layers and the thalamus, as hypothesized in the
\url{DeepLeabra} extension to standard Leabra:

\begin{itemize}
\tightlist
\item
  A \textbf{Trial} lasts 100 msec (10 Hz, alpha frequency), and
  comprises one sequence of expectation --- outcome learning, organized
  into 4 quarters.

  \begin{itemize}
  \tightlist
  \item
    Biologically, the deep neocortical layers (layers 5, 6) and the
    thalamus have a natural oscillatory rhythm at the alpha frequency.
    Specific dynamics in these layers organize the cycle of expectation
    vs. outcome within the alpha cycle.
  \end{itemize}
\item
  A \textbf{Quarter} lasts 25 msec (40 Hz, gamma frequency) --- the
  first 3 quarters (75 msec) form the expectation / minus phase, and the
  final quarter are the outcome / plus phase.

  \begin{itemize}
  \tightlist
  \item
    Biologically, the superficial neocortical layers (layers 2, 3) have
    a gamma frequency oscillation, supporting the quarter-level
    organization.
  \end{itemize}
\item
  A \textbf{Cycle} represents 1 msec of processing, where each neuron
  updates its membrane potential etc according to the above equations.
\end{itemize}

\subsubsection{Variables}\label{variables}

LeabraUnits are organized into LeabraLayers, which sometimes have unit
groups (which are now typically purely virtual, not actual Unit\_Group
objects). The LeabraUnit has the following key parameters, along with a
number of others that are used for other non-default algorithms and
various optimizations, etc.

\begin{itemize}
\tightlist
\item
  \textbf{act} = activation sent to other units
\item
  \textbf{act\_nd} = non-depressed activation --- prior to application
  of any short-term plasticity
\item
  \textbf{net\_raw} = raw netinput, prior to time-averaging
\item
  \textbf{net} = time-averaged excitatory conductance (net input)
\item
  \textbf{gc\_i} = inhibitory conductance, computed from FFFB inhibition
  function typically
\item
  \textbf{I\_net} = net current, combining excitatory, inhibitory, and
  leak channels
\item
  \textbf{v\_m} = membrane potential
\item
  \textbf{v\_m\_eq} = equilibrium membrane potential --- not reset by
  spikes --- just keeps integrating
\item
  \textbf{adapt} = adaptation current
\item
  \textbf{avg\_ss} = super-short term running average activation
\item
  \textbf{avg\_s} = short-term running average activation, integrates
  over avg\_ss, represents plus phase learning signal
\item
  \textbf{avg\_m} = medium-term running average activation, integrates
  over avg\_s, represents minus phase learning signal
\item
  \textbf{avg\_l} = long-term running average activation, integrates
  over avg\_m, drives long-term floating average for Hebbian learning
\item
  \textbf{avg\_l\_lrn} = how much to use the avg\_l-based Hebbian
  learning for this receiving unit's learning --- in addition to the
  basic error-driven learning --- this can optionally be dynamically
  updated based on the avg\_l factor and average level of error in the
  receiving layer, so that this Hebbian learning constraint can be
  stronger as a unit gets too active and needs to be regulated more
  strongly, and in proportion to average error levels in the layer.
\item
  \textbf{avg\_s\_eff} = effective avg\_s value used in learning ---
  includes a small fraction (.1) of the avg\_m value, for reasons
  explained below.
\end{itemize}

Units are connected via synapses parameterized with the following
variables. These are actually stored in an optimized vector format, but
the LeabraCon object contains the variables as a template.

\begin{itemize}
\tightlist
\item
  \textbf{wt} = net effective synaptic weight between objects ---
  subject to contrast enhancement compared to fwt and swt
\item
  \textbf{dwt} = delta-wt --- change in synaptic weights due to
  learning
\item
  \textbf{dwavg} = time-averaged absolute value of weight change, for
  normalizing weight changes
\item
  \textbf{moment} = momentum integration of weight changes
\item
  \textbf{fwt} = fast weight --- used for advanced fast and slow weight
  learning dynamic --- otherwise equal to swt --- stored as
  non-contrast enhanced value
\item
  \textbf{swt} = slow weight --- standard learning rate weight ---
  stored as non-contrast enhanced value --- optional
\end{itemize}

\subsubsection{Activation Update Cycle (every 1 msec): Net input,
Inhibition,
Activation}\label{activation-update-cycle-every-1-msec-net-input-inhibition-activation}

For every cycle of activation updating, compute the net input,
inhibition, membrane potential, and activation:

\begin{itemize}
\tightlist
\item
  \textbf{Net input} (see LeabraUnitSpec.cpp for code):

  \begin{itemize}
  \tightlist
  \item
    \textbf{\texttt{net\_raw}}\texttt{\ +=\ (sum\ over\ recv\ connections\ of:)\ scale\_eff\ *\ act\ *\ wt}

    \begin{itemize}
    \item
      \textbf{scale\_eff} =
      \verb\https://grey.colorado.edu/emergent/index.php/Leabra_Netin_Scaling\ factor that includes 1/N to compute an average, plus
      wt\_scale.rel and abs relative and absolute scaling terms.
    \item
      \textbf{act} = sending unit activation
    \item
      \textbf{wt} = receiving connection weight value between sender and
      receiver
    \item
      does this very efficiently by using a sender-based computation,
      that only sends \emph{changes} (deltas) in activation values ---
      typically only a few percent of neurons send on any given cycle.
    \end{itemize}
  \item
    \textbf{\texttt{net}}\texttt{\ +=\ dt.integ\ *\ dt.net\_dt\ *\ (net\_raw\ -\ net)}

    \begin{itemize}
    \tightlist
    \item
      time integration of net input, using net\_dt (1/1.4 default), and
      global integration time constant, dt.integ (1 = 1 msec default)
    \end{itemize}
  \end{itemize}
\end{itemize}

\begin{itemize}
\tightlist
\item
  \textbf{Inhibition} (see LeabraLayerSpec.cpp for code) -- earlier versions of Leabra used an explicit k-Winners-Take-All inhibition function, but the FFFB equations here are much simpler and produce desirable flexibility in overall activation levels:

  \begin{itemize}
  \tightlist
  \item
    \textbf{\texttt{ffi}}\texttt{\ =\ ff\ *\ MAX(netin.avg\ -\ ff0,\ 0)}

    \begin{itemize}
    \tightlist
    \item
      feedforward component of inhibition with ff multiplier (1 by
      default) --- has ff0 offset and can't be negative (that's what
      the MAX(.. ,0) part does).
    \item
      \textbf{netin.avg} is average of net variable across unit group or
      layer, depending on what level this is being computed at (both are
      supported)
    \end{itemize}
  \item
    \textbf{\texttt{fbi}}\texttt{\ +=\ fb\_dt\ *\ (fb\ *\ acts.avg\ -\ fbi)}

    \begin{itemize}
    \tightlist
    \item
      feedback component of inhibition with fb multiplier (1 by default)
      --- requires time integration to dampen oscillations that
      otherwise occur --- fb\_dt = 1/1.4 default
    \end{itemize}
  \item
    \textbf{\texttt{gc\_i}}\texttt{\ =\ gi\ *\ (ffi\ +\ fbi)}

    \begin{itemize}
    \tightlist
    \item
      total inhibitory conductance, with global gi multiplier ---
      default of gi=1.8 typically produces good sparse distributed
      representations in reasonably large layers (25 units or more)
    \end{itemize}
  \end{itemize}
\end{itemize}

\begin{itemize}
\tightlist
\item
  \textbf{Membrane potential} (see LeabraUnitSpec.cpp for code)

  \begin{itemize}
  \tightlist
  \item
    \textbf{\texttt{I\_net}}\texttt{\ =\ net\ *\ (e\_rev.e\ -\ v\_m)\ +\ gc\_l\ *\ (e\_rev.l\ -\ v\_m)\ +\ gc\_i\ *\ (e\_rev.i\ -\ v\_m)\ +\ noise}

    \begin{itemize}
    \tightlist
    \item
      net current = sum of individual ionic channels: e = excitatory, l
      = leak (gc\_l is a constant, 0.1 default), and i = inhibitory
    \item
      e\_rev are reversal potentials: in normalized values derived from
      biophysical values, e\_rev.e = 1, .l = 0.3, i = 0.25
    \item
      noise is typically gaussian if added
    \end{itemize}
  \item
    if ex:
    \textbf{\texttt{I\_net}}\texttt{\ +=\ g\_bar.l\ *\ exp\_slope\ *\ exp((v\_m\ -\ thr)\ /\ exp\_slope)}

    \begin{itemize}
    \tightlist
    \item
      this is the exponential component of AdEx, if in use (typically
      only for discrete spiking), exp\_slope = .02 default
    \end{itemize}
  \item
    \textbf{\texttt{v\_m}}\texttt{\ +=\ dt.integ\ *\ dt.vm\_dt\ *\ (I\_net\ -\ adapt)}

    \begin{itemize}
    \tightlist
    \item
      in , we use a simple midpoint method that evaluates v\_m with a
      half-step time constant, and then uses this half-step v\_m to
      compute full step in above I\_net equation. vm\_dt = 1/3.3
      default.
    \item
      v\_m is always computed as in discrete spiking, even when using
      rate code, with v\_m reset to vm\_r etc --- this provides a more
      natural way to integrate adaptation and short-term plasticity
      mechanisms, which drive off of the discrete spiking.
    \end{itemize}
  \item
    \textbf{\texttt{I\_net\_r}}\texttt{\ =\ net\ *\ (e\_rev.e\ -\ v\_m\_eq)\ +\ gc\_l\ *\ (e\_rev.l\ -\ v\_m\_eq)\ +\ gc\_i\ *\ (e\_rev.i\ -\ v\_m\_eq)\ +\ noise}

    \begin{itemize}
    \tightlist
    \item
      rate-coded version of I\_net, to provide adequate coupling with
      v\_m\_eq.
    \end{itemize}
  \item
    \textbf{\texttt{v\_m\_eq}}\texttt{\ +=\ dt.integ\ *\ dt.vm\_dt\ *\ (I\_net\_r\ -\ adapt)}

    \begin{itemize}
    \tightlist
    \item
      the \emph{equilibrium} version of the membrane potential does
      \emph{not} reset with spikes, and is important for rate code per
      below
    \end{itemize}
  \end{itemize}
\end{itemize}

\begin{itemize}
\tightlist
\item
  \textbf{Activation} (see LeabraUnitSpec.cpp for code)

  \begin{itemize}
  \tightlist
  \item
    \textbf{\texttt{g\_e\_thr}}\texttt{\ =\ (gc\_i\ *\ (e\_rev\_i\ -\ thr)\ +\ gc\_l\ *\ (e\_rev\_l\ -\ thr)\ -\ adapt)\ /\ (thr\ -\ e\_rev.e)}

    \begin{itemize}
    \tightlist
    \item
      the amount of excitatory conductance required to put the neuron
      exactly at the firing threshold, thr = .5 default.
    \end{itemize}
  \item
    \texttt{if(v\_m\ \textgreater{}\ spk\_thr)\ \{\ spike\ =\ 1;\ v\_m\ =\ vm\_r;\ I\_net\ =\ 0.0\ \}\ else\ \{\ spike\ =\ 0\ \}}

    \begin{itemize}
    \tightlist
    \item
      spk\_thr is spiking threshold (1.2 default, different from rate
      code thr), vm\_r = .3 is the reset value of the membrane potential
      after spiking --- we also have an optional refractory period
      after spiking, default = 3 cycles, where the vm equations are
      simply not computed, and vm remains at vm\_r.
    \item
      if using spiking mode, then \textbf{act} = spike, otherwise, rate
      code function is below
    \end{itemize}
  \item
    \texttt{if(v\_m\_eq\ \textless{}=\ thr)\ \{\ }\textbf{\texttt{new\_act}}\texttt{\ =\ NXX1(v\_m\_eq\ -\ thr)\ \}\ else\ \{\ }\textbf{\texttt{new\_act}}\texttt{\ =\ NXX1(net\ -\ g\_e\_thr)\ \}}

    \begin{itemize}
    \tightlist
    \item
      it is important that the time to first ``spike'' be governed by
      v\_m integration dynamics, but after that point, it is essential
      that activation drive directly from the excitatory conductance
      (g\_e or net) relative to the g\_e\_thr threshold --- activation
      rates are linear in this term, but not even a well-defined
      function of v\_m\_eq --- earlier versions of Leabra only used the
      v\_m\_eq-based term, and this led to some very strange behavior.
    \item
      NXX1 = noisy-x-over-x+1 function, which is implemented using a
      lookup table due to the convolving of the XX1 function with a
      gaussian noise kernel
    \item
      \texttt{XX1(x)\ =\ gain\ *\ x\ /\ (gain\ *\ x\ +\ 1)}
    \item
      gain = 100 default
    \end{itemize}
  \item
    \textbf{\texttt{act\_nd}}\texttt{\ +=\ dt.integ\ *\ dt.vm\_dt\ *\ (new\_act\ -\ act\_nd)}

    \begin{itemize}
    \tightlist
    \item
      non-depressed rate code activation is time-integrated using same
      vm\_dt time constant as used in v\_m, from the new activation
      value
    \end{itemize}
  \item
    \textbf{\texttt{act}}\texttt{\ =\ act\_nd\ *\ syn\_tr\ (or\ just\ act\_nd)}

    \begin{itemize}
    \tightlist
    \item
      if short-term plasticity is in effect, then syn\_tr variable
      reflects the synaptic transmission efficacy, and this product
      provides the net signal sent to the receiving neurons. otherwise
      syn\_tr = 1.
    \end{itemize}
  \item
    \textbf{\texttt{adapt}}\texttt{\ +=\ dt.integ\ *\ (adapt.dt\ *\ (vm\_gain\ *\ (v\_m\ -\ e\_rev.l)\ -\ adapt)\ +\ spike\ *\ spike\_gain)}

    \begin{itemize}
    \tightlist
    \item
      adaptation current --- causes rate of activation / spiking to
      decrease over time, adapt.dt = 1/144, vm\_gain = 0.04, spike\_gain
      = .00805 defaults
    \end{itemize}
  \end{itemize}
\end{itemize}

\subsubsection{Learning}\label{learning}

Learning is based on running-averages of activation variables, described
first:

\begin{itemize}
\tightlist
\item
  \textbf{Running averages} computed continuously every cycle, and note
  the compounding form (see LeabraUnitSpec.cpp for code)

  \begin{itemize}
  \tightlist
  \item
    \textbf{\texttt{avg\_ss}}\texttt{\ +=\ dt.integ\ *\ ss\_dt\ *\ (act\_nd\ -\ avg\_ss)}

    \begin{itemize}
    \tightlist
    \item
      super-short time scale running average, ss\_dt = 1/2 default ---
      this was introduced to smooth out discrete spiking signal, but is
      also useful for rate code
    \end{itemize}
  \item
    \textbf{\texttt{avg\_s}}\texttt{\ +=\ dt.integ\ *\ act\_avg.s\_dt\ *\ (avg\_ss\ -\ avg\_s)}

    \begin{itemize}
    \tightlist
    \item
      short time scale running average, s\_dt = 1/2 default --- this
      represents the ``plus phase'' or actual outcome signal in
      comparison to avg\_m
    \end{itemize}
  \item
    \textbf{\texttt{avg\_m}}\texttt{\ +=\ dt.integ\ *\ act\_avg.m\_dt\ *\ (avg\_s\ -\ avg\_m)}

    \begin{itemize}
    \tightlist
    \item
      medium time-scale running average, m\_dt = 1/10 average --- this
      represents the ``minus phase'' or expectation signal in comparison
      to avg\_s
    \end{itemize}
  \item
    \textbf{\texttt{avg\_l}}\texttt{\ +=\ avg\_l.dt\ *\ (avg\_l.gain\ *\ avg\_m\ -\ avg\_l);\ avg\_l\ =\ MAX(avg\_l,\ min)}

    \begin{itemize}
    \tightlist
    \item
      long-term running average --- this is computed just once per
      learning trial, \emph{not every cycle} like the ones above ---
      gain = 2.5 (or 1.5 in some cases works better), min = .2, dt = .1
      by default
    \item
      same basic exponential running average as above equations
    \end{itemize}
  \item
    \textbf{\texttt{avg\_s\_eff}}\texttt{\ =\ m\_in\_s\ *\ avg\_m\ +\ (1\ -\ m\_in\_s)\ *\ avg\_s}

    \begin{itemize}
    \tightlist
    \item
      mix in some of the medium-term factor into the short-term factor
      --- this is important for ensuring that when neuron turns off in
      the plus phase (short term), that enough trace of earlier
      minus-phase activation remains to drive it into the LTD weight
      decrease region --- m\_in\_s = .1 default.
    \item
      this is now done at the unit level --- previously was done at the
      connection level which is much less efficient!
    \end{itemize}
  \end{itemize}
\end{itemize}

\begin{itemize}
  \tightlist
  \item \emph{Optional, on by default:} dynamic modulation of amount of
    Hebbian learning, based on avg\_l value and level of err in a given
    layer --- these factors make a small (few percent) but reliable
    difference in overall performance across various challenging tasks
    --- they can readily be omitted in favor of a fixed avg\_l\_lrn
    factor of around 0.0004 (with 0 for target layers --- it doesn't
    make sense to have any Hebbian learning at output layers):

    \begin{itemize}
    \tightlist
    \item
      \textbf{\texttt{avg\_l\_lrn}}\texttt{\ =\ avg\_l.lrn\_min\ +\ (avg\_l\ -\ avg\_l.min)\ *\ ((avg\_l.lrn\_max\ -\ avg\_l.lrn\_min)\ /\ avg\_l.gain\ -\ avg\_l.min))}

      \begin{itemize}
      \tightlist
      \item
        learning strength factor for how much to learn based on avg\_l
        floating threshold --- this is dynamically modulated by
        strength of avg\_l itself, and this turns out to be critical
        --- the amount of this learning increases as units are more
        consistently active all the time (i.e., ``hog'' units).
        avg\_l.lrn\_min = 0.0001, avg\_l.lrn\_max = 0.5. Note that this
        depends on having a clear max to avg\_l, which is an advantage
        of the exponential running-average form above.
      \end{itemize}
    \item
      \textbf{\texttt{avg\_l\_lrn}}\texttt{\ *=\ MAX(1\ -\ cos\_diff\_avg,\ 0.01)}

      \begin{itemize}
      \tightlist
      \item
        also modulate by time-averaged cosine (normalized dot product)
        between minus and plus phase activation states in given
        receiving layer (cos\_diff\_avg), (time constant 100) --- if
        error signals are small in a given layer, then Hebbian learning
        should also be relatively weak so that it doesn't overpower it
        --- and conversely, layers with higher levels of error signals
        can handle (and benefit from) more Hebbian learning. The
        MAX(0.01) factor ensures that there is a minimum level of .01
        Hebbian (multiplying the previously-computed factor above). The
        .01 * .05 factors give an upper-level value of .0005 to use for
        a fixed constant avg\_l\_lrn value --- just slightly less than
        this (.0004) seems to work best if not using these adaptive
        factors.
      \end{itemize}
    \end{itemize}
\end{itemize}

\begin{itemize}
\tightlist
\item
  \textbf{Learning equation} (see LeabraConSpec.h for code) --- most of
  these are intermediate variables used in computing final dwt value

  \begin{itemize}
  \tightlist
  \item
    \textbf{\texttt{srs}}\texttt{\ =\ ru-\textgreater{}avg\_s\_eff\ *\ su-\textgreater{}avg\_s\_eff}

    \begin{itemize}
    \tightlist
    \item
      short-term sender-receiver co-product --- this is the
      intracellular calcium from NMDA and other channels
    \end{itemize}
  \item
    \textbf{\texttt{srm}}\texttt{\ =\ ru-\textgreater{}avg\_m\ *\ su-\textgreater{}avg\_m}

    \begin{itemize}
    \tightlist
    \item
      medium-term sender-receiver co-product --- this drives dynamic
      threshold for error-driven learning
    \end{itemize}
  \item
    \textbf{\texttt{dwt}}\texttt{\ +=\ lrate\ *\ {[}\ m\_lrn\ *\ XCAL(srs,\ srm)\ +\ ru-\textgreater{}avg\_l\_lrn\ *\ XCAL(srs,\ ru-\textgreater{}avg\_l){]}}

    \begin{itemize}
    \tightlist
    \item
      weight change is sum of two factors: error-driven based on
      medium-term threshold (srm), and BCM Hebbian based on long-term
      threshold of the recv unit (ru-\textgreater{}avg\_l)
    \item
      in earlier versions, the two factors were combined into a single
      threshold value, using normalized weighting factors --- this was
      more elegant, but by separating the two apart, we allow the
      hebbian component to use the full range of the XCAL function (as
      compared to the relatively small avg\_l\_lrn factor applied
      \emph{inside} the threshold computation). By multiplying by
      avg\_l\_lrn outside the XCAL equation, we get the desired contrast
      enhancement property of the XCAL function, where values close to
      the threshold are pushed either higher (above threshold) or lower
      (below threshold) most strongly, and values further away are less
      strongly impacted.
    \item
      m\_lrn is a constant and is typically 1.0 when error-driven
      learning is employed (but can be set to 0 to have a completely
      Hebbian model).
    \item
      XCAL is the ``check mark'' linearized BCM-style learning function
      (see figure) that was derived from the Urakubo Et Al (2008) STDP
      model, as described in more detail in the CCN textbook:
      \url{http://ccnbook.colorado.edu}
    \item
      \texttt{XCAL(x,\ th)\ =\ (x\ \textless{}\ d\_thr)\ ?\ 0\ :\ (x\ \textgreater{}\ th\ *\ d\_rev)\ ?\ (x\ -\ th)\ :\ (-x\ *\ ((1-d\_rev)/d\_rev))}
    \item
      d\_thr = 0.0001, d\_rev = 0.1 defaults
    \item
      x ? y : z terminology is C syntax for: if x is true, then y, else
      z
    \end{itemize}
  \end{itemize}
\end{itemize}

\begin{itemize}
\tightlist
\item
  \textbf{Momentum} --- as of version 8.2.0, momentum is turned on by
  default, and has significant benefits for preventing hog units by
  driving more rapid specialization and convergence on promising error
  gradients.

  \begin{itemize}
  \tightlist
  \item
    \textbf{\texttt{dwavg}}\texttt{\ =\ MAX(dwavg\_dt\_c\ *\ dwavg,\ ABS(dwt))}

    \begin{itemize}
    \tightlist
    \item
      increment the running-average weight change magnitude (dwavg),
      using abs (L1 norm) instead of squaring (L2 norm), and with a
      small amount of decay: dwavg\_dt\_c = 1 - .001 --- software uses
      dwavg\_tau = 1000 as a time-constant of this decay:
      \texttt{dwavg\_dt\_c\ =\ 1\ -\ 1/dwavg\_tau}.
    \end{itemize}
  \item
    \textbf{\texttt{moment}}\texttt{\ =\ m\_dt\_c\ *\ moment\ +\ dwt}

    \begin{itemize}
    \tightlist
    \item
      increment momentum from new weight change ---
      \texttt{m\_dt\_c\ =\ 1\ -\ 1/m\_tau} where m\_tau = 20 trial time
      constant for momentum integration by default, which works best
      (i.e., m\_dt\_c = .95 --- .9 (m\_tau = 10) is a
      traditionally-used momentum value that also works fine but .95
      (m\_tau = 20) works better for most cases.
    \end{itemize}
  \item
    \texttt{if(dwavg\ !=\ 0)\ dwt\ =\ moment\ /\ MAX(dwavg,\ norm\_min);\ else\ dwt\ =\ moment}

    \begin{itemize}
    \tightlist
    \item
      set the weight change used by following weight update equation to
      use momentum, normalized by dwavg if available (nonzero) --- this
      normalization is used in RMSProp, ADAM, and other related
      algorithms.
    \end{itemize}
  \end{itemize}
\end{itemize}

\begin{itemize}
\tightlist
\item
  \textbf{Weight update equation} (see LeabraConSpec.h for code) (see
  below for alternative version using differential fast vs. slow
  weights, not used by default)

  \begin{itemize}
  \tightlist
  \item
    The \textbf{fwt} value here is the linear, non-contrast enhanced
    version of the weight value, while \textbf{wt} is the sigmoidal
    contrast-enhanced version, which is used for sending netinput to
    other neurons. One can compute fwt from wt and vice-versa, but
    numerical errors can accumulate in going back-and forth more than
    necessary, and it is generally faster to just store these two weight
    values (and they are needed for the slow vs. fast weights version
    show below).
  \item
    \texttt{dwt\ *=\ (dwt\ \textgreater{}\ 0)\ ?\ (1-fwt)\ :\ fwt}

    \begin{itemize}
    \tightlist
    \item
      soft weight bounding --- weight increases exponentially
      decelerate toward upper bound of 1, and decreases toward lower
      bound of 0. based on linear, non-contrast enhanced fwt weights.
    \end{itemize}
  \item
    \textbf{\texttt{fwt}}\texttt{\ +=\ dwt}

    \begin{itemize}
    \tightlist
    \item
      increment the linear weights with the bounded dwt term
    \end{itemize}
  \item
    \textbf{\texttt{wt}}\texttt{\ =\ SIG(fwt)}

    \begin{itemize}
    \tightlist
    \item
      new weight value is sigmoidal contrast enhanced version of fast
      weight
    \item
      \texttt{SIG(w)\ =\ 1\ /\ (1\ +\ (off\ *\ (1-w)/w)\^{}gain)}
    \end{itemize}
  \item
    \textbf{\texttt{dwt}}\texttt{\ =\ 0}

    \begin{itemize}
    \tightlist
    \item
      reset weight changes now that they have been applied.
    \end{itemize}
  \end{itemize}
\end{itemize}

\begin{itemize}
\tightlist
\item
   \emph{Optional, on by default:} \textbf{Weight Balance} --- this option attempts to
  maintain more balanced weights across units, to prevent some units
  from hogging the representational space, by changing the rates of
  weight increase and decrease in the soft weight bounding function, as
  a function of the average receiving weights:

  \begin{itemize}
  \tightlist
  \item
    \texttt{dwt\ *=\ (dwt\ \textgreater{}\ 0)\ ?\ wb\_inc\ *\ (1-fwt)\ :\ wb\_dec\ *\ fwt}

    \begin{itemize}
    \tightlist
    \item
      wb\_inc = weight increase modulator, and wb\_dec = weight decrease
      modulator (when these are both 1, this is same as standard, and
      this is the default value of these factors)
    \end{itemize}
  \item
    \texttt{wt\_avg\ =\ }

    \begin{itemize}
    \tightlist
    \item
      average of all the receiving weights --- computed \emph{per
      projection} (corresponding to a dendritic branch perhaps)
    \end{itemize}
  \item
    \texttt{if\ (wt\_avg\ \textgreater{}\ hi\_thr)\ then\ wbi\ =\ gain\ *\ (wt\_avg\ -\ hi\_thr);\ wb\_inc\ =\ 1\ -\ wbi;\ wb\_dec\ =\ 1\ +\ wbi}

    \begin{itemize}
    \tightlist
    \item
      If the average weights are higher than a high threshold (hi\_thr =
      .4 default) then the increase factor wb\_inc is reduced, and the
      decrease factor wb\_dec is increased, by a factor wbi that is
      determined by how far above the threshold the average is. Thus,
      the higher the weights get, the less quickly they can increase,
      and the more quickly they decrease, pushing them back into
      balance.
    \end{itemize}
  \item
    \texttt{if\ (wt\_avg\ \textless{}\ lo\_thr)\ then\ wbd\ =\ gain\ *\ (wt\_avg\ -\ lo\_thr);\ wb\_inc\ =\ 1\ -\ wbd;\ wb\_dec\ =\ 1\ +\ wbd}

    \begin{itemize}
    \tightlist
    \item
      This is the symmetric version for case when weight averages are
      below a low threshold (lo\_thr = .2), and the weight balance
      factors go in the opposite direction (wbd is negative), causing
      weight increases to be favored over decreases.
    \end{itemize}
  \item
    The hi\_thr and lo\_thr parameters are specified in terms of a
    target weight average value \texttt{trg\ =\ .3} with a threshold
    \texttt{thr=.1} around that target value, with these defaults
    producing the default .4 and .2 hi and lo thresholds respectively.
  \item
    A key feature of this mechanism is that it does not change the sign
    of any weight changes, including not causing weights to change that
    are otherwise not changing due to the learning rule. This is not
    true of an alternative mechanism that has been used in various
    models, which normalizes the total weight value by subtracting the
    average. Overall this weight balance mechanism is important for
    larger networks on harder tasks, where the hogging problem can be a
    significant problem.
  \end{itemize}
\end{itemize}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "deep_leabra"
%%% End: