\chapter{Towards OWL-aware similarity} \label{chap:enhancements}

The biomedical informatics community is actively committed to the adoption of formal-logic knowledge representation languages, such as the Web Ontology Language (OWL), and the use of reference ontologies to annotate biomedical resources (\cf \appref{app:ontologies}). This adoption has resulted in the increase of
\begin{paralist}
    \item the amount of knowledge represented in ontologies, and
    \item the quality of these representations in respect to the reality.
\end{paralist}

As has been argued by \citet{Couto2013}, there are some benefits to considering exploiting formal axioms in the calculation of semantic similarity. In this chapter, I report the enhancements that I achieved in the pursuit of semantic similarity measures that can maximise the use of these axioms. I first report on a measure that can use disjointness axioms and then on another that can use existential quantifications.

\section{Disjointness axioms in semantic similarity} \label{sec:enhancements/disjointness}

\begin{note-paper}
    This section is adapted from a paper produced in collaboration with Janna Hastings from the European Bioinformatics Institute, published in Oxford Bioinformatics~\citep{Ferreira2013}.
\end{note-paper}

\subsection{The idea}

Disjointness axioms are one example of formal constructions in OWL ontologies that are gaining momentum in the biomedical informatics community. For example, the Chemical Entities of Biological Interest (\ontology{CHEBI}) ontology now includes this type of axioms~\citep{Hastings2013}. To explore this new type of information, I devised an algorithm that can be \emph{plugged} into some semantic similarity measures to take into account disjointness information.

A disjointness axiom declared for a pair of concepts express the constraint that an instance of one of them cannot also be an instance of the other. This constraint logically implies that the two concepts cannot have common subclasses. Should such shared subclasses be detected by a reasoner, the reasoner will flag the ontology as \emph{inconsistent}~\citep{Sattler2013}, which can be used by ontology developers to prevent errors in ontology development. For example, in \ontology{CHEBI}, this technique has been used to detect that a specific ion (a type of charged molecule) was misclassified as a group (a strict part of a molecule)~\citep{Hastings2012}.

\begin{figure}
    \centering
    \includegraphics[width=\textwidth]{images/shape-ontology.pdf}
    \caption[A snippet of a hypothetical Shape Ontology]{In this snippet, I use the term \term{Parallelogram} as ``a quadrilateral with two pairs of parallel sides'' and \term{Trapezoid} to mean ``a quadrilateral with two parallel sides and two obtuse angles''. Solid lines represent class-subclass relationships, dashed lines represent disjointness axioms. Note that a proper shape ontology would classify \term{Square} as a subclass of \term{Rectangle}, \term{Rhombus} and \term{Parallelogram}, and \term{Rhombus} as equivalent to \term{Parallelogram}. For the sake of the argument being exposed, however, assume that such information is yet unknown by the ontology creators.}
    \label{fig:shape}
\end{figure}

\figref{fig:shape} illustrates this situation: this ontology snippet asserts that no instance of \term{Rectangle} can simultaneously be an instance of \term{Trapezoid}. However, given the open-world assumption that underlies OWL ontologies (see \secref{sec:concepts/ontologies}), there can be instances of \term{Rectangle} that are also instances of \term{Parallelogram} (in fact, it is a consequence of the relevant geometric definitions that squares are both rectangles and parallelograms). For this reason, the similarity between \term{Rectangle} and \term{Parallelogram} should intuitively be higher than the similarity between \term{Rectangle} and \term{Trapezoid}. Using $\sigma$ to represent the function that returns the similarity between two concepts, this hypothesis can be mathematically stated with \eqref{eq:motivation}:
\begin{equation}
    \sigma(\term{Rectangle}, \term{Parallelogram}) >
    \sigma(\term{Rectangle}, \term{Trapezoid}).
    \label{eq:motivation}
\end{equation}

\subsection{The proposed measure} \label{sub:disjointness/measure}

As has been discussed in \secref{sec:sota/node}, there has been an effort to design measures that compute the shared information content between two concepts: while shared information content between concepts $x$ and~$y$ has been assumed to be well estimated by the maximal information content of the concepts that subsume both $x$ and~$y$, \citet{Couto2011} suggest DiShIn, a shared information content measure that builds upon existing measures (such as the ones in \eqref{eq:resnik-ssm,eq:lin}) and which behaves as a \emph{plug-in} to such measures. In its particular case, DiShIn explores \emph{multiple parentage} in order to ensure that all shared information across multiple ancestors is taken into account.

Likewise, instead of developing a new semantic similarity measure to deal with disjointness axioms, I proposed a \emph{plug-in} to be used on top of existing measures of shared information content. My \emph{plug-in} refines the estimation of shared information between two concepts by incorporating the disjointness axioms asserted in the ontology. In here, I denote the disjointness-aware measure of shared information content between concepts $x$ and~$y$ as $\ICsdis(x,y)$, which is calculated based on an existing measure of shared information content,~$\ICs(x,y)$.

Given the example presented in \figref{fig:shape} and the inequality of \eqref{eq:motivation}, it would be desirable for the measure of shared information content to decrease for concepts that are known to be disjoint, formalising the intuition that disjoint concepts are less similar because they cannot share subclasses. Furthermore, to respect the open-world assumption, the measure should stay unchanged when two concepts are \emph{not known} to be disjoint.

With these constraints in mind, I proposed the following measure of shared information content:
\begin{equation}
    \ICsdis(x,y) = \ICs(x,y) - k(x,y)
    \label{eq:new-disj}
\end{equation}
where $\ICs(x,y)$~is any measure of shared information content between $x$ and~$y$, $k(x, y) > 0$ if $x$ and~$y$ are disjoint and $k(x, y) = 0$ otherwise.

\begin{figure}
    \centering
    \includegraphics[width=0.35\textwidth]{images/toy-disjoints.pdf}
    \caption[Example ontology with disjointness axioms]{This illustrates a toy ontology that shows some concepts in a class-subclass hierarchy (represented with solid lines) as well as a disjointness axiom asserted between two concepts (dashed lines).}
    \label{fig:toy}
\end{figure}

Two points were crucial in the development of this measure. First, note that, as is, this equation presents a \emph{discontinuity}. In the hypothetical ontology of \figref{fig:toy}, this measure implies
\begin{equation}
    \ICsdis(\term D, \term E) < \IC(\term B),
\end{equation}
which, depending on the value $k(\term D, \term E)$, could lead to
\begin{equation}
    \ICsdis(\term D, \term E) < \IC(\term A) = \ICsdis(\term D, \term F).
\end{equation}
However, this should not be possible, since \term D and \term E share more information than \term D and~\term F. Therefore, $k$ must be bounded according to the $\IC$ of the most informative ancestor of the $\MICA$, which, in this case, results in
\begin{equation}
    k(\term D, \term E) \leq \IC(\term B) - \IC(\term A).
\end{equation}

\begin{figure}
    \centering
    \includegraphics[width=0.594\textwidth]{images/strength.pdf}
    \caption[The potential for implicit common superclasses between two concepts]{In both cases, $\MICA(\term X, \term Y) = \term M$, and the most informative ancestor of \term M is \term Z. The difference is in the location of the disjointness axiom. In situation A, there is a higher likelihood of implicit common ancestors (ICA) between \term X and~\term Y, because the axiom of disjointness is further down from their common ancestry.}
    \label{fig:strength}
\end{figure}

The second major decision in the development of this measure is related to an operational notion. In fact, I have not yet defined how to compute~$k(x,y)$. I propose an algorithm based on the notion of potential for \emph{implicit common superclasses} (ICS), which measures the likelihood that two concepts share non-asserted superclasses. Consider the ontology snippets in \figref{fig:strength}. In situation~A, given the open-world assumption, there is a small chance that \term Y turns out to be a subclass of~$\term{X'}$, while in situation~B that cannot happen, since \term Y is inferred to be disjoint with~$\term X'$ (the disjointness axiom is higher up in the hierarchy). This suggests that there is a higher potential for ICS between the concepts \term X and~\term Y in situation~A.

I model the \emph{unlikelihood} of ICS as~$f(x, y)$, a function that returns higher values for situations with lower potential for ICS:
\begin{equation}
    f(x, y) =
    \max\left\{
        \frac{1}{p(a,b)} \,\Big|\, a\in\anc(x) \wedge b\in\anc(y) \wedge J(a,b)
    \right\}\cup\{0\}
\label{eq:strength}
\end{equation}
where $\anc(c)$~is the set of superclasses of~$c$ (including~$c$), $J(a, b)$~is true when $a$ and~$b$ are disjoint (either by assertion or inference) and false otherwise, and $p(a, b)$~is the edge length of the shortest path from $a$ to~$b$, taking into account only class-subclass relationships, not the disjointness arcs (only the solid edges in the figures, not the dashed ones).

Using the example ontologies in \figref{fig:strength}, we can illustrate this definition by calculating~$f(\term X, \term Y)$. In A, $J(a,b)$~is true only for $(a,b)=(\term X, \term Y)$; the shortest path from \term X to~\term Y is $\term X \rightarrow \term{X'} \rightarrow \term M \rightarrow \term{Y'} \rightarrow \term Y$, which has length~$4$. Therefore,
\begin{equation}
    f(\term X, \term Y)=\max\left\{\frac14,0\right\}=\frac14.
\end{equation}

In B, $J(a,b)$~is true for $(a,b)\in\{(\term X, \term Y), (\term X, \term{Y'}), (\term{X'}, \term Y), (\term{X'}, \term{Y'})\}$. These correspond to paths of length $4$, $3$, $3$ and~$2$, respectively, leading to
\begin{equation}
    f(\term X, \term Y) = \max\left\{\frac14, \frac13, \frac12, 0\right\}=\frac12.
\end{equation}

Additionally, in situation A, $f(\term{X'}, \term{Y'}) = 0$, since the two concepts are not disjoint and as such $J(a,b)$~is always false.

The general procedure to calculate $\ICsdis(x,y)$ is, therefore:
\begin{enumerate}
    \item Determine $M = \MICA(x,y)$
    \item Determine $Z = \argmax_c \{\IC(c)\,|\,c\in\anc(M)\}$, \ie the most informative ancestor of~$M$;
    \item Calculate $f(x, y)$, as described in \eqref{eq:strength};
    \item Calculate $k(x,y) = f(x,y)\cdot(\IC(M) - \IC(Z))$;
    \item Calculate $\ICsdis(x,y)=\ICs(x,y)-k(x,y)$.
\end{enumerate}

With this procedure, the new shared information content is estimated as a weighted average between $\IC(M)$ and~$\IC(Z)$, where a low potential for ICS leads to a shared information content close to~$\IC(Z)$ and a higher potential for ICS leads to a shared information content close to~$\IC(M)$. This means that the shared information content decreases by a larger amount when there is a smaller potential for implicit common superclasses. Note that if the two concepts are not disjoint, $k(x,y)=0$ and $\ICsdis=\ICs$, which satisfies the open-world assumption mentioned previously.

\subsection{Validation}

According to the hierarchy presented in \chpref{chap:validation}, I classify the validation approach followed in this work as a ``Comparison with an anchor measure''. For this purpose, I calculated structural similarity between \ontology{CHEBI} concepts (details can be found on the paper), and semantic similarity using the algorithm described above, and measured the Pearson's correlation coefficient between the two measures.

The proposed measure was validated in three steps, by measuring the increase in coefficient
\begin{paralist}
    \item in the presence \vs absence of disjointness axioms,
    \item with increasing fractions of the total number of disjointness axioms, and
    \item in several random datasets.
\end{paralist}

For a fully detailed discussion of these three steps, I refer the reader to the published paper. Here I summarise the main results.

\point{Increase in correlation coefficient} I applied the new measure of shared information content to a subset of \ontology{CHEBI}, including some disjointness axioms~\citep{Hastings2012,Hastings2013}. Since it consists of a \emph{plug-in} and relies on a previously defined measure of shared information content, I used, in this assessment, the classical notion of shared information content proposed by~\citet{Resnik1995}:
\begin{equation}
    \ICs(x,y) = \IC(\MICA(x,y))
\end{equation}
where information content was calculated using \eqref{eq:seco}.

Given the sparsity of disjointness axioms relative to the size of \ontology{CHEBI}, random pairs of concepts would rarely touch any disjointness information and it would be difficult to detect the difference resulting from these axioms. As such, the concepts in the dataset were chosen so that a significant part of them had disjointness information. Additionally, given the need to calculate structural similarity, the concepts in the dataset were also selected in order to ensure that structural similarity was possible. While these two constraints slightly bias the generated dataset, I believe that the bias did not result in a dataset too different from reality: in fact, although the amount of disjointness axioms already added to \ontology{CHEBI} is not high, true disjointness exists for most pairs of concepts.

Structural similarity was calculated using PubChem's fingerprint method~\citep{Bolton2008}. Structural similarity between \ontology{CHEBI} concepts that do not represent actual molecules but rather chemical groups (\eg there is not a single chemical structure for \term{Hexose} but rather a collection of them) was achieved by comparing the collection of structures.

For the dataset created above, I compared all compounds with all
the other compounds using three measures: structural similarity, classical $\ICs$ and~$\ICsdis$. I used Wolfe's t-Test~\citep{Wolfe1976,Rosner2010} to determine the statistical significance of the increase in the correlation coefficient between the pair~$(\textrm{structural}, \ICs)$ and the pair~$(\textrm{structural}, \ICsdis)$.

The Pearson's correlation coefficient between the structural measure and~$\ICs$ is~$0.69883$, and after taking the disjointness axioms into account, the correlation between structural similarity and~$\ICsdis$ becomes~$0.71571$. This represents an increase of~$0.01688$. Despite the small absolute increase, this value is statistically significant, with a~\emph{p}-value of~$4.5 \times 10^{-8}$. The small increase of the correlation can be attributed to at least three
factors:
\begin{itemize}
    \item As the annotation of disjointness is still incomplete in \ontology{CHEBI}, we have access to only a small subset of all the \emph{real} disjointness axioms that can be expressed in \ontology{CHEBI}, which means that the shared information content changes only for a fraction of all the concept pairs ($39\%$ of the pairs in the dataset). I expect that, as the number of disjointness axioms added to \ontology{CHEBI} increases, both this fraction and the difference between correlation coefficients will increase.
    \item While highly correlated, structural similarity and semantic similarity measures are inherently different, and as such there is a maximum bound on the actual correlation that can be expected between the two (\cf \secref{sub:hierarchy/comparison}).
    \item Disjointness is only one of the logical axiom types that are used to express concept definitions in an OWL ontology. In fact, \ontology{CHEBI} contains a number of other properties that are also used to capture the meaning of its concepts, \eg the property \prop{has-tautomer}, which connects together closely structurally related chemicals, and \prop{has-role}, which connects a chemical concept to its biological activity.
\end{itemize}

\point{Effect of the number of axioms} The second assessment step measured the effect of the number of disjointness axioms on the correlation coefficient. I partitioned the $199$~disjoitness axioms into $10$~sets: the first contained $20$~random axioms, the second contained these plus another $20$~random axioms, \etc., with the final one containing the $199$~axioms (see \figref{fig:partitions}). For each set, I calculated $\ICsdis$ on the previous dataset and plotted a graph showing the increase in correlation \vs the number of axioms. To remove any bias that resulted from the random method used to partition the axioms, I repeated this process $20$~times.

\begin{figure}
    \centering
    \includegraphics[width=0.6\textwidth]{images/partition.pdf}
    \caption[Partitioning the set of disjointness axioms]{The process used to assess the effect of the number of axioms on the correlation coefficient between structural and semantic similarity. The axioms are randomly partitioned into clusters $p_1$ to~$p_{10}$. Consecutively, each of these clusters is joined with the previous ones to create the sets $s_i = \bigcup_{j=1}^i p_j$, which are then used to compute the increase in correlation coefficient.}
    \label{fig:partitions}
\end{figure}

This validation step has the objective of simulating the development of \ontology{CHEBI} ontology with respect to the number of disjointness axioms. For each of the $20$~repetitions, I studied the difference between the correlation coefficients as the number of disjointness axioms increases, and plotted a graph with this information.

The graphs in \figref{fig:evolution} show the result of some of these repetitions. These graphs illustrate that not all disjointness axioms are important for a given dataset. In fact, only for some of the sets of axioms is the correlation coefficient significantly affected, which suggests that those sets contained the axioms that change the logical meaning behind the concepts in the dataset. The graphs present an obvious trend (see \figref{fig:evolution-trend} for an average of the graphs of all the $20$~repetitions) that indicates an increase of the correlation, which, again, indicates that the disjointness axioms improve the correctness of semantic similarity.

\begin{figure}
    \centering
    \includegraphics[width=0.4\textwidth]{images/plot18.png}
    \includegraphics[width=0.4\textwidth]{images/plot17.png}
    \includegraphics[width=0.4\textwidth]{images/plot3.png}
    \includegraphics[width=0.4\textwidth]{images/plot1.png}
    \includegraphics[width=0.4\textwidth]{images/plot10.png}
    \includegraphics[width=0.4\textwidth]{images/plot8.png}
    \caption[The effect of increasing number of disjointness axioms]{These graphs illustrate the increase in correlation coefficient that results from increasing the number of disjointness axioms. In each graph, the abscissa is the number of axioms used by the semantic similarity measure and the ordinate is the correlation coefficient. The correlation coefficient for $0$~axioms is always equal to the correlation measured with the classical~$\ICs$, which is~$0.69883$; the correlation coefficient for the maximum number of axioms corresponds to the value~$0.71571$, presented in the first validation step. These graphs are representative of the behaviour obtained in all the $20$~repetitions.}
    \label{fig:evolution}
\end{figure}

\begin{figure}[p]
    \centering
    \includegraphics[width=0.6\textwidth]{images/trend.png}
    \caption[The trend corresponding to all the repetitions]{This graph shows the average of all the graphs produced in the $20$~repetitions. Although these values do not have any statistical significance in themselves, they clearly show the trend that the more disjointness axioms are considered, the better is the correlation between structural and semantic similarity.}
    \label{fig:evolution-trend}
\end{figure}

\point{Effect on other datasets:} As the third assessment step, I studied the increase in correlation coefficient on other datasets, since the dataset created for the first step resulted from a random selection process. Following the same selection process, I created $550$~more datasets and compared the correlation coefficient as previously explained.

The graph of \figref{fig:histogram} shows an histogram that represents the difference in the Pearson's correlation coefficient for all these datasets. As is visible in that graph and in \tabref{tab:histogram}, the vast majority of the datasets are associated with an increase in the correlation coefficient. In fact, the effect of considering the disjointness axioms for the semantic similarity only impacts negatively $6.2\%$ of the datasets. We observed a mean correlation increase of~$0.0149$, with a standard deviation for that value of~$0.0130$. Furthermore, in $72.5\%~$of the datasets, the increase in correlation is significant at a confidence value of $0.05$.

\begin{figure}[p]
    \centering
    \includegraphics[width=0.7\textwidth]{images/histogram.png}
    \caption[Distribution of the difference in correlation coefficient for random datasets]{The majority of the cases show a positive difference. I used Wolfe's t-Test to calculate the \emph{p}-value associated with the hypothesis that the increase was due to random chance, and marked with a darker shade the amount of datasets for which p-value${}<0.05$. The vertical line shows the zero of the axis, \ie where the two correlation coefficients are the same.}
    \label{fig:histogram}
\end{figure}

\begin{table}[p]
    \centering
    \small
    \caption[Statistics related to the histogram of \figref{fig:histogram}]{The last column shows the frequency relative to all the datasets created.}
    \begin{tabular}{lrr}
    \toprule
        & \bfseries \# datasets & \bfseries \% datasets \\
    \midrule
    Increase in correlation & 516 &  93.8\% \\
    p-value${}<0.05$        & 399 &  72.5\% \\
    \bottomrule
    \end{tabular}
    \label{tab:histogram}
\end{table}

\subsection{Limitations, future work and other conclusions}

This measure has some limitations:
\begin{enumerate}
    \item The formula of~$\ICsdis$ is not robust against ontology development. Sometimes, the addition of a concept to the ontology during development can considerably change the value of shared information content between two concepts (\cf \secref{sub:hierarchy/theoretical}, where I mention this problem in the context of semantic similarity validation). This issue is mitigated by the fact that the such additions are not common in biomedical ontologies (full details in the paper).
    \item The potential for implicit common superclasses is measured using an edge distance, which is a fragile measure in biomedical ontologies~\citep{Pesquita2009} (see \secref{sec:sota/edge}). It may be possible to explore the semantics of the edges themselves in order to overcome this issue.
    \item The measure of information content influences the results obtained with~$\ICsdis$. In this case, IC was calculated with an intrinsic measure of information content; it would be informative to see the effect of changing the IC measure to an extrinsic one.
\end{enumerate}

This validation shows that considering disjointness axioms improves the shared information content measure, with statistical significance. This new approach is able to successfully explore more than just the class-subclass hierarchy of an ontology, relying on a partial subset of the description logic axioms that are included in the ontology to refine the comparison algorithm. To the best of my knowledge, this represents the first attempt to explicitly use description logic expressivity in semantic similarity in the biomedical domain. I demonstrated this hypothesis using a rather naïve approach. More sophisticated approaches include the exploration of the semantics of edges, other types of information content based on external corpus, \etc.

\section{Semantic relatedness measure} \label{sec:enhancements/relatedness}

\begin{note-paper}
    This section is adapted from a papers accepted for oral presentation at the International Conference on Biomedical Ontology~2011 and published in the corresponding proceedings~\citep{Ferreira2011}.
\end{note-paper}

\subsection{The idea}

According to the definition provided in \secref{sec:concepts/semantic-similarity}, semantic similarity uses exclusively the hypernymy relationship of an ontology: the class-subclass hierarchy. In this sense, \term{Heart} and \term{Blood} are not similar at all. However, in some contexts, hypernymy is not enough to detect that two concepts are related to one another. Anatomy, specifically when used to detect similarity between diseases, is one of these contexts. For example, a heart disease can have implications in blood pressure, and thus a disease annotated with the concept \term{Heart} is somewhat related to one annotated with the concept \term{Blood}.

\begin{figure}[!b]
    \centering
    \includegraphics{images/heart-neighbourhood.pdf}
    \caption[The semantic neighbourhood of the concept \term{Heart}]{This picture illustrates the semantic neighbourhood of \term{Heart}, with a radius~$M=2$. As depicted, \term{Aortic valve} is part of the first layer since it directly links to \term{Heart}, while \term{Cardiac valve} is part of the second layer. Notice that some neighbours can be connected to the centre by more than one distinct path (\eg \term{Left ventricle}). The \emph{radius} can be increased, resulting in a larger neighbourhood. Given the proposed weighting mechanism (see \eqref{eq:weight}), the closer the concepts are to the centre, the higher is their contribution to the semantic neighbourhood of \term{Heart}.}
    \label{fig:heart-neighbourhood}
\end{figure}

I developed a measure that compares two anatomical entities based on their ``semantic neighbourhood'', an idea that was based on the mental processes that take place in the human mind when comparing concepts~\citep{Quillian1968,Collins1975,Tversky1977} (see \secref{sec:sota/art}). The semantic neighbourhood of a concept is a graph, where each node is a concept and an edge between two nodes is drawn if the two concepts are related in the ontology by means of an existential quantification axiom (see \secref{sec:concepts/owl}). Consider \figref{fig:heart-neighbourhood}, which shows a snippet of the semantic neighbourhood of \term{Heart} as defined in the Foundational Model of Anatomy (\ontology{FMA}). In here, we can see that \term{Heart} is related to \term{Aortic valve} by the property \prop{has-part}. Many other concepts are related to \term{Heart}. The neighbourhood is extended by allowing the neighbours of the neighbours to participate in it as well, recursively. I call the maximal distance between the centre concept and its neighbours the ``radius'' of the neighbourhood, represented by~$M$ and measured in number of edges in the graph. Notice, then, that we can compute the neighbourhood up to any particular radius, and that a wider neighbourhood can convey more information about a particular concept than a narrower one. Using this notion, comparing two concepts is a matter of comparing the two neighbourhoods, and this was exactly what I proposed~\citep{Ferreira2011}.

Notice that, again, this corresponds to extending the notion of semantic similarity to more logical constructions, in this case the existential quantification. In formal logic, we say that, for each \term{Heart} there is a relationship of type \prop{part-of} between that \term{Heart} and some \term{Aortic valve}:
\begin{axiom}
    \existential{Heart}{has-part}{Aortic valve}
\end{axiom}
Understanding this notation is not essential to appreciate the contributions presented in this document (details and further exploration of description logic symbols can be found, \eg in works by \citet{Nardi2003} and \citet{Baader2005}). Simply take into account that this means ``Each heart has an aortic valve'' (we will encounter this exact notation again in the next chapters).

\ontology{FMA} is one of the best test cases to assess the behaviour of this measure, as it is rich in existential quantifications. In fact, it contains $67$~distinct properties (\eg \prop{part-of}, \prop{surrounded-by}, \prop{continuous-with}, \etc.), which are used in more than $200{,}000$~axioms.

Finally, the same figure can be used to show the concepts of ``weight''. The path from \term{Heart} to \term{Aortic valve} is smaller than the one from \term{Heart} to \term{Vasculature}, which means that \term{Aortic valve} is somehow more related to \term{Heart} than \term{Vasculature}. I use the notion of ``weight'' to measure this relative relatedness within a semantic neighbourhood.

\subsection{The proposed measure}

Mathematically, the relatedness between two concepts $x$ and~$y$, is defined as:
\begin{equation}
\ferreira(x,y) = \frac
    {\sum_{i\in N_M(x) \cap N_M(y)} \, \weight ix \otimes \weight iy}
    {\sum_{i\in N_M(x) \cup N_M(y)} \, \weight ix \oplus  \weight iy}
    \label{eq:ferreira}
\end{equation}
where $N_M(c)$~is the semantic neighbourhood of a concept~$c$ calculated to a maximal radius~$M$, and $\weight ic$~is a weighting function that gives more relevance to the concepts that are closer to~$c$ and less relevance to concepts further away.

This formula is relatively complex in syntax, and uses non-standard mathematical operations, and as such needs to be explained bit by bit. In essence, it is a ratio between what is common in the semantic neighbourhoods of $x$ and~$y$, and the total amount of concepts in the two neighbourhoods. This idea is not far from what is used in $\sim[UI]$ and~$\sim[GIC]$ (see \eqref{eq:simui,eq:simgic}). Unlike those measures, each concept has now two weights, one for each neighbourhood. To deal with this multiplicity of weights, I used the binary operators of T-norm and T-conorm, represented mathematically by the symbols $\otimes$ and~$\oplus$, respectively, which can be applied to two values between $0$ and~$1$. There are several T-norms and T-conorms that could be applied. Mathematically I am interested in the following properties:
\begin{align*}
    0          & \le i \otimes j \le \min(i, j) \\
    \max(i, j) & \le i \oplus j \le 1.
\end{align*}
I chose $x \otimes y = x y$ and $x \oplus y = x + y - x y$~\citep{Klement2004}.

Consider two semantic neighbourhoods, one for concept~\term A and another for concept \term B. We want the concepts that belong to both neighbourhoods to increase the overall measure in a way that is related to how important these two concepts are in the two neighbourhoods. For a concept~$c$ that belongs to both neighbourhoods, let $w_A=\weight cA$ and~$w_B=\weight cB$ be the weights of this concept with respect to each of the two neighbourhoods. Consider the following fraction:
\begin{equation}
    \frac{w_A \otimes w_B}{w_A \oplus w_B} =
    \frac{w_A \times w_B}{w_A + w_B - w_A \times w_B}.
\end{equation}
If $c$~is highly relevant in both neighbourhoods (\eg $w_A=0.8$ and~$w_B=0.9$), both the numerator and the denominator of this fraction have a high value~($\frac{0.72}{0.98}$) and, as such, we observe an increase in the overall $\ferreira$ (\eqref{eq:ferreira}). If both have a low relevance (\eg $w_A=0.1$ and~$w_B=0.2$), the numerator will be a low value and the denominator will be a medium-range value~($\frac{0.02}{0.28}$), which contribute to a mild decrease in the overall measure. But if the concept has high relevance in one neighbourhood and low relevance in the other (\eg $w_A=0.2$ and~$w_B=0.9$), the numerator will be low and the denominator will be high~($\frac{0.18}{0.92}$), which will contribute to a large decrease in the overall measure.

By default, I propose that the weight of a concept with respect to a neighbourhood is computed based on the path that connects that concept to the centre of the neighbourhood. Let~$p_c$ be a path connecting concept~$c$ to the centre of the neighbourhood. This path is composed of a sequence of properties. For example, in \figref{fig:heart-neighbourhood}, the path from \term{Heart} to \term{Cardiac valve} is ``\prop{has-part}${}\rightarrow{}$\prop{is-a}''. The weight associated to a certain path is the product of the relevance of each of the properties in the path. If more than one path can be traversed from the centre to the concept, then I take the maximum relevance associated with these paths. Formally, let~$r(i)$ be the relevance of the property~$i$: then
\begin{equation}
    \weight cA=\max_{p_c}\prod_{i \in p_c} \mathrm{r}(i).
    \label{eq:weight}
\end{equation}
Finally, the relevance of each property must be predetermined before running this algorithm. I originally proposed using~$0.7$ as the default relevance, on the basis that it produced the best results from a selection of possible default values ($0.6$, $0.7$ and~$0.8$). With the passing of time, I came to realise that we could assign each property a relevance that is based on its own information content. Recall the formula used to calculate the information content of a concept based on the ontology alone, proposed by \citet{Seco2004}~(\eqref{eq:seco}). Reusing this formula here, and taking the frequency of a property to be the number of existential quantification axioms in the ontology that use it, we can also define the information content of properties:
\begin{equation}
    \IC(i) = 1 - \frac{\log f(i)}{\log N_e}
\end{equation}
where $f(i)$~is the number of existential axioms that use property~$i$ and $N_e$~is the number of existential axioms in the ontology.

Furthermore, notice that this measure can both calculate the similarity between concepts and the similarity between annotated entities. In fact, the construction of a semantic neighbourhood can either start on a single concept or on a set of concepts, making this a group-wise relatedness measure.

\subsection{Validation}

The validation strategy followed in this work can be classified as a ``Classification prediction''. I based this validation on the assumption that anatomical entities implicated in the same disease should be more related than a random pair of anatomical entities. I first created a map between diseases and \ontology{FMA} concepts. To do that, I used the Human Phenotype Ontology (\ontology{HPO}), an ontology that represents abnormalities in human anatomy (such as \term{Abnormality of the eye} or \term{Prostate cancer}). On the one hand, this ontology is used by its creators to annotate diseases from several disease databases, and associates $6882$ \ontology{HPO}~concepts with $8013$~diseases, with an average and median of $15$ and~$9$ \ontology{HPO}~concepts for each disease, respectively. On the other hand, the ontology itself provides semi-formal descriptions of its concepts, with references to \ontology{FMA} concepts. For example, \term{Microtia} is described as:
\begin{quote}
    Underdevelopment of the external ear (\ontology{FMA}:52781).
\end{quote}
By leveraging on the annotations mentioned above and these \ontology{FMA} references, it is possible to create a dataset of \ontology{FMA}-annotated diseases (see \figref{fig:disease2fma}). This dataset can then be used to find pairs of anatomical concept that are implicated in the same disease, which correspond to the positive dataset for this validation. The negative dataset was generated randomly.

\begin{figure}
    \centering
    \includegraphics{images/disease2fma.pdf}
    \caption[Workflow to find \ontology{FMA} annotations for some diseases]{The first step is to find the annotations from diseases to \ontology{HPO} concepts, which are provided as \texttt{TSV} files in the \ontology{HPO} website; the second step is to leverage on the cross-references that exist in the textual definitions of the \ontology{HPO} concepts to convert them into \ontology{FMA} concepts. This allows one disease to be associated (\ie annotated) with a set of \ontology{FMA} concepts.}
    \label{fig:disease2fma}
\end{figure}

The positive and negative datasets were then used to perform Receiving Operating Characteristic (ROC) analysis~\citep{Fawcett2006}. This is a common step in classification approaches, summarised as follows:
\begin{enumerate}
    \item Select a threshold~$t$ and create a binary classifier that classifies as positive all the pairs that have $\ferreira>t$ and as negative the other. For each~$t$, we can determine the true positive rate (TPR \mdash fraction of the related concepts correctly classified as related) and false positive rate (FPR \mdash fraction of the unrelated concepts incorrectly classified as related).
    \item The highest threshold results in a TPR of~$0$ and a FPR of~$0$ (all pairs are classified as negative); likewise, the lowest possible threshold results in a TPR of~$1$ and a FPR of~$1$ (all pairs are classified as positive).
    \item Plot the curve defined by the points $(\textrm{FPR}, \textrm{TPR})$ when the threshold varies from the maximum to the minimum. This is known as the ROC curve. The closer the graph approaches the point~$(0, 1)$ (which represents the ideal case where all the positive pairs have a higher relatedness measure than all negative pairs), the better is the measure of relatedness.
    \item \citet{Fawcett2004} proposes a way to repeat this experiment a number of times and to produce an average ROC curve from them (see Algorithm~5 of that paper), which I followed by repeating these steps $10$~times, each time with a different randomly generated negative dataset.
\end{enumerate}

The ROC curves obtained with this method are presented in \figref{fig:roc}. For comparison purposes, I applied the proposed $\ferreira$~measure, calculated for $M=3$ and~$M=4$, and I also applied~$\sim[GIC]$ to the dataset to study how the behaviour of a similarity measure contrasts with the behaviour of a relatedness measure.

\begin{figure}
    \centering
    \includegraphics[width=0.8\textwidth]{images/roc.pdf}
    \caption[ROC analysis of the results of~$\ferreira$]{These ROC curves are the average over $10$~runs, each one produced with a different, randomly selected, negative set (\ie randomly selected pairs of \ontology{FMA} concepts).}
    \label{fig:roc}
\end{figure}

As is evident from this figure, $\ferreira$ shows a better performance than $\sim[GIC]$, since high values of TPR are obtained for low values of FPR. The main difference between $\ferreira$ with $M=4$ and~$M=3$ is that the former has more resolution power in that it can differentiate between concepts $8$~properties apart, whereas in the latter concepts with a path distance greater than~$6$ have automatically a relatedness value of~$0.0$. Thus, the measure calculated for many of the negative pairs in the dataset and also some of the positive ones ended up being~$0$. Given the lower resolution of the measure for~$M=3$, more pairs in this setting have relatedness~$0.0$, resulting in the straight diagonal line that we see in the figure.

Additionally, $\sim[GIC]$ does not perform as well as the relatedness measure. The distinct form of the graph occurs because a majority of the positive pairs (\ontology{FMA} concepts implicated in the same disease) are not really similar but simply related, just as in the example above: \term{Heart} and \term{Aortic valve} are relatively frequently implicated in the same disease, and their relatedness is high, as one is part of the other; on the other hand similarity measures are unable to capture this, since one is an organ and the other a valve, two distinct concepts.

This demonstrates the superiority of semantic relatedness measures over semantic similarity, at least when applied to ontologies where there is a vast number of properties, such as \ontology{FMA}, an in contexts where relatedness is more important than mere similarity.

\subsection{Conclusions and future work}

The most important conclusion to take from this work is that measures of relatedness can, in some cases, be more accurate than state-of-the-art measures of similarity, suggesting that relatedness measures do indeed play a role in the biomedical domain, especially when the expressiveness of the relevant ontologies (as measured by the number of properties) is as high as in this case.

The measure that I propose here is based on the concepts of \emph{semantic neighbourhood} and \emph{relevance factors}, and can accommodate the needs of particular applications by fine tuning its parameters. For example, by giving different weights to different properties, the measure can give more importance to some neighbours than others. Machine-learning algorithms can be used to tune the weight of each property according to the needs of each application.

The concept of relevant neighbourhood introduced in this work is also a bridge to other methodologies, particularly in allowing the use of ontology mappings to define wider neighbourhoods that draw not only from a specific ontology but from related ontologies as well, as long as a mapping of some sort exists between them. For example, cross-references can be used for this effect. In this context, the measure can incorporate external knowledge, but is not required to do so. For example, the semantic neighbourhood of a disease can include its symptoms and known treatments; to find the neighbourhood of enzymes, it is possible to include the chemical compounds that they transform; \etc.

Finally, the analysis preformed here on \ontology{FMA} shows that this is a valid method to measure relatedness between biomedical concepts. As we will see later, I have successfully applied this measure to other ontologies and other datasets (see \chpref{chap:multidomain}).

\section{Summary} \label{sec:enhancements/summary}

This chapter delineates my efforts in incorporating OWL axioms other than hypernymy into semantic measures. The first section deals with disjointness axioms while the second with existential quantification axioms. While this is by no means a comprehensive approach to using OWL formalism in semantic similarity, it paves the way for future experimentation and research in this context. As argued by \citet{Couto2013}, increasing the amount of such constructions used in semantic similarity calculations will eventually improve the overall panorama in this field of research and, therefore, the utility of this technique. As such, the results presented here are one of my major contributions to the area of semantic similarity and relatedness.

Even though these results do not directly support the idea of multiple-ontology semantic similarity, they were invaluable to the whole corpus of research that I was committed to achieve. In fact, they represent real scientific advances. Additionally, although $\ferreira$ was not originally validated in a multi-domain context, it can be easily converted into a measure that is able to tackle that problem, as we will see in \chpref{chap:multidomain}, by allowing the semantic neighbourhood of concepts to be generated based on several ontologies.


% -*-LaTeX-*-

% $Log: fb-filters.tex,v $
% Revision 1.7  2007/12/25 17:47:12  stiber
% Some final cosmetic changes.
%
% Revision 1.6  2007/12/11 17:49:03  stiber
% Small changes to get ready for Witner 2008.
%
% Revision 1.5  2007/03/20 23:53:48  stiber
% Updated eqnarray to align.
%
% Revision 1.4  2007/03/20 18:14:34  stiber
% Modified for use as part of standalone text.
%
% Revision 1.3  2006/03/27 23:37:41  stiber
% Fixed errors in formulas; deleted end-of-chapter problem that lacked
% prerequisite material upon switching of textbook.
%
% Revision 1.2  2004/03/29 19:54:44  stiber
% Updated for Spring 2004 and new textbook (DSP First).
%
% Revision 1.1  2004/02/19 00:24:45  stiber
% Initial revision
%

\chapter{Feedback Filters}
\label{ch:fb-filters}

\section{Introduction}

In this chapter, we will continue to introduce filters, this chapter
focusing on \emph{feedback filters}, in which previous outputs are
combined with new inputs to produce new outputs. We will learn about
their structure, function, and the differences between feedforward and
feedback filters.  We will see that sometimes it is better to combine
these two kinds of filters. We will also briefly examine digital
filter design. After this chapter, you should understand important
concepts like the \emph{poles} (as compared to \emph{zeros}) of a
transfer function, \emph{impulse response}, and \emph{bandwidth}. You
should know about features of feedback filters and special types of
such filters, such as \emph{resons}.  You should be able to design
simple digital filters, implement them on a computer and use them to
solve some simple signal processing problems.

\subsection{Poles}

\begin{figure}
\centerline{\includegraphics[width=0.5\textwidth]{ch-iir/fb-bdiag}}
\caption{Block diagram of a simple filter with both feedforward and
  feedback terms.\label{fig:fb-bdiag}}
\end{figure}

Figure~\ref{fig:fb-bdiag} presents the block diagram of a filter with
one feedforward and one feedback term; the feedback filter's signal
flowgraph is shown on the right. Compared to the feedforward one on
the left in the figure, we notice that instead of combining the input
signal with a delayed version, here the output signal is delayed and
``fed back'' to be combined with the input. The feedback processing
alone is expressed by
\begin{equation}
y[n] = x[n] + a_1 y[n-1]
\label{eq:fb-1p}
\end{equation}
(ignoring the factor of $b_0$) which is the equation for a simple
feedback filter with one delayed component. $a_1 y[n-1]$ is the
feedback term. You can see that the output at $n-1$ is used to compute
the output at time $n$. The combined filter's full defining equation
is
\begin{equation}
  y[n] = b_0 x[n] + b_1 x[n-1] + a_1 y[n-1]
  \label{eq:fb-ff}
\end{equation}
but for the moment we will concentrate on just the feedback part of
the filter, from equation~(\ref{eq:fb-1p}).

From equation~(\ref{eq:fb-1p}), the z-transforms of the input and
output signals, and the delay operator $z^{-1}$, we can get the
filter's transfer function as follows:
\begin{align}
Y(z) &= X(z) + a_1z^{-1}Y(z) \notag\\
Y(z)[1-a_1z^{-1}] &= X(z) \notag\\
Y(z) &= \frac{1}{1-a_1z^{-1}}X(z)
\end{align}
Finally we have the transfer function:
\begin{align}
H(z) &= \frac{Y(z)}{X(z)} = \frac{1}{1-a_1z^{-1}} \notag \\
     &= \frac{z}{z-a_1} \label{eq:fb-xfer}
\end{align}
The magnitude response is:
\begin{equation}
|\mathcal{H}(\hat{\omega})|
  = |H(z)|
  = \left|\frac{z}{z-a_1}\right|=\left|\frac{1}{z-a_1}\right|
\label{eq:fb-1pmh}
\end{equation}
($|z|=1$ because we are only concerned with the magnitude when $z$ is
on the unit circle, and so its magnitude is always one).

\index{filter!poles|emph} The values of $z$ that make the denominator
of the transfer function zero (the roots of the denominator
polynomial, where the transfer function becomes infinite) are called
its \emph{poles}. In~(\ref{eq:fb-1pmh}), there is one pole at
$z=a_1$. In general, $a_1$ is not on the unit circle, and so the
phasor $z$ approaches it but is never equal to it.  Just as we did for
zeros, we can draw a line from the pole to the unit circle to indicate
the distance between $z$ and the pole.  However, now this distance is
in the denominator. Therefore, instead of $H(z)$ having a notch or dip
when $z$ nears a zero, it has a \emph{peak} when $z$ nears a pole.

\begin{figure}
\centerline{\includegraphics[height=3.5in]{ch-iir/fbexp_1p_c90}}
\caption{One pole in the $z$-plane, located at $r=0.9$ and
$\hat{\omega}_0=\pi/2$.\label{fig:fb-exp1pc90}}
\end{figure}

\begin{figure}
\centerline{\includegraphics[height=3in]{ch-iir/fbexp_1p_h90_r1_r0-9}}
\caption[Magnitude response of two filters]{Magnitude response of two
  filters with poles located at $\hat{\omega}_0=\pi/2$ and $r=0.9$
  (bottom) or $r=1$ (top).\label{fig:fb-exp1ph90r2}}
\end{figure}

\paragraph*{Example~1}
For the one pole filter~(\ref{eq:fb-1p}),
$a_1=re^{j\hat{\omega}_0}$. Figure~\ref{fig:fb-exp1pc90} shows this
pole when $r=0.9$ and
$\hat{\omega}_0=\pi/2$. Figure~\ref{fig:fb-exp1ph90r2} shows its
magnitude response. As expected, $|\mathcal{H}(\hat{\omega})|$ has a
``hill'' near $\hat{\omega}_0=\pi/2$. As $r$ moves closer to one, the
hill becomes steeper.

\subsection{Example: Computing Transfer Function and Impulse Response}

Compute the transfer function and impulse response of the system
described by the feedback filter,
\begin{equation}
y[n] = 2x[n] + \frac{1}{2}y[n-1]
\end{equation}

By computing the z-transform of the both side of above equation and
using the fact that
\begin{equation}
y[n-1]\stackrel{\mathbf{Z}}{\longleftrightarrow} z^{-1}Y(z)
\end{equation}
we obtain,
\begin{equation}
Y(z)= 2X(z) + \frac{1}{2}z^{-1}Y(z).
\end{equation}
Hence the transfer function is
\begin{equation}
H(z)= \frac{Y(z)}{X(z)} = \frac{2}{1-\frac{1}{2}z^{-1}}
    = \frac{2z}{z-\frac{1}{2}} \label{eq:fb-exxfer}
\end{equation}

$H(z)$ has a pole at $z=1/2$ and zero at $z=0$. Now we
compute its impulse response. Previously we had an example of an
exponential signal~(\ref{eq:zt-expof}) and its
z-transform~(\ref{eq:zt-expoF}), so we have
\begin{equation}
a^n\stackrel{\mathbf{Z}}{\longleftrightarrow}
\frac{1}{1-az^{-1}}, \quad n \ge 0, |z|>a
\end{equation}
This is of the same form as~(\ref{eq:fb-exxfer}), with $a=1/2$.  Using
this result, we obtain the inverse transform
\begin{equation}
h[n] = 2\left(\frac{1}{2}\right)^n, \quad n\ge 0 \label{eq:fb-eximp}
\end{equation}
where the left and right sides of equation~(\ref{eq:fb-eximp}) are the
inverse z-transforms of $H(z)$ and $2/(1-\frac{1}{2}z^{-1})$
from equation~(\ref{eq:fb-exxfer}), respectively (the latter making
use of our example from section~\ref{sc:zx-exp}).  This is the
corresponding impulse response.

\subsection{Stability}

Let's examine equation~(\ref{eq:fb-1p}) again. Note that $y[n]$
doesn't only depend on $x[n]$. This means that, even if the input
signal is zero after some initial value, this feedback term can
continue to have a nonzero value and so the filter can still produce
some output. This is a major difference from feedforward filters,
where the output depends only on the input. As an extreme example,
consider the case where the input signal is a \emph{unit impulse},
\begin{equation}
x[n] = \delta[n] =
\left\{\begin{array}{rl}
1 & n=0\\
0  & n>0
\end{array}\right.
\end{equation}
which means it only has a value of one at the beginning, and then
turns off.  The output of any filter for a unit impulse is called,
logically enough, its \emph{impulse response}.  For the filter
in~(\ref{eq:fb-1p}) with coefficient $a_1=0.5$, the impulse response
is:
\begin{align}
y[0] &= x[0] = 1 \notag\\
y[1] &= x[1] + 0.5 y[0] = 0+0.5 \times 1 = 0.5 \notag\\
y[2] &= x[2] + 0.5 y[1] = 0+0.5 \times 0.5=0.25=0.5^2 \notag\\
y[3] &= x[3] + 0.5 y[2] = 0+0.5 \times 0.25=0.125=0.5^3 \notag\\
& \vdots  \notag \\
y[n] &= 0.5^n \label{eq:fb-1pexp}\\
& \vdots  \notag
\end{align}
You can see that the output $y[n]$ continues after $n=0$, but the
value becomes smaller each time, and will decay towards zero.  On the
other hand, if $a_1>1$, the output becomes bigger and bigger, and goes
to infinity. This second behavior is called \emph{unstable}.  From
equation~(\ref{eq:fb-1pmh}), we know that $a_1$ is a pole of the
filter.  In fact, a feedback filter with multiple poles $p_i$ is
stable if and only if
\begin{equation}
|p_i| < 1,  \text{ for all } i
\end{equation}
In other words, \emph{all} the poles must be inside the unit
circle. Here is the basic idea for a four-step (well,
three-step-and-a-note) proof of this:

\begin{enumerate}
\item \textbf{A feedback filter with $N$ feedback terms can be decomposed into
$N$ one-pole feedback filters.}

\begin{figure}
\centerline{\includegraphics[width=0.5\textwidth]{ch-iir/fb-nterm-bdiag}}
\caption{Block diagram of a filter with $M$ feedforward and $N$
  feedback terms.\label{fig:fb-nterm-bdiag}}
\end{figure}

The equation for a feedback filter with $N$ feedback terms (the right
hand side of the general feedforward/feedback filter block diagram in
figure~\ref{fig:fb-nterm-bdiag}) can be written as
\begin{equation}
y[n] = x[n] - \sum_{k=1}^N a_k y[n-k]
\end{equation}
Its transfer function is
\begin{align}
H(z) &= \frac{1}{1 + \sum_{k=1}^N a_k z^{-k}} \notag \\
     &= \frac{z^N}{z^N+\sum_{k=1}^N a_k z^{N-k}} \label{eq:manypoles}
\end{align}
where we've multiplied by $z^N/z^N$, as is our usual practice, to
produce a ratio of polynomials with positive exponents.

Let $p_i, i=1, 2,...,N$ be its poles (the roots of the denominator
polynomial). Let's not worry about how we can get them (we know how if
we can factor the denominator polynomial, so let's assume we have
already done that).  So,~(\ref{eq:manypoles}) can be rewritten in the
form
\begin{align}
H(z) &= \frac{z^N}{(z-p_1)(z-p_2)\hdots (z-p_N)}\\
     &= \frac{z^N}{\prod_{k=1}^N(z-p_k)}\label{eq:mp-denomprod}
\end{align}
Recalling our pre-calculus, a fraction with a product of terms as
in~(\ref{eq:mp-denomprod}) can be rewritten using a partial fraction
expansion as \index{partial fraction expansion}
\begin{align}
H(z) &= \frac{A_1}{(1-p_1z^{-1})}+\frac{A_2}{(1-p_2z^{-2})}+\hdots +
\frac{A_N}{(1-p_Nz^{-N})}\\
&= \sum_{k=1}^N\frac{A_k}{(1-p_k z^{-k})}
\end{align}
Each of the terms
\begin{equation}
\frac{A_k}{(1-p_k z^{-k})}
\end{equation}
is a one-pole feedback filter. So, the complete $N$-pole filter's
output is the sum of $N$ one-pole filters' outputs. This is called a
parallel one-pole filter.
\index{filter!parallel one-pole}

\item \textbf{A one-pole feedback filter is stable if and only if its
impulse response is stable.}

Consider the unit impulse $\delta[n]$,
\begin{equation}
\delta[n] =
\left\{\begin{array}{rl}
1 & n=0\\
0 & n>0
\end{array}\right.
\end{equation}
Any discrete input signal can be thought of as a weighted sum of
delayed unit impulses. This is a straightforward application of the
knowledge that $\delta[n]=1$ when $n=0$,  $\delta[n-1]=1$  when $n=1$,
$\delta[n-2]=1$ when $n=2$, etc. Each of these is zero for all other
time indices, so we can use a sum of impulses, each multiplied by the
corresponding element of $x[n]$, to express a discrete signal:
\begin{align}
x[n] &= \{x[0], x[1], x[2], \ldots\} \notag \\
     &= x[0]\delta[n] + x[1]\delta[n-1] + x[2]\delta[n-2] + \cdots
\end{align}
We can see that the filter's response to any signal will be stable if
and only if its response to each of these impulses is stable.

\item \textbf{An impulse response is stable if and only if its pole's
$|p|<1$.}

Equation~(\ref{eq:fb-1pexp}) tell us that the \spscript{$i$}{th}
one-pole feedback filter's impulse response is the signal with samples
\begin{equation}
1, p_i, p_i^2, p_i^3, \ldots
\end{equation}
This impulse response is stable only when the pole's $|p_i|<1$, so
that the sequence is decreasing in magnitude.

\item \textbf{An impulse response is neither stable nor unstable if
$|p|=1$.}
\end{enumerate}

\subsection{Resonance and Bandwidth}

\index{filter!resonance}
\emph{Resonance} is the increase in a filter's magnitude response in
the region near a pole. An example is shown in
figure~\ref{fig:fb-exp1ph90r2}. The pole is at frequency
$\hat{\omega}=\pi/2$, which is the peak of the magnitude response. The
magnitude is small when away from the pole.

One other thing mentioned before is that the peak becomes steep
when $r$ nears one. The steepness is measured by the filter's
\emph{bandwidth}.
\index{bandwidth}
Bandwidth, $B$, is defined as the width of the
filter's response (i.e., the range of frequencies) at half its maximum
power output. Bandwidth is an important measure of a filter's
performance; it indicates which frequencies are passed and which are
filtered out.

The filter's power is the square of its amplitude, so the bandwidth
can also be measured at $1/\sqrt{2}$ of its peak amplitude value.
These half power levels are denoted by
$|\mathcal{H}(\hat{\omega})|_B^2$ (the $1/\sqrt{2}$ amplitude points
written as $|\mathcal{H}(\hat{\omega})|_B$). If the peak value of
power is $|\mathcal{H}(\hat{\omega})|_p^2$ and the peak amplitude is
$|\mathcal{H}(\hat{\omega})|_p$, the bandwidth points are located at
the (two) values of $\hat{\omega}$ that satisfy the following
relationship:
\begin{equation}
\underbrace{|\mathcal{H}(\hat{\omega})|_B}_{\stackrel{\text{cutoff}}{_\text{amplitude}}}
  = \frac{1}{2}\underbrace{|\mathcal{H}(\hat{\omega})|_p^2}_{\stackrel{\text{peak}}{_\text{power}}}
  = \frac{1}{\sqrt{2}}\underbrace{|\mathcal{H}(\hat{\omega})|_p}_{\stackrel{\text{peak}}{_\text{amplitude}}}
\label{eq:fb-band}
\end{equation}

Since $20\log_{10}(1/\sqrt{2})=-3\mathrm{dB}$
\begin{equation}
\frac{|\mathcal{H}(\hat{\omega})|_B}{|\mathcal{H}(\hat{\omega})|_p}
  = -3\mathrm{dB}
\end{equation}
and so the \emph{cutoff amplitude} for a filter are those for which
its output is reduced $3\mathrm{dB}$ from its peak.
\index{filter!cutoff amplitude}
\index{filter!cutoff frequency}
The values of $\hat{\omega}$ at which the filter's output is reduced
by this much define the edges of its \emph{passband} and are called
its \emph{cutoff frequencies}; they are sometimes called
its ``minus three deebee points''.
\index{filter!passband}
We will call these frequencies $\hat{\omega}_B$ in the simple case
where the passband is symmetrical and the two cutoff frequencies are
the same amount above and below the frequency at peak filter output,
$\hat{\omega}_p$.

\begin{figure}
\centerline{\includegraphics[height=3.5in]{ch-iir/fbexp_1p_c0_phi}}
\caption{One pole $a_1=re^{j\hat{\omega}_0}$ in z-plane. $r=R$,
$\hat{\omega}_0=0$ and $z=e^{j\hat{\omega}}$
\label{fig:fb-exp1pc0_phi}}
\end{figure}

Let's make matters simple and take a look at the case in which a pole
is on the real axis and no other poles are nearby; the pole is
$a_1=re^{j\hat{\omega}_0}$, $r=R$ and $\hat{\omega}_0=0$ (so
$a_1=R$). A point on the unit circle is $z=e^{j\hat{\omega}}$. This
situation is illustrated in figure~\ref{fig:fb-exp1pc0_phi}.  We know
that the one pole transfer function is
\begin{equation}
H(z)=\frac{1}{1-a_1z^{-1}}
\end{equation}
with magnitude
\begin{equation}
|H(z)|=\left|\frac{1}{1-a_1z^{-1}}\right| \label{eq:reson-mag}
\end{equation}

Consider the inverse square of the magnitude, substituting in our
expressions for $a_1$ and $z$,
\begin{align}
\frac{1}{|\mathcal{H}(\hat{\omega})|^2}
    &= |1-R e^{j\hat{\omega}}|^2 \notag\\
    &= |1 - R\cos\hat{\omega} - jR\sin\hat{\omega}|^2
    &&\text{(using Euler's formula)}
    \notag\\
    &= (1 - R\cos\hat{\omega})^2 + R^2\sin^2\hat{\omega}
    &&\left(|H|^2 = \Real[\mathcal{H}]^2 + \Imag[\mathcal{H}]^2\right)
    \notag\\
    &= 1 - 2R\cos\hat{\omega} +R^2\cos^2\hat{\omega} +
       R^2\sin^2\hat{\omega} \notag\\
    &= 1-2R\cos\hat{\omega} + R^2
    &&\left(\cos^2\theta + \sin^2\theta = 1\right)
    \label{eq:reson-inv-mag2}
\end{align}
The peak of $|\mathcal{H}(\hat{\omega})|^2$ should be the value of
angle $\hat{\omega}_p=\hat{\omega}$ that makes
$1/|\mathcal{H}(\hat{\omega})|^2=1-2R\cos\hat{\omega} + R^2$ minimum
(and so $|\mathcal{H}(\hat{\omega})|^2$ maximum),
that is when $\hat{\omega}=0$. The power at this peak is determined by
substituting $\hat{\omega}=0$ into equation~(\ref{eq:reson-inv-mag2})
and taking its reciprocal:
\begin{equation}
|\mathcal{H}(\hat{\omega})|^2_p=\frac{1}{(1-R)^2}
\end{equation}
According to equation (\ref{eq:fb-band}),
\begin{equation}
|\mathcal{H}(\hat{\omega})|_B
  =\frac{1}{2}|\mathcal{H}(\hat{\omega})|^2_p=\frac{1}{2(1-R)^2}
\end{equation}
The corresponding $\hat{\omega}_B$ can be obtained by substituting
$|\mathcal{H}(\hat{\omega})|_B$ for $|\mathcal{H}(\hat{\omega})|$ in
equation~(\ref{eq:reson-inv-mag2}) and solving for frequency:
\begin{align}
2(1-R)^2 &= 1-2R\cos\hat{\omega_B}+R^2 \label{eq:fb-band2a}\\
\cos\hat{\omega}_B &= 2-\frac{1}{2}\left(R+\frac{1}{R}\right)
\label{eq:fb-band2b}
\end{align}

The filter's bandwidth is the span $[-\hat{\omega}_B, \hat{\omega}_B]$
--- a distance of $2\hat{\omega}_B$.  When $R$ is close to one, we can
express $R$ as a small amount $\epsilon$ less than one:
$R=1-\epsilon$. We can then take advantage of the expansions:
\begin{equation}
\frac{1}{R}=\frac{1}{1-\epsilon}=1+\epsilon+\epsilon^2+\epsilon^3+\cdots
\end{equation}
and
\begin{equation}
\cos\epsilon=1-\frac{\epsilon^2}{2!}+\frac{\epsilon^4}{4!}+\cdots
\end{equation}
So, from~(\ref{eq:fb-band2b}),
\begin{align}
\cos\hat{\omega}_B &= 2-\frac{1}{2}[\underbrace{(1-\epsilon)}_{R} +
            \underbrace{(1+\epsilon+\epsilon^2+\epsilon^3+\cdots)}_{\frac{1}{R}}]
              \notag\\
   &= 1-\frac{\epsilon^2}{2}-O(\epsilon^3) \notag\\
   &\approx \cos\epsilon
\end{align}
(where $O(\epsilon^3)$ is shorthand for terms in the expansion of
order $\epsilon^3$ or higher).  Therefore,
$\hat{\omega}_B\approx\epsilon$.  So, when $R$ is close to the unit
circle,
\begin{equation}
B = 2\hat{\omega}_B \approx 2\epsilon =2(1-R)
\end{equation}
or
\begin{equation}
R \approx 1-B/2
\label{eq:calc-bw}
\end{equation}
If we want a low-pass filter with a particular bandwidth,
equation~(\ref{eq:calc-bw}) gives us a way to determine its pole
location.

\begin{figure}
\centerline{\includegraphics[height=3.5in]{ch-iir/design-example}}
\caption[Feedback filter design]{Designing a simple one-pole feedback
  filter. A filter's frequency
  response is sketched for the situation where we want the center of
  its passband at $\hat{\omega}_p$ and we wish to ensure that another
  frequency, $\hat{\omega}_2 = \hat{\omega}_p + \hat{\omega}_d$, is
  well outside the passband. We choose
  the filter's cutoff frequencies, $\hat{\omega}_{B_1} =
  \hat{\omega}_p - \hat{\omega}_B$ and $\hat{\omega}_{B_2} =
  \hat{\omega}_p  + \hat{\omega}_B$, to achieve this.
  \label{fig:design-example}}
\end{figure}

Let's consider how we might use these observations to design a filter,
as sketched in figure~\ref{fig:design-example}. We want the peak of the
filter's frequency response (the center of its passband) to be at the
frequency $\hat{\omega}_p$ and we want to ensure that it filters out
some other frequency, $\hat{\omega}_2$. We choose the -3dB points to
be at a distance $\pm\hat{\omega}_B$ from $\hat{\omega}_p$, labelled
$\hat{\omega}_{B_1}$ and $\hat{\omega}_{B_2}$ in the figure. These
cutoff frequencies are chosen to ensure that $\hat{\omega}_2$ is well
outside the passband.

Our design problem is to choose the pole location. The peak frequency
for the passband, $\hat{\omega}_p$, gives us the pole angle, so all we
need to do is determine the pole radius, $R$, to give us
$a_1=Re^{j\hat{\omega}_p}$. Looking at equation~(\ref{eq:reson-mag}),
we can substitute this value of $a_1$ and $z=e^{j\hat{\omega}}$ and
collect terms:
\begin{align}
|\mathcal{H}(\hat{\omega})| &=
\left|\frac{1}{1-Re^{j\hat{\omega}_p}e^{-j\hat{\omega}}}\right| \notag\\
  &= \left|\frac{1}{1-Re^{j(\hat{\omega}_p - \hat{\omega})}}\right|
\end{align}
We just need to make sure that when the frequency $\hat{\omega} =
\hat{\omega}_2$, $|\mathcal{H}(\hat{\omega})|$ is below -3dB. Clearly,
we can achieve this by ensuring that the half bandwidth, $B$, is less
than $\hat{\omega}_d$. We can then use equation~(\ref{eq:calc-bw}) to
determine the value of $R$ that ensures this, and we have the correct
pole location.

\problemset{
\subsubsection{Self-Test Exercises}

See~\ref{sc:ch4ex} \#\ref{it:ch4ex1}--\ref{it:ch4ex2} for answers.

\begin{enumerate}
\item Derive equation~(\ref{eq:fb-band2b})
  from~(\ref{eq:fb-band2a}).
\item In the situation where the sampling rate is 44,100Hz and the
  desired bandwidth is 20Hz, $R$ in~(\ref{eq:calc-bw}) is 0.998575.
  Solve for $R$ the situation where the desired bandwidth is 200Hz. Is
  it true that when $R$ is far away from one, $B$ grows large?
\end{enumerate}}

\section{Mixing Feedback and Feedforward Filters}

We have now seen feedforward filters with zeros in the transfer
function and feedback filters with poles. Zeros suppress frequency
components and poles enhance them.  Quite often, we want to combine
poles and zeros to improve the filter's features, such as the flatness
of the passband and the abruptness with which its response transitions
between the passband and the stop band.  Generally, the transfer
function of a filter with block diagram in
figure~\ref{fig:fb-nterm-bdiag} can written as
\begin{equation}
H(z)=\frac{b_0+b_1z^{-1}+\cdots+b_Mz^{-M}}{1+a_1z^{-1}+\cdots+a_Nz^{-N}}
\label{eq:gen-xfer}
\end{equation}
and the filter is
\begin{equation}
y[n] = \underbrace{b_0x[n] + b_1x[n-1] + \cdots +
      b_Mx[n-M]}_{\text{feedforward terms}}
      -
      \underbrace{a_1y[n-1] - \cdots - a_ny[n-N]}_{\text{feedback terms}}
\label{eq:gen-impl}
\end{equation}

\afterpage{
\begin{figure}
\centerline{\begin{tabular}{cc}
\noindent\parbox[c]{0.45\textwidth}{\includegraphics[width=0.45\textwidth]{ch-iir/Elliptic-Filter-Response}}
  &
\noindent\parbox[c]{0.45\textwidth}{\includegraphics[width=0.45\textwidth]{ch-iir/Butterworth-Filter-Response}}
\end{tabular}}
\caption[Example filter frequency responses]{Example filter frequency
  responses: elliptic (left)\protect\footnotemark and Butterworth
  (right)\protect\footnotemark.\label{fig:filter-fr-examples}}
\end{figure}
\addtocounter{footnote}{-1}
\footnotetext{By Inductiveload (Own work) [Public domain], via
    Wikimedia Commons.}
\footnotetext{By Alejo2083 (Own work) [GFDL
  (\url{http://www.gnu.org/copyleft/fdl.html)}, CC-BY-SA-3.0
  (\url{http://creativecommons.org/licenses/by-sa/3.0/}) or CC BY-SA
  2.5-2.0-1.0
  (\url{http://creativecommons.org/licenses/by-sa/2.5-2.0-1.0})], via
  Wikimedia Commons.}
}

Depending on the coefficients $\{b_k\}$ and $\{a_\ell\}$, the
filter will show different features. Some of these features have
proven so useful that these forms of~(\ref{eq:gen-xfer}) have acquired
special names, such as elliptic, Butterworth, etc. (usually, based on
the form of the numerator and/or denominator
polynomial). Figure~\ref{fig:filter-fr-examples} presents some example
frequency responses.

\section{Implementation}

Just as in chapter~\ref{ch:filt-intro}, feedback filters have their
delays implemented with queues which are ``circular arrays''.
However, there are two additional complications that we must deal
with: complex poles and accuracy of numerical computation.

\subsection{Avoiding Complex Numbers}

In principle, there is no reason that we couldn't perform all our
computations using complex numbers and just output the real part of
the result as the filtered signal.  However, we often want to perform
filtering in real time, and so would like to avoid any unnecessary
computation. A cute trick allows us to eliminate the need for complex
numbers.

\emph{Remember that all complex poles will appear in conjugate pairs.}
Since each pole is a root of a polynomial, that means that the
denominator of the filter's transfer function will have even degree
(excepting the real-valued poles, of course), and when it is factored
the conjugate pairs $(z_i,z_i^*)$ will appear as $(z-z_i)(z-z_i^*)$.
If we multiply this out, we get $z^2 - 2\Real[z_i] z +|z_i|^2$. In
other words, the imaginary parts of poles cancel each other out.
Since we're already using $b_k$ for the feedforward coefficients and
$a_\ell$ for the feedback ones in the filter equation, let's set $c_i
= -2\Real[z_i]$ and $d_i = |z_i|^2$.  If the transfer function's
denominator is of order $N$ ($N$ even), then we can multiply out the
polynomials for each complex conjugate pair and rewrite the
denominator as the product of $N/2$ second-order polynomials
\begin{equation}
(z^2+c_0z+d_0)(z^2+c_1z+d_1)\cdots (z^2+c_{N/2-1}z+d_{N/2-1})
\end{equation}
with no need for complex arithmetic.  We've done this before
symbolically, using Euler's formula to eliminate the imaginary part of
a complex conjugate pair. The point here is that, since complex poles
always appear in conjugate pairs, we can do this with even quite
complicated filters to eliminate the need for performing complex
arithmetic or processing complex numbers --- a significant
optimization.

\subsection{Limitations of Numerical Accuracy}

When we talk of the mathematics of filters, we assume that numbers
have infinite precision.  Unfortunately, this is not the case for
computer implementation.  These days, computers (including digital
signal processors) typically use either 32 or 64 bits to represent
numbers (be they fixed or floating point).  That may seem like a great
deal of precision, but it is unfortunately not uncommon for
intermediate results in numerical computation to require many more
bits to retain needed precision.  There are a number of ways this can
happen, but one is where a computation is performed iteratively, so
that a long chain of operations is applied to inputs before they
become outputs. At each step of this chain, the result has limited
precision. In other words, the number we get is not exactly correct
--- in general, it can't be, since we don't have infinite precision.

This loss of precision can mount rapidly, eventually destroying the
result. We can think of this limited precision as an \emph{error}. If
the input is from a 16-bit A/D, for example, and we assume it does its
conversion absolutely accurately to the smallest bit (which, as you
have seen in chapter~\ref{ch:computer-signals} and will revisit in
chapter~\ref{ch:compression}, is probably not realistic), then that's
around $\log_{10} 2^{16} \approx 5$ decimal digits from the A/D, to be
generous. Each arithmetic operation we perform, regardless of the
number of bits we use, can have the undesirable effect of decreasing
the number of digits of precision we have.  Sums of approximate values
have errors which are the sums of their addends' errors.
Multiplication tends to magnify errors. Let's see this with a simple
example.

\paragraph{Example~4}
Consider the following two recurrence relations for computing the
\index{recurrence relation}
series $\{x[n]\} = \{1/3^n\}$ ($n = 0, 1, 2, \ldots$):
\begin{align}
x'[n]  &= \frac{1}{3} x'[n-1] \label{eq:series-direct} \\
x''[n] &= \frac{10}{3} x''[n-1] - x''[n-2]
\label{eq:series-twoterms}
\end{align}
Both of these equations are mathematically ``correct''. \emph{They
also look like the expressions we use for our filters.}  Yet, they
yield very different results because of loss of significance.  Let's
use an initial value of $x'[0] = 0.99996$ for~(\ref{eq:series-direct})
and initial values of $x''[0]=1$ and $x''[1]=0.33332$
for~(\ref{eq:series-twoterms}). This is an initial error of 0.00004
for $x'[0]$ and $0.00001\bar{3}$ for $x''[1]$. I'll use MATLAB to
compute the first ten terms in each sequence with double-precision
accuracy for each calculation.  The MATLAB code is
\index{MATLAB code!numerical error|(}

\begin{small}
\begin{verbatim}
% significance.m
% 2/22/02 MDS
% Examine how error grows in an iterative computation
stdout = 1;

% This isn't efficient, but it is straightforward

% Values of n for computation
n = [0:10];

% Series 1: the real McCoy
x = 1./3.^n;

% Series 2: x'[n] = 1/3 x'[n-1]
xp = 0.99996;

for n = [1:10],
  xp(n+1) = 1/3 * xp(n);  % Remember, MATLAB indices start at 1, not 0
end;

% Series 3: x''[n] = 10/3 x''[n-1] - x''[n-2]
xpp = [1 0.33332];

for n = [2:10],
  xpp(n+1) = 10/3 * xpp(n) - xpp(n-1);
end;

% Print out the results
fprintf(stdout, ' n          x[n]            x''[n]           x''''[n]\n');
for n = [0:10],
  fprintf(stdout, '%2.1d\t%12.10f\t%12.10f\t%12.10f\n', ...
          n, x(n+1), xp(n+1), xpp(n+1));
end;
\end{verbatim}
\end{small}

The output this script produces is:
\begin{verbatim}
 n          x[n]            x'[n]           x''[n]
 0      1.0000000000    0.9999600000    1.0000000000
 1      0.3333333333    0.3333200000    0.3333200000
 2      0.1111111111    0.1111066667    0.1110666667
 3      0.0370370370    0.0370355556    0.0369022222
 4      0.0123456790    0.0123451852    0.0119407407
 5      0.0041152263    0.0041150617    0.0029002469
 6      0.0013717421    0.0013716872    -0.0022732510
 7      0.0004572474    0.0004572291    -0.0104777503
 8      0.0001524158    0.0001524097    -0.0326525834
 9      0.0000508053    0.0000508032    -0.0983641945
10      0.0000169351    0.0000169344    -0.2952280648
\end{verbatim}
\index{MATLAB code!numerical error|)}

The first column is accurate to the full precision of IEEE floating
point (that is, the computation is that accurate; the output was
limited to ten decimal places).  The second column's error is
\emph{stable}: it decreases in an exponential manner, and the value
for $x'[10]$ is within $7 \times 10^{-10}$ of the actual value. The
third column's error is \emph{unstable}: it increases exponentially.

Couple this with the need for precise pole placement, and you
can see that we need to be careful about our implementations. We saw
in this chapter's section on combining feedforward and feedback
filters the general transfer function for a filter~(\ref{eq:gen-xfer})
and its direct implementation~(\ref{eq:gen-impl}), which involved
$M$ delayed values of $x$ and $N$ delayed values of $y$.

\index{filter!cascade}
However, it is better to implement such a digital filter as a cascade
of second-order ones, by factoring the numerator and denominator
(which should be ``easy,'' since we know the locations of the poles
and zeros) and multiplying out terms with complex conjugate pairs as
described in the previous section. The resultant fraction will have
numerator and denominator which are each a product of second-order
polynomials.  Each ratio of second-order polynomials is a subfilter,
which can be implemented by a simple update equation. If we take the
output of one subfilter and present it as the input of the next, the
output of the last subfilter will be equivalent to the output of the
entire original filter (we saw this cascade of filters in
chapter~\ref{ch:filt-intro}). Note that, though each filter has a very
short (two step) queue, the series combination produces what amounts
to a longer overall queue. In practical applications, this shouldn't
be much of a computational penalty.

\section{Problems}

\section{Further Reading}

\begin{itemize}
\item James H McClellan, Ronald W. Schafer, and Mark A. Yoder,
  \textit{DSP First: A Multimedia Approach}, Prentice Hall, 1998,
  chapter 8 (\S 8.1--8.6, 8.8).
\end{itemize}

% LocalWords:  feedforward phasor reson


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                                                                      %
% GLE - Graphics Layout Engine <http://www.gle-graphics.org/>          %
%                                                                      %
% Modified BSD License                                                 %
%                                                                      %
% Copyright (C) 2009 GLE.                                              %
%                                                                      %
% Redistribution and use in source and binary forms, with or without   %
% modification, are permitted provided that the following conditions   %
% are met:                                                             %
%                                                                      %
%    1. Redistributions of source code must retain the above copyright %
% notice, this list of conditions and the following disclaimer.        %
%                                                                      %
%    2. Redistributions in binary form must reproduce the above        %
% copyright notice, this list of conditions and the following          %
% disclaimer in the documentation and/or other materials provided with %
% the distribution.                                                    %
%                                                                      %
%    3. The name of the author may not be used to endorse or promote   %
% products derived from this software without specific prior written   %
% permission.                                                          %
%                                                                      %
% THIS SOFTWARE IS PROVIDED BY THE AUTHOR "AS IS" AND ANY EXPRESS OR   %
% IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED       %
% WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE   %
% ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY       %
% DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL   %
% DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE    %
% GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS        %
% INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER %
% IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR      %
% OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN  %
% IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.                        %
%                                                                      %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Contour}
\index{contour}
\index{.z file}

The contour block produces contour lines of a function $z = f(x,y)$.

The function $f(x,y)$ is given by a .z file. The .z file format is discussed on page~\pageref{zfile:pg}. Recall that a .z file can be created from sample data points, that is $(x,y,z)$ tuples, with the {\sf fitz} block (Section~\ref{fitz:sec}), or from an implicit definition of $f(x,y)$ with a {\sf letz} block (Section~\ref{letz:sec}).

\begin{minipage}[c]{8cm}
\begin{Verbatim}
include "contour.gle"

begin contour
   data "saddle.z"
   values 0.5 1 1.5 2 3
end contour

begin graph
   title "Saddle Plot Contour Lines"
   data  "saddle-cdata.dat"
   d1 line color blue
end graph

contour_labels "saddle-clabels.dat" "fix 1"
\end{Verbatim}
\end{minipage}
\hfill
\begin{minipage}[c]{7cm}
\mbox{\input{utilities/fig/saddle-contour.inc}}
\end{minipage}

The contour block can contain the following commands:

\begin{commanddescription}
\item[{\sf data {\it file\$}}]\index{data} Specifies the name of the .z file.

\item[{\sf values $v_1,\ldots, v_n$}]\index{values} Specifies the z-values to contour at.

\item[{\sf values from $v_1$ to $v_n$ step $s$}] Specifies the z-values to contour at by means of from/to/step.

\item[{\sf smooth {\it integer}}]\index{smooth} Specifies the smoothing parameter.
\end{commanddescription}

The contour block creates the files ``data-clabels.dat'' and ``data-cdata.dat'' with the prefix ``data'' the name of the .z file. The file ``data-clabels.dat'' contains information for drawing labels on the contour plot. This is done by the subroutine {\sf contour\_labels} defined in the library ``contour.gle'' in the example above. The file ``data-cdata.dat'' contains the $(x,y)$ values of the contour lines. This file can be used as input to a graph block and plotted with the ``d1 line'' command as shown in the example above.

\section{Color Maps}
\label{colormap}
\index{colormap}

Color maps plot a function $z = f(x,y)$ by mapping $z$ to a color range. The following example combines a color map with a contour plot.

\begin{minipage}[c]{8cm}
\begin{Verbatim}
begin contour
   data "volcano.z"
   values from 130 to 190 step 10
end contour

begin graph
   title "Auckland's Maunga Whau Volcano"
   data "volcano-cdata.dat"
   xaxis min 0 max 20
   yaxis min 0 max 20
   d1 line color black
   colormap "volcano.z" 100 100
end graph
\end{Verbatim}
\end{minipage}
\hfill
\begin{minipage}[c]{7cm}
\mbox{\input{utilities/fig/volcano.inc}}
\end{minipage}

The options to the colormap command are as follows:

\begin{commanddescription}
\item[{\sf colormap {\it fct} {\it pixels-x} {\it pixels-y} [color] [invert] [zmin $z_1$]  [zmax $z_2$] [palette {\it pal}]}]

\mbox{}\\
\begin{itemize}
\item {\it fct} specifies the function to map. This can either be the name of a .z file, or it can be a function definition $f(x,y)$.

\item {\it pixels-x}, {\it pixels-x} specify the dimension of the color map. A color map is a stored as a bitmap image and ({\it pixels-x}, {\it pixels-x}) are the resolution of this bitmap. A larger resolution yields more detail, but at the cost of longer computation time and a larger file size.

\item {\it color} is an optional argument and indicates that the color map should be drawn in color (as opposed to grayscale).

\item {\it invert} is an optional argument that inverts the color map. That is, large function values will be drawn in black and small function values in white.

\item {\it zmin}, {\it zmax} are optional arguments that specify the range of the function.

\item {\it palette} is an optional argument that specifies the palette to use. A palette is a subroutine that maps $z$ values to colors. A number of example palette subroutines are included in the library ``color.gle''.
\end{itemize}
\end{commanddescription}

The following example is a color map of a two dimensional Gaussian.

\preglegraph{}
\begin{minipage}[c]{8cm}
\begin{Verbatim}
include "color.gle"

sub gauss x y
   s = 0.75
   return exp(-(x^2+y^2)/(2*s^2))
end sub

begin graph
   title "2D Gaussian"
   xaxis min -2 max 2
   yaxis min -2 max 2
   colormap gauss(x,y) 200 200 zmin 0 zmax 1 color
end graph

amove xg(xgmax)+0.3 yg(ygmin)
color_range_vertical zmin 0 zmax 1 zstep 0.1 format "fix 1"
\end{Verbatim}
\end{minipage}
\hfill
\begin{minipage}[c]{7cm}
\mbox{\input{utilities/fig/gaussian.inc}}
\end{minipage}
\postglegraph{}