%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Chapter: Introduction to Algorithms
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\documentclass[../main.tex]{subfiles}
\begin{document}
\begin{chapquote}
{Niklaus Wirth, \textit{Algorithms + Data Structures = Programs, 1976}}
``We problem modeling with data structures and problem solving with algorithms, Data structures often influence the details of an algorithm. Because of this the two often go hand in hand.''
\end{chapquote} % this is a fake quote, need to change later

In this chapter, we build up a global picture of algorithmic problem solving  to guide the reader through the whole ``ride''.
%
% ``Problem modeling'' are about understanding the problem and followed by ``problem solving''--set up a series of operations on data structures that can get the output from the input. The quote might not be totally true; when we are modeling/understanding a problem, the importance of data structure is highly dependable on the complexity of a problem--for simple problems, fixing its data structures might help you come up with a solution right away, whereas, in a more complex scenario where it seems no data structures or classical algorithms can fit in to solve it, data structures will be the last element to worry about. In all, it tells us the inseparable relation between data structures and algorithms.
\section{Introduction}
\label{sec_introduction_introduction}
In the past, a person who is capable of solving  complex math/physics computation problem faster than the ordinaries stands out and is highly seek out. For example, during world war two, Alen Turing hired engineer who was fast solving the Sudoku problems. These kind of stories die with the rise of powerful machines, with which the magic sticks are handed over to ones--programmers who are able to harness the continually growing computation power of the hardwares to solve those once only a handful or none of people that can solve, with algorithms.

There are many kinds of programmers. Some of them code  the real-world, obvious and easy rules to implement applications, some others challenge more computational problems with  knowledge in math, calculus, geometry, physics, and so. We give a universal definition of algorithmic problem solving--information processing. Three essential parts include: Data structures, algorithms, and programming languages. Knowing some basic data structures, some types of programming languages and some basic algorithms are enough for the first type of programmers. They might focus more on the front-end, such as mobile design, webpage design. The second type of programmers, however, need to be equipped with more advanced data structures and algorithm design and analysis techniques. Sadly, it is all just a start, the real powerful lie in the combination of these algorithm design methodologies and the other subjects. Math among all is the most important, for both design and analysis, as we will see in this book. Still a candidate with strong algorithmic skills is off a good start, at least with some basic math knowledge, we can almost always manage to solve problems with brutal force searching, and some others with dynamic programming.

Let us continue to define the  algorithmic problem solving as information processing, just \textbf{what} it is, and not \textbf{how} at this moment.
\subsection{What?}
\paragraph{Introduction to Data Structure} Information is the data we care about, which needs to be structured. And we can think of data structure as our low-level file manager, what it needs to do is to support four basic operations--'find' a file belongs to Bob, 'Add' Emily's file, 'Delete' Shown's file, 'Modify' Bod's file. Why structured? If you are the file manager, would you just throw all the hundreds of files over the floor or just throwing over in the drawer? Nope, you line them up in the drawer, or you even put a name on top of each file and order them by their first name. The way data is structured in program is similar to real-world system, simply lining up, or organize like a tree structure if there is some belonging and hierarchical ordering which appears in institutions and companies.

\paragraph{Introduction to Algorithms} Algorithms further process  data with a series of basic operations--searching, modifying, inserting, deleting, and so--that come with input data's structures or even auxiliary data's  structures if  necessary. How to design and analyze this series of operations are the field of algorithmic problem solving.

Same problem can be solved with different level of complexities in time and storing space. Deep down, algorithm designers

With this information processing step, we get our task done--computing our high school math, sorting the student ids in order, searching a word in a document, you name it.

\paragraph{Programming Language} A programming language especially higher level of language such as Python would come with data structures that might already have the basic operations: search, modify, insert, delete. For example, the \texttt{list} module in Python, it is used to store an array of items, it comes with \texttt{append()}, \texttt{insert()}, \texttt{remove}, \texttt{pop} that you can operate your data, thus, a \texttt{list} can be viewed as a data structure. If we know what data structure we save our input instance, what algorithms to use to operate the data, we can code these rules with a certain programming language and let the computer take over and if it won't demands billions of operations, it will get the result way more faster than humans are capable of, this is why we need computers anyway.

\subsection{How?}
Knowing what it is, now, you would ask how.  How can we know how to organize our data, how to design and analysis our algorithm? how to program it?  We need to study existing and well-designed data structures, algorithm design principle and algorithm analysis techniques, understand and analyze our problems, and study classical algorithms that our predecessors invented for solving a classical problem, only then when we are seeing a problem, old or new, we are prepared, we compare it with problems we know how to solve: if it is exact the same same, congratulations, we would solve our problem; if it is similar to a certain category of problems, at least we start from a direction and not from scratch; if it is totally new, at least we have our algorithm design principle and analysis techniques, we design one after understanding the problem and relate it to all our skills. Of course, there are problems that no body has been able to solve it yet. We will study it in the book so that you would identify when the problem you are solving is too hard.

\paragraph{The Tree of Algorithmic  Problem Solving} Back to the question,how? We study and build up our knowledge and skill base. A well-organized and explained knowledge base will surely ease our nerves and make things easier.  The field of algorithms and computer science is highly flexible. Assuming the knowledge of computer science is a tree, and assume that each leaf is a specific algorithm to solve a specific type of problem. What would be the root of the tree? The main trunk, branches? It is impossible for us to check or even count the number of leaves. But, we can understand the tree by knowing its structures. This book is fascinated with this belief and shows a lot of effort into organizing the algorithm design and analysis methodologies, data structures, and problem patterns. It starts with  the rooting algorithm design and analysis principle, and we study classical leaves by relating it and explained with our principle rather than treating each one individually.

The algorithm design and analysis principles comprise the trunk of the algorithm tree. A branch would be applying a type of algorithm design principle on a certain type of data structure, for example, algorithms related to tree structures, to graph structures, to string, to list, to set, to stack, to priority queue and so on.

\subsection{Organization of the Contents}
Based on our understanding of what is algorithmic problem solving and how to solve it, we organize the content of the book as:
\begin{itemize}
    \item Part~\ref{} includes the abstract commonly used data structures in computer science, the math tools for design, correctness prove, and some geometry knowledge that we need to solve geometry problems that are still often seen in the interviews.
    \item Part~\ref{}  strengthens the programming skills by implementing data structures and some basic coding.
    \item Part~\ref{} is our main trunk of the algorithmic programming solving.
    \item Part~\ref{} to Part~\ref{} takes us to different branches and showcases classical algorithms within that branch. One or many algorithm design principles can be applied to solve these problems.
    \item Part.~\ref{} is the problem patterns. Actually, if we have a good grasp of the sections before, this section is more of a review and exercises section. The finding of the patterns are to ease our coding interview preparation.
\end{itemize}

% \begin{itemize}
%     \item Data Structures: Algorithms are like the brain of an animal, and the data structures are like the skeletons. Data structures decide where and how the data is saved and accessed. And algorithms need to be shaped in a way to fit on the skeletons. For example, if you are given a skeleton of a dog, your brain needs to manage to walk and run by coordinating the four legs instead of two legs that comes with a human skeleton. We introduce each type of data structure in Part.~\ref{part_data_structure}.

%     \item Algorithm Design and Analysis: With the problem at hand, we need algorithm design methodologies to derive some sets of rule and actions that can fully or partially solve our problems with all the knowledge we can -- common sense, instinct, math, physics, and you name it! Analyzing the algorithms is like estimating or evaluating its performance. In realty, a problem can be solved with multiple different algorithms, which makes algorithm analysis an essential tool we live upon to make the best choice among options. We introduce the principle of algorithm design and analysis in Part.~\ref{part_algorithm_design_and_analysis}, searching in Part.~\ref{part_complete_searching} and optimization methods in Part.~\ref{part_dp_greedy}

%     \item Programming Language: At this step, we have all of our solutions derived and analyzed on paper! With a programming language such as Python that we use in this book, we are able to put it into construction and followed by execution to get real solution for complex problems that would be infeasible to compute by us humans.

% \end{itemize}

%  I'm not going to expand anything about algorithm design principle or analysis skills, data structures or classical algorithms or problems, cause that is the content of the book! Here, we focus on the approaches; we separate the algorithmic problem solving into two steps: Problem Modeling and Problem Solving. Dont' worry that your knowledge base is not large enough, just focus on how to solve our problem with your limited knowledge base.  We will extend the content of problem modeling and problem solving in the next two sections.

As a part of the introduction part,  As I always believe, setting up the big picture should be be very first part of any technical book; it helps to know how each part  plays its role global-wise with  more details comes from the preface. The organization of this chapter follows the global picture and each element of algorithmic problem solving is further briefed on in each section:
\begin{itemize}
    \item Problem Modeling (Section.~\ref{sec_problem_modeling}), includes Data structures, hands-on examples.
    \item Problem Solving (Section.~\ref{sec_problem_solving}), includes Algorithm Design and Analysis Methodologies (Section.~\ref{sec_algorithm_design})  and Programming Language(Section.~\ref{sec_programming_languages}).
\end{itemize}

\section{Introduction}
\label{sec_history_computer_science}
\paragraph{Algorithms are Not New} Algorithms should not be considered purely abstract and obscure. It origins from real-life problem solving including time before there even exist computers (machines).  The recurrence were studied as early as 1202 by L. Fibonacci, for whom the fibinocci number is named. Algorithms, as a set of rules/actions to solve a problem, they leverage any form of knowledge -- math, physics. Math stands out among all, as it is our tool to understand problems, present relations, solve problems, and analyze complexity. In this book, we use math in the most practical way and only at places where it really matters.  The difference is, with computer program written in a certain computer language to execute the algorithm is way more efficient generally than doing it in person.

\paragraph{Algorithms are Everywhere} in our daily life.   Assume you are given a group of people, your task is to find if there is a person in the group that is born on a certain day.  The most intuitive way to do is to check each of them and see if his/her birthday matches with the target, this needs you to go a full-round of this group of people. If you observed that this group of people is grouped by the months, then you can nail down the times of checking by checking the subgroup that matches the month of your target day. The first way is the easiest and most straightforward way to solve a problem, which is called brute force. The second one is involves more observation and might takes less time to get the answer.  However, they both have one thing in common, need us to nail down the possibilities; in the first way, we nail it down one by one, and in the second, we nail it down by almost 11/12 of the original possibility. We can say solving the problem is to find its solution  in solution space, and different way of finding the solution is called different algorithm.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Problem Modeling}
\label{sec_problem_modeling}
 The very first thing is to find or be given a problem exist in the world and solving it can bring practical value and hopefully make some good effect on the mother natural or humanity. In problem modeling, we analyze the characteristics of problem and relate it on certain data structures. %This element is the most difficult to abstract since it is highly dependable on problems and might require multi-disciplined knowledge. We will try to offer some guideline in this chapter, however, mostly we will rely on learning classical problems and its corresponding algorithms to improve our instinct or feeling of the algorithm problem solving.

% The meaning of existence of computer science is to solve real-world problems. Understanding how to extract from real-world problems and restate them using a computer science languages and terminologies is the main purpose of this section.

In the stage of problem modeling, we define the problem and model our problems with data structures. In this section, we first answer the question, ``what is a problem in computer science?'' Then, we introduce the ``skeleton'' --Data Structures to prepare for our next step --problem solving. Then, we give hands-on Examples about how to model a problem with data structures.%of the algorithms, In order to do this,  the We answer three questions here: (1)

If you are a zoologist, these are how you define a species: describe the fresh and appearance, put together its skeletons, search similar well-studied species from dataset, match observed behaviors,  and induce the unknown ones from similar species.   There are two key steps to problem modeling:
\begin{enumerate}
    \item Understand the problems (Section.~\ref{sec_problem_statement}): We give the definition of problems, followed by the problem categories which categorizing problems without the context of data structures. This is like describe the fresh of a species
    \item Apply data structures to the problems (Section.~\ref{sec_data_structures}): We then describe our problem in terms of data structures; connecting the fresh and the skeletons. We also analyze the the problem by exploring its solution space and simulating the process; finding the series of actions between the input and output instance.
\end{enumerate}

\subsection{Understand Problems}
\label{sec_problem_statement}
\subsubsection{Problem Formulation} A problem can be a task or something to be done according to the definition of ``problem'' in English dictionary, such as finding a ball numbered $11$ from a pool of numbered balls,  sorting the list of students by their IDs. The first thing we need to understand should be \textit{problem formulation} and the closest knowledge we need to define a problem comes from the field of math. The intuitive definition of a problem is that it is a set of related tasks, usually infinite.

The formal definition of problems: A problem is characterized by:
\begin{enumerate}
    \item \textbf{A set of input instances:} The \textit{instance} represents some real examples of this type of problems. And input instances are data, which needed to be saved and accessed from the machine. This mostly requires us to define a data structure, however, different data structures can be used to define.
    \item \textbf{A task to be preformed on the input instances:} The problem definition should usually comes with examples to better explain how the task decides the output of the exemplary input instances.
\end{enumerate}

For example, we formulate the problem of drawing a call from the pool as: Given a list of unsorted integers, find if the number $11$ is in the list, return true or false.
\begin{lstlisting}[numbers=none]
Example:
Given the list: [1, 34, 8, 15, 0, 7]
Return False because 11 does not appear in the list.
\end{lstlisting}

\subsubsection{Problem Categories} Now, to better understand what computer science deals with, we categorize problems commonly solved in the field.

\paragraph{Continuous or Discrete?}
Based on whether the variables are continuous or discrete, we have two categories of problems:
\begin{enumerate}
    \item \textbf{Continuous problems:} relates to continuous solution spaces.
    \item \textbf{Discrete problems:} relates to discrete solution spaces.
\end{enumerate}
The field of algorithmic problem solving is highly correlated to \textit{Discrete Mathematics}, which covers topics such as arithmetic and geometric sequence, recurrence relations, inductions, graph theory, generating functions, number theory, combinatorics, and so. Through this book. some important parts are detailed (recurrence relation, induction, combinatorics, graph theory) which serves as powerful tools to do good job in computer science.
\paragraph{What do They Ask?}
We may be asked to answer four types of questions:
\begin{enumerate}
    \item \textbf{YES/No Decision Problems:} answering whether a number is prime, odd or even are examples of such decision problems.
    \item \textbf{Search problems:} Find one/all \textit{feasible solutions} that meets problem requirement, which requires the identification of a solution from within a potentially infinite set of possible solutions. For example, finding the $n^{th}$ prime number. Almost all problems are or can be converted to a search problem in some way. Further, search problems can be divided into:
    \begin{itemize}
    \item \textbf{Counting Problems:} Count all feasible solutions to a search problem, such as answering, `how many of the 100 integers are prime?'.
    \item \textbf{Optimization Problems:} Find the \textit{best solution} among all feasible solutions to a search problem.  In addition to the search problem, optimization problems answers the decision, `is the solution the best among all feasible ones?'.
    \end{itemize}
\end{enumerate}

\paragraph{Combinatorics}
When discrete problems are asked with counting or optimization questions, in computer science we further have combinatorial problems, which is also widely called \textit{combinatorics}.

Combinatorics originates from discrete mathematics and become part of computer science. As the name suggested, combinatorics is about combining things; it answers questions: "How many ways can these items be combined?", and "Whether a certain combination is possible, or what combination is the `best' in some sense?"

Through this book, permutations, combinations, subsets, strings, points in the linear order, and trees, graphs, polygons in the non-linear ordering will be examined (suggest contents in the book). We will have some briefy study on this topic in Chapter~\ref{part_combinatorial_problems}.
% \begin{enumerate}
%     \item Combinatorial problems: such as permutations, subsets, strings, trees, graphs, points, polygons.
%     \item Combinatorial optimization problems, aka \textit{discrete optimization}: combinatorial optimization problems are a subset of the combinatorial problems. \textcolor{red}{This subject originally grew out of graph theoretic concerns like edge colorings in undirected graphs and matchings in bipartite graphs. Much of the initial progress is due entirely to theorems in graph theory and their duals. With the advent of linear programming, these methods were applied to problems including assignment, maximal flow, and transportation. In the modern era, combinatorial optimization is useful for the study of algorithms, with special relevance to artificial intelligence, machine learning, and operations research.}
% \end{enumerate}
\paragraph{Tractable or Intractable?}
The complexity of a problem is normally described in relation with the size of the input instance. If a problem is algorithmic and computable, being able to produce a solution may depend on the size of the input or the limitations of the hardware used to implement it. Based on if  a problem can be possibly solved by existing machines we have:
\begin{enumerate}
    \item \textbf{Tractable problems:} If a problem has reasonable solution, that it can be solved in no more than polynomial time complexity, it is said to be tractable.
    \item \textbf{Intractable problems:} Some problems can only be solved with algorithms whose execution time grows too quickly in relation to their input size, say exponential, then these problems are considered to be intractable. For example, the classical Traveling Salesperson Problem.
\end{enumerate}

Problems can also be categorized as:
\begin{enumerate}
    \item \textbf{P Problems:}
    \item \textbf{NP Problems:}
\end{enumerate}
There are more types, such as \textit{undecidable problems} and \textit{the halting problems}, feel free to look them up if interested.
\subsection{Understand Solution Space}
\label{sec_data_structures}
A data structure is a specialized format of organizing, storing, processing, and  retrieving data. As Dr. Wirth states in the chapter quote, we problem modeling with data structures and the data structures often influence the details of an algorithm: the input/output instances, and the intermediate results in the process of an algorithms all associates to data structures.

In this section, we do not intend to get into details of data structures, but rather pointing out directions. Quickly skim the first section of Part.~\ref{} and get a sense of the categories of data structures. When a problem is modeled with data structures, the problems can further be classified based on its data structures. At this stage, we should try to model our input on a data structure, and analyze the following five components to even better understand our problem.

\paragraph{Five Components} There are generally five components of a problem that we can define and depends on to correlate the problem to data structures, and to algorithms--searching, divide and conquer, dynamic programming, and greedy algorithms. We introduce the five components with a dummy example:
\begin{lstlisting}[numbers=none]
Given a list of items A=[1, 2, 3, 4, 5, 6], find the position of item with value 4.
\end{lstlisting}
\begin{figure}[!ht]
    \centering
     \includegraphics[width=0.7\columnwidth]{fig/problem_formulation_2.png}
    \caption{The State Space Graph. This may appears as a tree, but we can redraw it as a graph. }
    \label{fig:problem_formulation}
\end{figure}

\begin{enumerate}
    \item \textbf{Initial State:} state that where our algorithm starts. In our example, we can scan the whole list starting from leftmost position 0, we denote it as $S(0)$. Note that a state does not equal to a point on the input instance, it can be a range --such as from position 0 to 5, or from 0 to 2, or any state you define.
    \item \textbf{Actions or MOVES:} describe possible actions relating to a state. Now, given position 1 with 2 as value, we can move only one step forward and get to position 2 or we can move 2, 3, 4, 5 steps and so. Thus, we should find all possible actions or moves that we can take to progress to next state. We can denote it as ACTIONS(1)={MOVE(1), MOVE(2), MOVE(3), MOVE(4), MOVE(5)}.

    \item \textbf{State transfer Model}: decides the state results from doing an action $a$ at state $s$. We denote it as $T(a, s)$. For example, if we are at position 1 and move one step, MOVE(1), then we can reach to state 2, which can be denote as $2=T(MOVE(1), 1)$. %We also use the term \textit{successor} to refer to any state reachable from a given state by a single action.

   \item \textbf{State Space:} is the set of all states reachable from the initial state by any sequence of actions, in our case, it can be {0, 1, 2, 3, 4, 5}. We can infer state space of the problem from the initial state, actions, and transfer model. The state space forms a directed network or \textit{graph} in which the nodes are states and the links between nodes are actions. Graph, with all its flexibity, is a universal and natural way to represent relations. For example, if we limit the maximum moves we can make at each state to two, the state space will be formed as follows in Fig.~\ref{fig:problem_formulation}. In practice, draw the graph as a tree structure is another option; in the tree, we observe repeat states due to the expansion of nodes in graph with multiple ingoing links.  A \textit{path} in the state space is a sequence of states connected by a sequence of actions.

   \item  \textbf{Goal Test:} the determines whether a given state is a goal state. %Sometimes there is an explicit set of possible goal states, and the test simply checks whether the given state is one of them.
   Such as in this example, the goal state is $4$. The goal is not limited to such enumerated sets of states, it can also be specified by an abstract property. For example, in the constraint state problems(CSP) such as the n-queen, the goal is to reach to a state that not a single pair of queens will attack each other.
\end{enumerate}

In this example, the space graph is an analysis tool; we use it to represent the transition relationship between different states not the exact data structure that we use to operate and define algorithms on.

\paragraph{Apply Data Structures}
\begin{figure}[!ht]
    \centering
    \includegraphics[width=0.9\columnwidth]{fig/problem_formulation_1.png}
    \caption{State Transfer process on a linear structure }
    \label{fig:problem_formulation_1}
\end{figure}

With the state space graph, our problem is abstracted to finding a node with value 4 and graph algorithms--more specifically, graph search--can be applied to solve the problem. It does not take an expert to tell us, "This graph just complicated the situation, because our intuition can lead us to a much simpler and straightforward solution: scan the items from the leftmost to the rightmost one by one". True! As is depicted in Fig.~\ref{fig:problem_formulation_1}, the problem can be modeled using a linear structure, possiblly a list or linked list, and we only need to consider one action out of all options, MOVE(1), then our searching covers the whole state space, which makes the algorithm we designed \textit{complete}~\footnote{Check complexity analysis}. On the other side, in the state space graph, if we insist on moving two steps each time, we would not be able to cover the whole state space, and might end up not finding our target, which indicates this algorithm is \textit{incomplete}.
\begin{figure}[!ht]
    \centering
    \includegraphics[width=0.7\columnwidth]{fig/problem_formulation_3.png}
    % \includegraphics[width=0.7\columnwidth]{fig/problem_formulation_4.png}
    \caption{State Transfer Process on the tree}
    \label{fig:problem_formulation_2}
\end{figure}

Instead of using linear data structure, we can restructure the states as a tree if we refine the state as a range of items. The initial state  is the possible subarray the target can be found, denote as $S(0, 5)$. Start from initial state, each time, we divide the space into two halves: $S(s, m)$ and $S(m, e)$, where $s, e$ is the start and end index respectively, and $m=(s+e)//2$, meaning the integer part of $s+e$ divided by 2. We do this to all nodes repeatedly, and we will have another state transfer graph shown in Fig.~\ref{fig:problem_formulation_2}.From this graph we can see, the last node will be where we can not divide further, that is when $s=e$. From state $0-5$ to $3-5$ needs an action--move to the right. Similarly, from $0-5$ to $0-3$ needs the action of moving to the left.  We use {MOVE(L), MOVE(R)} to denote the whole set of possible actions to take.

In this example, we showed how to same simple problem can be modeled using two different data structures--linked list and tree.

%we introduce the definition and the type of data structures; and demonstrate how it can be used to model our problems.   of data structures as a way to demonstrate our organization of part.~\ref{part_data_structure} and a premise knowledge for the next section where we examine and discuss the problem modeling with real examples.

% \subsubsection{Data Structures Definition}

% \subsubsection{Data Structures Categorizes} Put figures here for visualization and comparison.
% \begin{enumerate}
%     \item Linear Data Structure: such as arrays, strings, heaps, priority queue.
%     \item Non-linear Data structure: such as tree, graph
%     \item Sturcutual: points, polygons.
% \end{enumerate}

% Throughout this book, we will see that all problems are associated with fundamental data structures such as array, strings, trees, graphs, points, polygons. With different data structure, specific algorithms will be applied to solve these two problems. Data Structure is a way of collecting and organizing data in such a way that we can perform operations on these data in an effective way. Data Structures is about rendering data elements in terms of some relationship, for better organization and storage. \textit{In practice, data structures are utilized to model a problem so that it can be solved with a corresponding algorithm. }

% \textit{Data strutures and algorithms are inseparable in computer programming. }

% In order to do comparison between all possible devised algorithms for our problem, we need to learn how to  evaluate their performance with time complexity and space complexity. There are some techniques we will introduce before we dive into the four problem solving paradigm and Cracking LeetCode Problems (Part~\ref{part_cracking_leetcode_problem}), we will learn how to do complexity analysis of algorithms in Section~\ref{sec_complexity_analysis}.

% The above example is to show us, how learning and practice using the data structures and four problem solving paradigms can help us making smarter decision about problem modeling and problem solving.

\section{Problem Solving}
\label{sec_problem_solving}
In this section, we will first demonstrate how algorithm can be applied on these two data structures with its corresponding state transfer process. Following this, we introduce the four fundamental algorithm design and analysis methodologies--the ``soul/brain''.
 We end this section by briefing on categorizing algorithms.

 \subsection{Apply Design Principle}
\begin{figure}[!ht]
    \centering
    \includegraphics[width=0.9\columnwidth]{fig/problem_formulation_1_1.png}
    \caption{Linear Search on explicit linear data structure }
    \label{fig:problem_formulation_3}
\end{figure}
Given the state transfer graph in Fig.~\ref{fig:problem_formulation_1}, we simply iterate each state and compare each item with our target to see if it equals; if true, we find our target and return, if not, we continue to the end. This simple search method is depicted in Fig.~\ref{fig:problem_formulation_3}.
\begin{figure}[!ht]
    \centering
    \includegraphics[width=0.7\columnwidth]{fig/problem_formulation_4.png}
    \caption{Binary Search on an implicit Tree Structure}
    \label{fig:problem_formulation_4}
\end{figure}
What if we know that the data is already organized in ascending order? With the tree data structure, when given a specific target, we only need to choose one action from the actions set; either move to left or right to search with a condition: if target is larger or smaller than the item in the middle of the state. When 4 is target, we have the search process depicted in  Fig.~\ref{fig:problem_formulation_4}.

All these state space, data structure, algorithm, and analysis might appear overwhelming to you for now. But as you learn, some of these steps are not necessary, but knowing these elements are good for you to analyze and learn new algorithms, think of it more gathering terminologies into your language base.

\subsection{Algorithm Design and Analysis Principles}
\subsubsection{Algorithm Design}
\label{sec_algorithm_design}
More of the time, the most naive and inefficient solution -- \textit{brute-force solution} would strike us right away, which is simply searching a feasible solution  to the problem in its solution space using the massive computation power of the hardware. Although the naive solution is not preferred by your boss nor it will be incorporated into the real product, it offers the baseline for your complexity comparison and to showcase how good your well-designed algorithm is.

In the dummy example, we actually used two different searching algorithms--linear search and binary search. The process of looking for a sequence of actions that reaches the goal is called search. Therefore, \textit{searching} is the fundamental strategy and and the very first step to problem-solving. How could it not be? Algorithms are about to find answers to problems, if and assuming we can define out potential state/solution space, then a naive/exhuastive searching would do the magic and solve the problem. However, back to reality, we are limited by computation resource and speed, we comprise by:
\begin{enumerate}
    \item being smarter that we can be decrease the cost, increase the speed, and yet still gives out the exact solution we are looking for. This comes down to \textit{optimization}, which we have \textit{divide and conquer}(Chapter~\ref{}), \textit{dynamic programming}(Chapter~\ref{}), and \textit{greedy algorithms}(Chapter~\ref{}). What are the commonality between them? They all in some way need us to get \textit{recurrence relation}((Chapter~\ref{}), which is essentially \textit{mathematical induction}(Chapter~\ref{}), which I generalized from another book, \textit{Introduction to Algorithms: A Creative Approach}, by Udi Manber. Explain it in another way, these principles are using recurrence relation to find the relation of a problem with its smaller instance. Why is it smarter? First, smaller problems are just easier to solve than larger problems. Second, the cost of assembling the answer to smaller problems to answers to the larger problems is possibly smaller.
    \item by approximating the answer. Instead of trying to get the exact answer, we find one that is good enough. Here goes to all heuristic search, machine learning, artificial intelligence. Guess, currently my limited knowledge is not enough for me to give more context that this.
\end{enumerate}
Equally, we can say all algorithms can be described and categorized as searching algorithms. Yet, there are three algorithm design paradigms-- \textit{Divide and Conquer}, \textit{Dynamic Programming}, and \textit{greedy Algorithms}, can be applied in the searching process for faster speed or using less space.
Don't worry, this is just the introduction chapter, all these concepts and algorithm design principles will be explained later.

% \subsubsection{Algorithm Design Methodologies}
% In this section we will briefly offer a glimpse into their concepts and discerns.
% \begin{figure}[h]
%     \centering
%     \includegraphics[width=0.6\columnwidth]{fig/divide_dynamic.png}
%     \caption{The dividing of problems of Divide and Conquer VS Dynamic programming. (Note: the left side in the red box is the Divide and Conquer, and the blue box is the dynamic programming.) }
%     \label{fig:divide_conquer_vs_dynamic_programming}
% \end{figure}

% \paragraph{Divide and Conquer} is the most fundamental programming philosopy.  It first recursively break a problem into smaller non-overlapping subproblems till a small base subproblem which can be solved easilty, and then combining the results of the subproblems into the solution to its superme problem in some way. The process is demonstrated in Fig~\ref{fig:divide_conquer_vs_dynamic_programming}, and it usually be implemented with recursive function. It usually decrease the time complexity of logarithm level. For instance, it optimize a $O(n^2)$ time complexity to $O(n\log n)$.

% \paragraph{Dynamic programming} follows the same philosophy of Divide and Conquer and commonly used to tackle optimization problems. It also first break a problem into subproblems. But instead of the non-overlapping subproblems, their subproblems overlaps in a way as demonstrated in Fig~\ref{fig:divide_conquer_vs_dynamic_programming}, which means a larger size subproblem grows from smaller previous subproblems. The solution to current subproblem can depends on any number of previous smaller subproblems.  With dynamic programming, intermediate results are cached and can be used in subsequent operations.
% %Second, we use the knowledge we learned from normal algorithm books with algorithm methodology like divide and conquer, dynamic programming, greedy algorithms and so on. This normally be more efficient, if we are pick the right method. There methods will be introduced in details in Part~\ref{part_algorithms}.

% \paragraph{Greedy algorithms} often involve optimization and combinatorial problems; the classic example is applying it to the traveling salesperson problem, where a greedy approach always chooses the closest destination first. This shortest path strategy involves finding the best solution to a local problem in the hope that this will lead to a global solution.

% \textcolor{red}{\paragraph{Complete Search} Complete search, also known as brute force or recursive backtracking, is a method for approaching a problem by naively searching through the whole solution spaces to obtain the required solution. Optimization is through pruning the searching space by ending invalid searching early. Complete search is used when there is clearly no clever algorithms available ( algorithms that use one of previous three paradigms), for instance with permutation and combination stated in Section~\ref{sec_combination}, or when such clever algorithms exist, but overkill when the input size happen to be small for complete search. }

% On the LeetCode, a lot of times, complete search should be the first considered soltion that come to mind. With bug-free complete search solution, we should never receive Wrong Answer response, but we might get Time Limited Error (TLE) instead due to its high time complexity.

\subsubsection{Algorithm Analysis of Performance}
How to measure problem-solving performance? Up till now, we have some basic ways to solve the problem, we need to consider the criteria that might be used to measure them. We can evaluate an algorithm's performance in four ways:
\begin{enumerate}
    \item \textbf{Completeness:} Is the algorithm guaranteed to find a solution when there is one?
    \item \textbf{Optimality:} Does the stategy find the optimal solution, as defined?
    \item \textbf{Time Complexity:} How long does it take to find a solution?
    \item \textbf{Space Complexity:} How much memory is needed to perform the search?
\end{enumerate}

Time and space complexity are always considered with respect to some measure of the problem difficulty. In theoretical computer science, the typical measure is the size of the state space graph, |V | + |E|, where V is the set of vertices (nodes) of the graph and E is the set of edges (links). This is appropriate when the graph is an explicit data structure that is input to the search program. However, in reality, it is better to describe the search tree that applied to search for our solutions. For this reason, complexity can be expressed in terms of three quantities: $b$, the \textbf{branching factor} or a maximum number of successors of any node; $d$, the \textbf{depth} if the shallowest goal node ; and $m$, the maximum length of any path in the state space. Time is often measured in terms of the number of nodes in the search tree, and the space are in terms of the maximum number of nodes stored in memory.

For the most part we describe time and space complexity for search on a tree; for a graph, the answer depends on how ``redundant" paths  or ``loops" in the state space are.

\subsection{Algorithm Categorization}
There are countless algorithms invented, however, these traditional data-independent algorithms (not the current data-oriented deep learning models which are trained with data), it is important for us to be able to categorize the algorithms and understand the similarities and characteristics of each type and also be able to compare each type:
\begin{itemize}
    \item By implementation: the most useful in our book is recursive and iterative. Understand the difference of these two, and the special usage of recursion (Chapter~\ref{chapter_iteration_recursion}) is fundamental to the further study of algorithm design.  We can also have serial and parallel/distributed, deterministic and non-deterministic algorithms. In our book, all the algorithms we learn are  serial and deterministic algorithms.
    \item By design: algorithms can be interpreted to one or several of the four fundamental problem solving paradigms, Divide and Conquer (Part~\ref{part_divide_conquer}), Dynamic Programming and Greedy (Part~\ref{part_dp_greedy}).  In Section~\ref{four_paradigm}, we will briefly introduce and compare these four problem solving paradigms to gain a global picture of the spirit of algorithms.
    \item By complexity: mostly algorithms can be categorized by its time complexity. Given an input size of $n$, we normally have categories of $O(1)$, $O(\log n)$, $O(n\log n)$, $O(n^2)$, $O(n^3)$, $O(2^n)$, and $O(n!)$. More details and the comparison is given in Section~\ref{complexity_subsec_cheat_sheet}.
\end{itemize}
% \subsubsection{Searching Algorithms}
% After the problem formulation, we will have a sense of state spaces, and to reach to the goal state, we need to search a path to reach to this state or a sequence of actions. Searching algorithms will be the most fundamental footstone of all algorithms. Essentially our state spaces are graphs. Under certain limitations, we can simplify it to linear data structure or tree structures.

% \paragraph{How searching works?} We start from the initial state and form a \textit{search tree} with the initial state as root; the branches are actions and the nodes corresponds to states in the state space of the problem. We search by \textit{expanding} current state; that is, applying each legal action to the current state, thereby reaching to a new set of states. The nodes available for expansion at any given point is called the \textbf{frontiers} (also called open list). There are fundamentally three types searching techniques:
% \begin{enumerate}
%     \item Breath-first Search: for example, if we are at 1, then our next sets of states are {2, 3}. Then we expand the states of 2, then states of 3, and save them to a data structures to expand later, they will be {3, 4, 4, 5}. So first, the frontier is {1}, then it becomes {2, 3}, then at node 2, we expand 3, 4, will end up with {3, 3, 4}. This is called breath-first search which expand the frontier in a First-in, first out or FIFO fashion (FIFO queue). Breath-first search is usually implemented iteratively.

%     \item Depth-first Search: similarily at state 1, we get our frontiers to be {2, 3}. But if we deal with the nodes in the frontier in a last-in, first out or LIFO fashion (LIFO queue and also known as stack). At node 1, our generated frontier will be [2, 3], then we go to state 3, and expand more states here, the resulting frontier will be [2, 4, 5]. Then move to state 5, with resulting frontier [2, 4, 6]. Depth-first search can either be implemented iteratively or recursively.

%     \item Priority based Search: if we pop the elements of the queue with the highest priority according to some ordering function, this is called proprity based search, this can be implemented with a priority queue.
% \end{enumerate}

The intractable problems are still get solved by computer. We can limit our input instance size. However, it is not very practical, when the size of the input size is large and we are still hoping to get solutions, maybe not the best, but are good enough in a a reasonable(polynomial) time.   \textit{Approximate algorithms} comes into our hand,  such as \textit{heuristic algorithm}. In this book, we focus more on the non-approximate algorithmic methods to solve problems in \textit{discrete} solution spaces, and only brief on the part of approximate algorithms.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Programming Languages}
\label{sec_programming_languages}

hird, for certain type of problems, there are algorithms specically designed and tuned to optimize that type of question. This wil be introduced in Part~\ref{part_specific_algorithms}. Which might give us almost the best efficiency we can find.

% The fourth level, is the ''clever'' way. For some problems, if you are creative enough, and you comprehensively analyze the problem and think about each case, we might come up with a very brilliant solution that beats all the previous solutions in both efficiency and code elegancy. This will not be systematically introduced in this book, because it varies problem to problem. But we will put some ``clever'' solutions to some problems either in examples or exercises used in this book.
% \subsection{Complete Search}

% \subsection{Divide and Conquer}
% \subsection{Dynamic Programming}
% \subsection{Greedy Algorithm}

\section{Tips for Algorithm Design}
\paragraph{Principle}
\begin{enumerate}
    \item Understand the problem, analyze with searching and combinatorics to get the complexity of the naive solution.
    \item If it is a exponential problem, check if the dynamic programming applies. If not, we have to stick to a search algorithm. If it applies, we can decrease the complexity to polynomial.
    \item If it is polynomial already or polynomial after the dynamic programming applied, check if the greedy approach or the divide and conquer can be applied to further decrease the polynomial complexity. For example, if it is $O(n^2)$, divide and conquer might decrease it to $O(n\log n)$, and the greedy approach might end up with $O(n)$.
    \item If none of these design principle applies: we stick to the searching and try to optimize with better searching techniques--such as backtracking, bidirectional search, $A^{*}$, sliding window and so on.
\end{enumerate}
This process can be generalized with ``BUD''--bottleneck, unncessary work, and D.

\section{Exercise}
\subsection{Knowledge Check}
\paragraph{Longest Increasing Subsequence}

\begin{bclogo}[couleur = blue!30, arrondi=0.1,logo=\bccrayon,ombre=true]{Practice first before you check up the solution. (put the solution at next page)}
\end{bclogo}
\begin{figure}[!ht]
    \centering
    \includegraphics[width=0.9\columnwidth]{fig/problem_formulation_1.png}

     \includegraphics[width=0.7\columnwidth]{fig/problem_formulation_2.png}
    \caption{The State Spaces Graph }
    \label{fig:problem_formulation}
\end{figure}
 Given a list of items $A=[1, 2, 3, 4, 5, 6]$, find the position of item with value $4$.
\begin{enumerate}
    \item \textbf{Initial State:} state that where our algorithm starts. In our example, we can scan the whole list starting from leftmost position 1. $S(0)$
    \item \textbf{Actions or MOVES:} A description of possible actions available at a state. If we are at position 1, we can have different possible actions, we can move only one step forward and get to position 2. Or we can move 2, 3, 4, 5 steps. We can denote it as ACTIONS(1)={MOVE(1), MOVE(2), MOVE(3), MOVE(4), MOVE(5)}.

    \item \textbf{State transfer or transition model}: It returns the state results from doing an action $a$ at state $s$. We denote it as T(a, s). For example, if we are at position 1 and take action that move one step, MOVE(1), then we can reach to state 2, denote as $2=T(MOVE(1), 1)$. We also use the term \textbf{successor} to refer to any state reachable from a given state by a single action.

  \item \textbf{State Space:} Together, the initial state,  actions, and transition model implicitly define the state space of the problem--the set of all states reachable from the initial state by any sequence of actions. The state space forms a directed network or \textbf{graph} in which the nodes are states and the links between nodes are actions. For example, if we limit the maximum moves we can make at each state to be one and two, the state space will be formed as follows in Fig.~\ref{fig:problem_formulation}. A \textbf{path} in the state space is a sequence of states connected by a sequence of actions.

  \item  \textbf{Goal Test:} the goal test determines whether a given state is a goal state. Sometimes there is an explicit set of possible goal states, and the test simply checks whether the given state is one of them. Such as in this example, the goal state is $4$. Sometimes the goal is specified by an abstract property rather than explicitly enumerated sets of states. For example, in the constraint state problems(CSP) such as the n-queen, the goal is to reach to a state that not a single pair of queens will attack each other.
\end{enumerate}

In practice, analyzing and solving a problem is not answering a yes or no question. There are always mutiple angels to model a problem, the way to model and formalize a problem decides the corresponding algorithm that can be used to solve this problem. And it might also decide the efficiency and difficulty to solve the problem. For example, using the Longest Increasing Subsequence:
\begin{figure}[h]
    \centering
    \includegraphics[width=\columnwidth]{fig/LIS_tree.png}
    \caption{State Transfer Tree Structure for LIS, each path represents a possible solution. Each arrow represents an move: find an element in the following elements that's larger than the current node.}
    \label{fig:tree_lis}
\end{figure}
\textbf{Ways to model the problem.} There are different ways to model this LIS problem, including:
\begin{enumerate}
    \item Model the problem as a directed graph, where each node is the elements of the array, and an edge $\mu$ to $v$ means node $v>\mu$. The problem now becomes finding the longest path from any node to any node in this directed graph.
    \item Model the problem as a tree. The tree starts from empty root node, at each level i, the tree has n-i possible children: nums[i+1], nums[i+2], ..., nums[n-1]. There will only be an edge if the child's value is larger than its parent. Or we can model the tree as a multi-choice tree: for combination problem, each element can either be chosen or not chosen. We would end up with two branch, and the nodes would become a path of the LIS, therefore, the longest LIS exist at the leaf nodes which has the longest length.
    \item Model it with divide and conquer and optimal substructure.
\end{enumerate}

\end{document}


\documentclass[main.tex]{subfiles}
\begin{document}

\subsection{Matter effects in neutrino oscillations}

\marginpar{Wednesday\\ 2021-12-15}

This is also called the MSW effect.
Neutrinos interact weakly with matter, meaning that they are rarely absorbed;
however, neutrinos may feel the presence of matter even without changing direction.

Through some interaction, a propagating neutrino may feel the presence of background fermions (matter).
This idea is called ``coherent forward scattering''.

The full Hamiltonian will include a neutrino ``interaction energy potential''
%
\begin{align}
H = \frac{1}{2E} U M^2 U ^\dag + \left[\begin{array}{ccc}
V_{ee} & V_{e \mu } & V_{e \tau } \\
V_{\mu e} & V_{\mu \mu } & V_{\mu \tau } \\
V_{\tau e} & V_{\tau \mu } & V_{\tau \tau }
\end{array}\right]
+ \text{terms proportional to } \mathbb{1}_3
\,.
\end{align}

Pictorially, this matrix \(V\) has a diagonal term driven by neutral current interactions,
with any neutrino interacting through a \(Z\) boson.
The coupling of the \(Z\) does not depend on the flavor, so that component is
proportional to \(\mathbb{1}\), meaning it does not affect oscillations.

A second term is the one corresponding to the \(W\) boson: for this, we only have
an effect for electron neutrinos, in the \(V_{ee}\) component.

At tree level, this is proportional to the Fermi constant \(G_F\).
Also, it will linearly depend on the electron number density.

The calculation yields
%
\begin{align}
V_{ee} &= \sqrt{2} G_F N_e
\qquad \text{for neutrinos}  \\
V_{ee} &= -\sqrt{2} G_F N_e
\qquad \text{for antineutrinos}
\,.
\end{align}

This effect is quite analogous to the propagation of light within a medium with a
non-1 index of refraction.
It alters the oscillation pattern both in amplitude and in frequency.

The main oscillation channel for atmospheric neutrinos is \(\nu _\mu \to \nu _\tau \),
meaning that these matter effect did not need to be included to study it.

It is convenient to introduce a term \(A = 2 E V_{ee} = 2 \sqrt{2} G_F N_e E\); then the Hamiltonian reads
%
\begin{align}
H = \frac{1}{2E} \left( U M^2 U ^\dag + \left[\begin{array}{ccc}
A & 0 & 0 \\
0 & 0 & 0 \\
0 & 0 & 0
\end{array}\right] \right)
\,.
\end{align}
%

Qualitatively, we can understand that this term will be important when \(A\)
is \(\gtrsim \delta m^2\) or \(\Delta m^2\).

It turns out that
%
\begin{align}
\frac{A}{\Delta m^2_{ij}} \approx \num{1.526e-7} \left( \frac{N_e}{\SI{}{mol/cm^3}}\right)
\left( \frac{E}{\SI{}{MeV}}\right)
\left( \frac{\SI{}{eV^2}}{\Delta m^2_{ij}} \right)
\,.
\end{align}

For the Sun, in the core, \(N_e \sim \SI{e2}{mol/cm^3}\), so we get that as an order of magnitude \(A \sim \delta m^2\).

Since the number density of neutrinos depends on position, we need to integrate the Schrödinger equation numerically.

Integrating these oscillatory functions, however, can be tricky!
Errors accumulate.

For the crust of the Earth, say, the LHC to LNGS beam, approximating the density as constant can work quite well.

Let us consider a two-neutrino case:
%
\begin{align}
H = \frac{1}{2E} U \left[\begin{array}{cc}
m_1^2 & 0 \\
0 & m_2 ^2
\end{array}\right]
U ^\dag
+ \frac{1}{2E}
\left[\begin{array}{cc}
A & 0 \\
0 & 0
\end{array}\right]
= \frac{1}{2E} \widetilde{U} \widetilde{M}^2 \widetilde{U} ^\dag
\,,
\end{align}
%
so we find a corrected diagonal matrix \(\widetilde{M}^2\) as well as corrected mixing angles!
They read:
%
\begin{align}
\sin 2 \widetilde{\theta}_{12} &= \frac{\sin 2 \theta_{12} }{\sqrt{(\cos 2 \theta_{12} - A / \delta m^2)^2 + \sin^2 2 \theta_{12} }}  \\
\widetilde{\delta m}^2 &= \delta m^2 \frac{\sin 2 \theta_{12} }{\sin 2 \widetilde{\theta}_{12}}
\,,
\end{align}
%
which looks like a Breit-Wigner resonance.

The corrected \(\sin 2 \widetilde{\theta}_{12}\) has a maximum when \(\cos 2 \theta_{12} = A / \delta m^2 \), at which point we also have a minimum for \(\widetilde{\delta m}^2\).

This resonance does not happen for antineutrinos, which have a different sign for \(A\).

For the Sun, we make two approximations: the density is slowly changing, \(\dv*{N_e}{x} \) is nonzero but small, and there are many oscillations.

The transition is ``adiabatic'', so everything will only depend on the initial and final mixing angles, not on \(\delta m^2\).

The result is
%
\begin{align}
P_{ee}^{2 \nu } ( \text{solar}) \approx \cos^2 \widetilde{\theta}_i \cos^2 \theta + \sin^2 \widetilde{\theta}_i \sin^2 \theta
\,.
\end{align}

This probability has been tested!
It looks a bit like a decreasing ``sigmoid'', with a transition at few \SI{}{MeV}.

This oscillation probability is no longer octant symmetric!
It solves the degeneracy of the mixing angles in the KamLand experiment.

Uncertainties in the solar model do not really create problems in this regard; our models work well enough.

\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% This work is licensed under the Creative Commons Attribution 4.0 International %
% License. To view a copy of this license, visit                                 %
% http://creativecommons.org/licenses/by/4.0/.                                   %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\documentclass[11pt]{article}
\usepackage[cm]{fullpage}
%%AVC PACKAGES
\usepackage{avcgreek}
\usepackage{avcfonts}
\usepackage{avcmath}
\usepackage[numberby=section]{avcthm}
\usepackage{qcmacros}
\usepackage{goldstone}
%%MACROS FOR THIS DOCUMENT
\numberwithin{equation}{section}
\usepackage[
  margin=1.5cm,
  includefoot,
  footskip=30pt,
  headsep=0.2cm,headheight=1.3cm
]{geometry}
\usepackage{fancyhdr}
\pagestyle{fancy}
\fancyhf{}
\fancyhead[LE,RO]{Quiz 1, Handout 2: Second Quantization}
\fancyfoot[CE,CO]{\thepage}
\usepackage{url}

\begin{document}

\setlength{\abovedisplayskip}{3pt}
\setlength{\belowdisplayskip}{3pt}

\section{Second Quantization}

\begin{dfn}\label{dfn:slater-determinant}
\thmtitle{Slater determinant}
A \textit{Slater determinant} is a normalized antisymmetric product of spin-orbitals
\begin{align}\label{eq:slater-determinant-position-representation}
  \F_{(p_1\cd p_n)}(1,\ld,n)
=
  \tfr{1}{\sqrt{n!}}
  \sum_{\pi}^{\mr{S}_n}
  \e_{\pi}
  \y_{p_{\pi(1)}}(1)\cd\y_{p_{\pi(n)}}(n)
\end{align}
where $\pi\in\mr{S}_n$ is a permutation of $(1,\cd, n)$ with signature $\e_{\pi}$.\footnote{The signature of a permutation is $(-)^{\text{\# transpositions}}$.  See \url{https://en.wikipedia.org/wiki/Symmetric_group} for more on $\mr{S}_n$.}
\end{dfn}

\subsection{Deriving the second-quantized Hamiltonian from first quantization}\label{ssec:direct-derivation-of-second-quantization}

Let $\mc{F}_n$ denote the span of $n$-electron determinants and consider the integral operator $\op{a}_p:\mc{F}_n\rightarrow \mc{F}_{n-1}$ given by
\begin{align}
  (\op{a}_p\Y)(2,\cd,n)
\equiv
  \sqrt{n}\int d(1) \y_p^*(1)\Y(1,2,\cd,n)\,.
\end{align}
This operator acts on Slater determinants as follows.
\begin{align}
  (\op{a}_p\F_{(p_1\cd p_n)})(2,\cd,n)
=
\left\{
\ar{
  (-)^{k-1}\F_{(p_1\cd \cancel{p_k}\cd p_n)}(2,\ld,n) & p=p_k\in(p_1\cd p_n)\\[5pt]
  0 & \text{otherwise}
}
\right.
\end{align}
In words, it deletes $\y_p$ from $\F_{(p_1\cd p_n)}$ if present, otherwise killing the determinant.
The restriction to an antisymmetric space makes these operators anticommute, $\op{a}_p\op{a}_q=-\op{a}_q\op{a}_p$, since it can be shown that for $\Y\in\mc{F}_n$
\begin{align*}
  \int d(1)d(2)\y_p^*(1)\y_q^*(2)\Y(1,2,\cd,n)
=&\
-
  \int d(1)d(2)\y_q^*(1)\y_p^*(2)\Y(1,2,\cd,n)
\end{align*}
by swapping integration variables.
These operators can be used to generate the following decompositions.\footnote{These follow from substituting in the definition of $\op{a}_p$ and applying resolution of the identity to each argument.}
\begin{align}
  \Y(1,\cd,n)
=&\
  \tfr{1}{\sqrt{n}}
  \sum_p^\infty\y_p(1)\pr{\op{a}_p\Y}(2,\cd,n)
\\=&\
  \tfr{1}{\sqrt{n(n-1)}}
  \sum_{pq}^\infty
  \y_p(1)\y_q(2)(\op{a}_q\op{a}_p\Y)(3,\cd,n)
\end{align}
Therefore, matrix elements of the electronic Hamiltonian with respect to $\Y,\Y'\in \mc{F}_n$ can be expressed as
\begin{align*}
  \ip{\Y|\op{H}\Y'}
=
  \sum_{i=1}^n\ip{\Y|\op{h}(i)\Y'}
+
  \sum_{i<j}^n\ip{\Y|\op{g}(i,j)\Y'}
=&\
  n\ip{\Y|\op{h}(1)\Y'}
+
  \tfr{n(n-1)}{2}
  \ip{\Y|\op{g}(1,2)\Y'}
\\=&\
  \sum_{pq}^\infty
  h_{pq}\ip{\op{a}_p\Y|\op{a}_q\Y'}
+
  \tfr{1}{2}
  \sum_{pqrs}^\infty
  \ip{pq|rs}\ip{\op{a}_q\op{a}_p\Y|\op{a}_s\op{a}_r\Y'}
\end{align*}
in terms of the usual one- and two-electron integrals.
Since $\Y$ and $\Y'$ are arbitrary elements of $\mc{F}_n$, this implies
\begin{align}\label{eq:second-quantized-hamiltonian}
  \left.
  \op{H}
  \right|_{\mc{F}_n}
=
  \sum_{pq}^\infty
  h_{pq}
  \op{a}_p\dg \op{a}_q
+
  \tfr{1}{2}
  \sum_{pqrs}^\infty
  \ip{pq|rs}
  \op{a}_p\dg\op{a}_q\dg\op{a}_s\op{a}_r
\end{align}
which is the \textit{second quantized} form of the Hamiltonian, as opposed to the \textit{first quantized} form which is not restricted to antisymmetric functions.
A defining feature of the second quantization formalism is that $\op{H}$ is independent of the number of electrons, because \cref{eq:second-quantized-hamiltonian} holds for all $n$.

\subsection{Formal treatment of second quantization}

\begin{dfn}\label{dfn:direct-sum-and-direct-product}
\thmtitle{Direct sums and products}
\begin{samepage}
The \textit{direct sum}, $\oplus$, and \textit{direct product}\footnote{Also known as a \textit{tensor product}}, $\otimes$, are operations defining two different ways of combining vector spaces.
Each operation takes a vector from one space and a vector the other to form an ordered pair, but they behave differently under vector addition and scalar multiplication.
In a direct sum space $V\oplus V'\equiv\{v\oplus v'\,|\,v\in V,\,v'\in V'\}$,
vector addition and scalar multiplication are defined by
\begin{align}
  v_1\oplus v_1'
+
  v_2\oplus v_2'
=
  (v_1 + v_2)
\oplus
  (v_1' + v_2')
&&
  c(v\oplus v')
=
  cv\oplus cv'\,,
\end{align}
whereas, in a direct product space $V\otimes V'\equiv\{\sum v\otimes v'\,|\,v\in V,\,v'\in V'\}$, they are defined as follows.
\begin{align}
  v_1\otimes v'
+
  v_2\otimes v'
=
  (v_1 + v_2)\otimes v'
&&
  v\otimes v_1'
+
  v\otimes v_2'
=
  v\otimes(v_1' + v_2')
&&
  c(v\otimes v')
=
  (cv)\otimes v'
=
  v\otimes(cv')
\end{align}
Note that $\oplus$ behaves like addition and $\otimes$ behaves like multiplication.
If $\{e_i\}$ and $\{e_{i'}'\}$ are basis sets for $V$ and $V'$, respectively, then $\{e_i\oplus0'\}\cup\{0\oplus e_{i'}'\}$ is a basis for their direct sum and $\{e_i\otimes e_{i'}'\}$ is a basis for their direct product.
The dimension of the direct sum space is the sum of their dimensions,
$\dim V+\dim V'$,
and that of the direct product space is the product of their dimensions,
$\dim V\cdot \,\dim V'$.
Finally, if $\ip{\cdot|\cdot}_V$ and $\ip{\cdot|\cdot}_{V'}$ are inner products on $V$ and $V'$, then the following are inner products on the combined spaces.
\begin{align}
  \ip{v\oplus v'|w\oplus w'}_{V\oplus V'}
\equiv
  \ip{v|w}_V
+
  \ip{v'|w'}_{V'}
&&
  \ip{v\otimes v'|w\otimes w'}_{V\otimes V'}
\equiv
  \ip{v|w}_V
\cdot
  \ip{v'|w'}_{V'}
\end{align}
\end{samepage}
\end{dfn}

\begin{dfn}
\thmtitle{Hilbert space}
If $\mc{H}$ is a one-electron Hilbert space spanned by a set of spin-orbitals $\{\y_p\}$, then $\mc{H}^{\otimes n}=\mc{H}\otimes\cd\otimes\mc{H}=\spn\{\y_{p_1}{}\otimes\cd\otimes\y_{p_n}\}$ is an \textit{$n$-electron Hilbert space}.\footnote{These basis vectors are abstract representations spin-orbital product functions, $\ip{1\otimes\cd\otimes n|\y_{p_1}\otimes\cd\otimes\y_{p_n}}=\y_{p_1}(1)\cd\y_{p_n}(n)$, which are known as \textit{Hartree products}.}
\end{dfn}

\begin{dfn}\label{dfn:fock-space}
\thmtitle{Fock space}
Let $\mc{F}_n(\mc{H})$ denote $\spn\{\F_{(p_1\cd p_n)}\}$,\footnote{%
These basis vectors are Slater determinants, abstracted from position space:
$
  \F_{(p_1\cd p_n)}
=
  \fr{1}{\sqrt{n!}}
  \sum_{\pi\in\mr{S}_n}
  \y_{p_{\pi(1)}}\otimes\cd\otimes
  \y_{p_{\pi(n)}}
$.
Equation~\ref{eq:slater-determinant-position-representation} corresponds to
$
  \ip{1\otimes\cd\otimes n|\F_{(p_1\cd p_n)}}= \F_{(p_1\cd p_n)}(1,\ld,n)
$.
}
the antisymmetric subspace of $\mc{H}^{\otimes n}$.
\textit{Fock space} is the union of these spaces, $\mc{F}(\mc{H})=\mc{F}_0(\mc{H})\oplus \mc{F}_1(\mc{H})\oplus \mc{F}_2(\mc{H})\oplus\cd\oplus \mc{F}_{\infty}(\mc{H})$, comprising all possible electronic wavefunctions.
\end{dfn}

\begin{dfn}\label{occupation-number-representation}
\thmtitle{Occupation vectors}
In the \textit{occupation number formalism}, Fock space basis states are represented as \textit{occupation vectors}.
These are denoted by a series of bits,
$
  \kt{\bo{n}}
\equiv
  \kt{n_1,n_2,n_3,\cd,n_\infty}
$,
where $n_p=1$ when $\y_p$ is occupied and $n_p=0$ when it isn't.
The fully unoccupied state is called the \textit{vacuum}, denoted $\kt{\vac}$, which spans $\mc{F}_0(\mc{H})$.
\end{dfn}

\begin{dfn}\label{dfn:particle-hole-operators}
\thmtitle{Particle-hole operators}
\textit{Particle-hole operators} change the occupation numbers of one-particle states.
The \textit{annihilation operator} of $\y_p$ is a linear mapping $a_p:\mc{F}_n(\mc{H})\rightarrow \mc{F}_{n-1}(\mc{H})$ defined by
\begin{align}\label{eq:occ-num-annihilation-operator-action}
  a_p\kt{\cd n_p\cd}
=
  (-)^{n_1+\cd+n_{p-1}}
  \kt{\cd n_p-1\cd}
\ \ \ \text{if $n_p=1$}
&&
  a_p\kt{\cd n_p\cd}
=
  0
\ \ \ \text{if $n_p=0$}
\end{align}
and the \textit{creation operator} of $\y_p$ is a linear mapping $c_p:\mc{F}_n(\mc{H})\rightarrow \mc{F}_{n+1}(\mc{H})$ defined by
\begin{align}\label{eq:occ-num-creation-operator-action}
  c_p\kt{\cd n_p\cd}
=
  (-)^{n_1+\cd+n_{p-1}}
  \kt{\cd n_p+1\cd}
\ \ \ \text{if $n_p=0$\ }
&&
  c_p\kt{\cd n_p\cd}
=
  0
\ \ \ \text{if $n_p=1$.}
\end{align}
\end{dfn}

\begin{prop}
\thmtitle{$c_p=a_p\dg$}
\thmstatement{Creation and annihilation operators of the same state $\y_p$ are adjoints of each other.}
\thmproof{
  $\ip{n_1'n_2'\cd|a_p[n_1n_2\cd]}$ vanishes unless $n_p'=0$, $n_p=1$, and $n_q'=n_q\ \forall q\neq p$.
  Likewise for $\ip{c_p[n_1'n_2'\cd]|n_1n_2\cd}$.
  Therefore, $\ip{\Y|a_p\Y'}=\ip{c_p\Y|\Y'}$ for all $\Y,\Y'\in \mc{F}(\mc{H})$ and $c_p=a_p\dg$ by the definition of adjoint.
}
\end{prop}

\begin{prop}\label{prop:particle-hole-operator-anticommutator}
\thmtitle{$[q,q']_+=\d_{q'q\dg}$}
\thmstatement{Particle-hole operators $q$ and $q'$ anticommute unless $q'=q\dg$, for which $[q,q\dg]_+=1$.}~\footnote{These are anticommutator brackets, $[q, q']_+ \equiv qq' + q'q$.}
\thmproof{
  Let $q$ and $q'$ be arbitrary particle-hole operators acting on $\y_p$ and $\y_{p'}$, respectively.
  First, suppose $p\neq p'$. Then
  \begin{align*}
  &
    qq'\kt{\cd n_p\cd n_{p'}\cd}
  =
    (-)^{n_p+\sum_{r=p+1}^{p'}n_r}
    \kt{\cd\ol{n_p}\cd\ol{n_{p'}}\cd}
  \,\text{, and}
  \\
  &
    q'q\kt{\cd n_p\cd n_{p'}\cd}
  =
    (-)^{\ol{n_p}+\sum_{r=p+1}^{p'}n_r}
    \kt{\cd\ol{n_p}\cd\ol{n_{p'}}\cd}
  \end{align*}
  where $\ol{n_p}$ and $\ol{n_{p'}}$ are the occupations after applying $q$ and $q'$.
  Since $n_p$ and $\ol{n_p}$ differ by one, $qq'=-q'q$.
  The second case, $p=p'$, implies $q'\in\{q,q\dg\}$.
  If $q'=q$, then $qq'=-q'q=0$.
  If $q'=q\dg$, either $n_p=1\implies(a_p\dg a_p + a_pa_p\dg)\kt{\cd n_p\cd}=(1+0)\kt{\cd n_p\cd}$ or $n_p=0\implies(a_p\dg a_p + a_pa_p\dg)\kt{\cd n_p\cd}=(0+1)\kt{\cd n_p\cd}$.
  Either way, $q'=q\dg\implies(qq' + q'q)=1$.
}
\end{prop}

\begin{rmk}
\thmtitle{Relating the determinant and occupation number formalisms}
When $p_1<\cd<p_n$, $\F_{(p_1\cd p_n)}$ is equivalent to the occupation vector $\kt{\bo{n}_{(p_1\cd p_n)}}$ with ones at $p_1,\cd,p_n$.
Otherwise, this determinant is equivalent to $\e_{\pi}\kt{\bo{n}_{(p_1\cd p_n)}}$ for $\pi\in\mr{S}_n$ such that $p_{\pi(1)}<\cd<p_{\pi(n)}$.
The actions of $a_p$ and $a_p\dg$ on $\F_{(p_1\cd p_n)}$ are given by
\begin{align}\label{eq:abstract-annihilation-operator-action}
  a_p\F_{(p_1\cd p_n)}
=
  (-)^{k-1}\F_{(p_1\cd\cancel{p_k}\cd p_n)}
  \ \text{if $p=p_k\in(p_1\cd p_n)$}
&&
  a_p\F_{(p_1\cd p_n)}
=
  0
  \ \text{if $p\notin(p_1\cd p_n)$}
\\\label{eq:abstract-creation-operator-action}
  a_p\dg\F_{(p_1\cd p_n)}
=
  (-)^{k-1}\F_{(p_1\cd p_{k-1}pp_k\cd p_n)}
  \ \text{if $p\notin(p_1\cd p_n)$}
&&
  a_p\dg\F_{(p_1\cd p_n)}
=
  0
  \ \text{if $p\in(p_1\cd p_n)$}
\end{align}
which follows directly from \Cref{eq:occ-num-annihilation-operator-action,eq:occ-num-creation-operator-action} when $p_1<\cd<p_n$.
Other cases follow from the fact that any sign factors for permuting $(p_1\cd p_n)$ cancel on both sides of the equation, including the position of insertion or deletion, $p_k$, whose phase is tracked by $(-)^{k-1}$ on the right.
That is, both sides of the equation are antisymmetric to permutations of $(1\cd n)$.
Note that \Cref{eq:abstract-annihilation-operator-action} was also derived in \Cref{ssec:direct-derivation-of-second-quantization} using the position-space representation of $a_p$.
One advantage of the determinant basis is that, unlike occupation vectors, determinants translate directly into strings of creations operators
\begin{align}
  \kt{\F_{(p_1\cd p_n)}}
=
  a_{p_1}\dg\cd a_{p_n}\dg\kt{\vac}
\end{align}
without any phase ambiguity.
Together with the second quantized form of the electronic Hamiltonian, this boils much of the grunt work of electronic structure theory down to particle-hole operator algebra.
\end{rmk}

\begin{dfn}
\thmtitle{Excitation operators and excited determinants}
Operator strings of the form $a_{p_1}\dg\cd a_{p_m}\dg a_{q_m}\cd a_{q_1}$ are called \textit{excitation operators}.
For a given reference determinant $\F$, excited determinants can be constructed as
\begin{align}
  \F_{i_1\cd i_m}^{a_1\cd a_m}
=
  a_{a_1}\dg\cd a_{a_m}\dg a_{i_m}\cd a_{i_1}\F
=
  a_{a_1}\dg a_{i_1}\cd a_{a_m}\dg a_{i_m}\F
\end{align}
where $i_1,\cd,i_m$ are occupied and $a_1,\cd,a_m$ are virtual indices with respect to $\F$.
\end{dfn}

\begin{dfn}\label{dfn:particle-hole-isomorphism}
\thmtitle{Particle-hole isomorphism}
The \textit{particle-hole isomorphism} with respect to the reference determinant $\kt{\F}=\kt{\underset{\text{$n$ times}}{1\cd1}\,0\,0\cd}$ is a mapping $F(\mc{H})\rightarrow F(\mc{H})$ that inverts the bits occupied in $\F$:
\begin{align*}
  \kt{k_1\cd k_nk_{n+1}k_{n+2}\cd}
\mapsto
  \kt{\ol{k}_1\cd \ol{k}_nk_{n+1}k_{n+2}\cd}
\sp
  \text{where $\ol{k}_i=1-k_i$.}
\end{align*}
Physically, this corresponds to shift in perspective from the \textit{particle frame} to a \textit{quasiparticle frame}, where the first $n$ states are viewed as \textit{holes} rather than \textit{particles}.
This makes $\kt{\vac}\mapsto\kt{\underset{\text{$n$ times}}{\ol{1}\cd\ol{1}}\,0\,0\cd}$ a state of $n$ holes and no particles.  $\kt{\F}\mapsto\kt{\underset{\text{$n$ times}}{\ol{0}\cd\ol{0}}\,0\,0\cd}$ becomes the \textit{quasiparticle vacuum state}, in which all hole and particle states are unoccupied.
\end{dfn}

\begin{dfn}
\thmtitle{Quasiparticle creation and annihilation operators}
If we apply \Cref{dfn:particle-hole-operators} to the new quasiparticle Fock space, we end up with a new system of \textit{quasi-particle-hole operators} $\{b_p\}\cup\{b_p\dg\}$, related to the old set via
\begin{align}
&&
  a_i
\mapsto
  b_i\dg
&&
  a_i\dg
\mapsto
  b_i
&&
  a_a
\mapsto
  b_a
&&
  a_a\dg
\mapsto
  b_a\dg
\end{align}
where $i$ and $a$ are occupied and virtual indices with respect to the reference determinant $\F$.
$\{a_i\dg\}\cup\{a_a\}\mapsto\{b_p\}$ are therefore \textit{quasiparticle annihilation operators} and $\{a_i\}\cup\{a_a\dg\}\mapsto\{b_p\dg\}$ are \textit{quasiparticle creation operators}.
\end{dfn}

\begin{rmk}
The standard expression for the second-quantized Hamiltonian is
\begin{align}
  H
=
  \sum_{pq}
  h_{pq}\,
  a_p\dg a_q
+
  \tfr{1}{4}
  \sum_{pqrs}
  \ip{pq||rs}\,
  a_p\dg a_q\dg a_sa_r
\end{align}
where the summations run over the full set of spin-orbitals.
Note that this matches \Cref{eq:second-quantized-hamiltonian}, except that we have rearranged the second term to express it in terms of antisymmetrized integrals.
In terms of quasi-particle-hole operators,
\begin{align}
\nonumber
  H
=&\
  \sum_{ab}
  h_{ab}\,
  b_a\dg b_b
+
  \sum_{ai}
  h_{ai}\,
  b_a\dg b_i\dg
+
  \sum_{ia}
  h_{ia}\,
  b_i b_a
+
  \sum_{ij}
  h_{ij}
  b_i b_j\dg
\\\nonumber&\
+
  \tfr{1}{4}
  \sum_{abcd}
  \ip{ab||cd}\,
  b_a\dg b_b\dg b_d b_c
+
  \tfr{1}{2}
  \sum_{abci}
  \ip{ab||ci}\,
  b_a\dg b_b\dg b_i\dg b_c
+
  \tfr{1}{2}
  \sum_{aibc}
  \ip{ai||bc}\,
  b_a\dg b_i b_c b_b
+
  \tfr{1}{4}
  \sum_{abij}
  \ip{ab||ij}\,
  b_a\dg b_b\dg b_j\dg b_i\dg
+
  \sum_{aibj}
  \ip{ai||bj}\,
  b_a\dg b_i b_j\dg b_b
\\&\
+
  \tfr{1}{4}
  \sum_{ijab}
  \ip{ij||ab}\,
  b_i b_j b_b b_a
+
  \tfr{1}{2}
  \sum_{iajk}
  \ip{ia||jk}\,
  b_i b_a\dg b_k\dg b_j\dg
+
  \tfr{1}{2}
  \sum_{ijka}
  \ip{ij||ka}\,
  b_i b_j b_a b_k\dg
+
  \tfr{1}{4}
  \sum_{ijkl}
  \ip{ij||kl}\,
  b_i b_j b_l\dg b_k\dg
\end{align}
where we have split the full summations above into summations over occupied and virtual orbitals and grouped like terms.
\end{rmk}

\begin{dfn}
\thmtitle{Normal order}
A string $q_1\cd q_n$ of particle-hole operators is in \textit{normal order} when all of its creation operators sit to the left of its annihilation operators.
That is, when the string has the form $a_{p_1}\dg\cd a_{p_m}\dg a_{r_1}\cd a_{r_{m'}}$.
This guarantees that its vacuum expectation value vanishes, $\ip{\vac|q_1\cd q_n|\vac}=0$.
More generally, we say that $q_1\cd q_n$ is in \textit{$\F$-normal order} if it maps into a string of the form $b_{p_1}\dg\cd b_{p_m}\dg b_{r_1}\cd b_{r_{m'}}$ under particle-hole isomorphism referenced to $\F$, since this guarantees that
$\ip{\F|q_1\cd q_n|\F}=0$.
\end{dfn}

\begin{ex}
In second quantization, any operator string can be expanded as a linear combination of strings which are in normal order.
The expectation value of a string is always equal to the constant term in this expansion.
For example:
\begin{align*}
  a_p a_q\dg
=
-
  a_q\dg a_p
+
  \d_{pq}
&\implies
  \ip{\vac|a_p a_q\dg|\vac}
=
  \d_{pq}
\\
  a_p a_q a_s\dg a_r\dg
=
  a_r\dg a_s\dg a_q a_p
+
  \d_{ps}
  a_r\dg a_q
-
  \d_{pr}
  a_s\dg a_q
-
  \d_{qs}
  a_r\dg a_p
+
  \d_{qr}
  a_s\dg a_p
-
  \d_{ps}\d_{qr}
+
  \d_{pr}\d_{qs}
&\implies
  \ip{\vac|a_p a_q a_s\dg a_r\dg|\vac}
=
  \d_{pr}\d_{qs}
-
  \d_{ps}\d_{qr}
\end{align*}
where we have made repeated use of \Cref{prop:particle-hole-operator-anticommutator} to arrive at these expansions.
This strategy becomes unwieldy for expectation values of a reference determinant $\F$, in which case it is more convenient to use particle-hole isomorphism.
For example, consider the following matrix element of the core Hamiltonian.
\begin{align*}
  \sum_{pq}
  h_{pq}
  \ip{\F|a_p\dg a_q|\F_i^a}
=
  \sum_{bc}
  h_{bc}\,
  \cancel{\ip{\F|b_b\dg b_c b_a\dg b_i\dg|\F}}
+
  \sum_{bj}
  h_{bj}\,
  \cancel{\ip{\F|b_b\dg b_j\dg b_a\dg b_i\dg|\F}}
+
  \sum_{jb}
  h_{jb}\,
  \cancelto{\d_{ab}\d_{ij}}{\ip{\F|b_j b_b b_a\dg b_i\dg|\F}}
+
  \sum_{jk}
  h_{jk}
  \cancel{\ip{\F|b_j b_k\dg b_a\dg b_i\dg|\F}}
=
  h_{ia}
\end{align*}
Only the third term survives, because the others generate a ket state with a different number of quasi-particles from the bra state.
\end{ex}

\end{document}


\documentclass[twocolumn]{article}

\usepackage{abstract}
\usepackage{algorithm}
\usepackage{appendix}
\usepackage{amsmath}
\usepackage{amsfonts} % \mathbb
\usepackage[margin=0.6in]{geometry}
\usepackage{graphicx}
\usepackage{hyperref}
\usepackage{mathtools} % \multlined
\usepackage{multirow}
\usepackage{siunitx}  % use \si{\angstrom} for Angstrom

\usepackage{subfig}
\usepackage{xspace}

\newcommand*\samethanks[1][\value{footnote}]{\footnotemark[#1]}
\newcommand{\pygbe}{\texttt{PyGBe}\xspace}
\newcommand{\gmres}{\textsc{gmres}\xspace}
\newcommand{\bem}{\textsc{bem}\xspace}
\newcommand{\fmm}{\textsc{fmm}\xspace}
\newcommand{\kifmm}{\textsc{kifmm}\xspace}
\newcommand{\ncrit}{n_{\mathrm{crit}}}  % number of particles per leaf
\newcommand{\ses}{\textsc{ses}\xspace}
\newcommand{\msms}{\texttt{\textsc{msms}}\xspace}
\newcommand{\ie}{\textit{i}.\textit{e}., }

\graphicspath{{figs/}}

\title{High-productivity, high-performance workflow for virus-scale electrostatic simulations with Bempp-Exafmm}

\author{%
    Tingyu Wang\thanks{Department of Mechanical and Aerospace Engineering, The George Washington University, Washington, DC, USA}%
    \and Christopher D. Cooper\thanks{Department of Mechanical Engineering and Centro Cient\'ifico Tecnol\'ogico de Valpara\'iso, Universidad T\'ecnica Federico Santa Mar\'ia, Valpara\'iso, Chile}%
    \and Timo Betcke\thanks{Department of Mathematics, University College London, UK}%
    \and Lorena A. Barba\samethanks[1]%
}

\date{}

\begin{document}

\twocolumn[
\maketitle

%% abstract
\begin{onecolabstract}
Biomolecular electrostatics is key in protein function and the chemical processes affecting it.
Implicit-solvent models via the Poisson-Boltzmann (PB) equation provide insights with less computational cost than atomistic models, making large-system studies---at the scale of viruses---accessible to more researchers.
    Here we present a high-productivity and high-performance PB solver based on Exafmm, a fast multipole method library, and Bempp, a Galerkin boundary element method package.
    The workflow integrates an easy-to-use Python interface with optimized computational kernels, and
    can be run interactively via Jupyter notebooks, for faster prototyping.
    Our results show the capability of the software, confirm code correctness, and assess performance with between 8,000 and 2 million elements.
    Showcasing the power of this interactive computing platform, we study the conditioning of two variants of the boundary integral formulation with just a few lines of code.
    Mesh-refinement studies confirm convergence as $1/N$, for $N$ boundary elements, and
    a comparison with results from the trusted APBS code using various proteins shows agreement.
    Performance results include timings, breakdowns, and computational complexity.
    Exafmm offers evaluation speeds of just a few seconds for tens of millions of points, and $\mathcal{O}(N)$ scaling.
    Computing the solvation free energy of a Zika virus, represented by 1.6 million atoms and 10 million boundary elements, took 80-min runtime on a single compute node (dual 20-core).

\end{onecolabstract}
]
%% keyword
%\begin{keyword}
%    boundary integral equation \sep boundary element method \sep Galerkin method \sep fast multipole method \sep
%    Python \sep biomolecular electrostatics \sep implicit solvent \sep Poisson-Boltzmann \sep solvation free energy
%\end{keyword}

% body of paper
\section{Introduction}\label{sec:intro}
\input{introduction}

\section{Results}\label{sec:results}
\input{results}

\section{Discussion} \label{sec:discussion}
\input{discussion}

\small{
\section{Methods}\label{sec:methods}
\input{methods_formulation}
\input{methods_bempp}
\input{methods_exafmm}
}

\section{Data availability}
We deposited the meshes and \texttt{pqr} files on the Zenodo service: \href{http://doi.org/10.5281/zenodo.4568768}{doi:10.5281/zenodo.4568768}.
The raw and secondary data for all results are available in the archival deposit of our paper’s GitHub repository: \href{http://doi.org/10.5281/zenodo.4568951}{doi:10.5281/zenodo.4568951}.

\section{Code availability}
Exafmm is available at \href{https://github.com/exafmm/exafmm-t}{https://github.com/exafmm/exafmm-t} under the BSD 3 license.
Bempp-cl is available at \href{https://github.com/bempp/bempp-cl}{https://github.com/bempp/bempp-cl} under the MIT license.
The scripts for plotting and rerunning our experiments are available in the archival deposit of our paper’s GitHub repository: \href{http://doi.org/10.5281/zenodo.4568951}{doi:10.5281/zenodo.4568951}.

\bibliography{./reference}{}
\bibliographystyle{elsarticle-num}

\section*{Acknowledgments}
We thank Dr. Sergio Pantano for providing us with the parameterized structure of the Zika virus capsid.
CDC acknowledges support by ANID (Agencia Nacional de Investigaci\'{o}n y Desarrollo) through PIA/APOYO AFB180002.
TB was supported by Engineering and Physical Sciences Research Council Grant EP/V001531/1.
LAB acknowledges funding from the National Science Foundation via award \#1747669.

\section*{Author contributions}
LAB and TB conceived this project.
TW wrote the version of the Exafmm code used in this work, the Python bindings and the Bempp integration.
TB gave technical support on Bempp usage and wrote code to aid the integration.
CDC gave conceptual advice and helped set up computational experiments.
TW ran the calculations and prepared the figures.
TW, CDC, TB, and LAB discussed and guided the conduct of the research, and interpreted the results.
TW wrote the first draft of the manuscript, and all authors contributed materially to the writing and revising.
LAB guided the data management and guarantees the preservation of the full research compendium for this work.
All authors confirm that the figures and conclusions accurately reflect the research.

\section*{Competing interests}
The authors declare no competing interests.

\appendix
\input{appendix}

\end{document}


\chapter{BDS test for independence}

\section{Name}

bdstest --- BDS test for independence

\section{Synopsis}

{\small \begin{verbatim}
#include "shg/bdstest.h"
using namespace SHG;
class BDS_test {
public:
     struct Result {
          double stat;
          double pval;
     };
     BDS_test(const std::vector<double>& u,
              int maxm,
              const std::vector<double>& eps);
     BDS_test(const std::vector<double>& u);
     inline int maxm() const;
     inline const std::vector<double>& eps() const;
     inline const std::vector<std::vector<Result>>& res() const;
private:
     /* ... */
};

std::ostream& operator<<(std::ostream& stream, const BDS_test& b);
\end{verbatim}}

\section{Description}

The class performs the BDS test for independence~\cite
{brock-dechert-scheinkman-lebaron-1996}.

Let $(u_t)_{t = 1}^n$ be a time series. The BDS test statistic for an
embedding dimension $m \geq 1$ and a threshold $\epsilon > 0$ is
defined as
\begin{equation} \label{eq:statistic}
  W(m, \epsilon) = \sqrt{n} \, \frac{C_{m, n}(\epsilon) - [C_{1,
        n}(\epsilon)]^m} {V_{m, n}(\epsilon)},
\end{equation}
where\footnote{$\chi_{\epsilon}$ is the characteristic function of the
  interval $[0, \epsilon)$.}
\begin{equation} \label{eq:Cmn}
  C_{m, n}(\epsilon) = \frac{2}{(n - m + 1)(n - m)} \sum_{1 \leq s < t
    \leq n - m + 1} \chi_{\epsilon} \left( \max_{0 \leq i < m} |u_{s +
    i} - u_{t + i}| \right)
\end{equation}
is the correlation integral\footnote{Cf.~\cite [p.~120]
  {baker-gollub-1998}.} and
\begin{equation} \label{eq:bds2.11}
  \begin{split}
  \frac{1}{4}V_{m, n}^2(\epsilon) &= m(m - 2) C^{2m - 2} (K - C^2) +
  K^m - C^{2m} + \\
  & \quad + 2 \sum_{j = 1}^{m - 1} \left[ C^{2j} (K^{m - j} - C^{2m -
      2j}) - mC^{2m -2} (K - C^2) \right],
  \end{split}
\end{equation}
where
\begin{align} \label{eq:bds2.12}
  C &= C_n(\epsilon) = \frac{1}{n^2} \sum_{s = 1}^n \sum_{t = 1}^n
  \chi_{\epsilon}(|u_s - u_t|), \\
  \label{eq:bds2.13}
  K &= K_n(\epsilon) = \frac{1}{n^3} \sum_{r = 1}^n \sum_{s = 1}^n
  \sum_{t = 1}^n \chi_{\epsilon}(|u_r - u_s|) \chi_{\epsilon}(|u_s -
  u_t|).
\end{align}
If $(u_t)_{t = 1}^n$ is a series of independent identically
distributed random variables, then for $m \geq 2$ the
statistic~(\ref{eq:statistic}) converges in distribution to the
standard normal distribution.

The first constructor requires the time series $(u_t)_{t = 1}^{n}$,
arranged in a vector with index running from 0 to $n - 1$, the maximum
embedding dimension $\mathit{maxm}$ and the vector of thresholds
$\mathit{eps}$. The second constructor requires only the time series
and arbitrarily sets $\mathit{maxm} = 8$ and \[ \mathit{eps}
= \begin{bmatrix} 0.5s & 0.75s & s & 1.25s & 1.5s & 1.75s &
  2s \end{bmatrix}, \] where
\begin{equation} \label{eq:mean_var}
  s^2 = \frac{1}{n} \sum_{t = 1}^n (u_t - \bar{u})^2, \quad
  \bar{u} = \frac{1}{n} \sum_{t = 1}^n u_t.
\end{equation}

After successful construction, the function \verb|res()| returns an
array of structures of type \verb|BDS_test::Result|, whose member
\verb|res()[m][i].stat| reports the value of the statistic $W(m,
\epsilon)$ defined by~(\ref{eq:statistic}) for the embedding dimension
$2 \leq m \leq \mathit{maxm}$ and the $i$-th threshold in
$\mathit{eps}$. The member \verb|res()[m][i].pval| reports the
probability
\[ \left\{ \begin{array}{ll}
  \Phi(W(m, \epsilon)) & \mbox{if $W(m, \epsilon) < 0$,} \\
  1 - \Phi(W(m, \epsilon)) & \mbox{if $W(m, \epsilon) \geq 0$,}
\end{array} \right. \]
where $\Phi$ is the cumulative distribution function of the standard
normal distribution. The values of \verb|res()[m][i]| are undefined
for $m = 0, 1$.

The functions \verb|maxm()| and \verb|eps()| return $\mathit{maxm}$
and the vector $\mathit{eps}$ used during construction, respectively.

The operator \verb|<<| outputs the four-column plain table of results
with $\mathit{\epsilon}$, $m$, $W(m, \epsilon)$ and the p-value on
each row.

\section{Implementation}

In the implementation,~(\ref{eq:bds2.11}) is simplified to
\begin{equation} \label{eq:simpleV}
  \frac{1}{4}V_{m, n}^2(\epsilon) = K^m + (m - 1)^2 C^{2m} - m^2 K
  C^{2m -2} + 2 \sum_{j = 1}^{m - 1} C^{2j} K^{m - j}
\end{equation}
and~(\ref{eq:bds2.13}) is simplified to
\begin{equation} \label{eq:bds2.13s}
  K = \frac{1}{n^3} \sum_{s = 1}^n \left[ \sum_{t = 1}^n
    \chi_{\epsilon}(|u_s - u_t|) \right]^2.
\end{equation}

\section{Errors}

The constructors can throw \verb|std::invalid_argument| if $n < 1$ or
$n$ is too big or $\mathit{maxm} < 2$ or $\mathit{maxm} \geq n$. They
can also throw \verb|std::range_error| if due to rounding errors
$V_{m, n}^2(\epsilon) / n$, required to
calculate~(\ref{eq:statistic}), is negative or too small or the
variance in~(\ref{eq:mean_var}) is negative or too small.

\section{Example}

The following program:

{\footnotesize \begin{verbatim}
#include <iostream>
#include <iomanip>
#include "shg/bdstest.h"

/** Borosh-Niederreiter random number generator. See Donald E. Knuth,
    Sztuka programowania. Tom 2. Algorytmy seminumeryczne, WNT,
    Warszawa 2002, p. 113. */
double bn() {
     static unsigned long int x = 1ul;
     x = (1812433253ul * x) & 0xfffffffful;
     return x / 4294967296.0;
}

using namespace std;
using SHG::BDS_test;

int main() {
     vector<double> u(1000);
     cout << fixed << setprecision(5);

     for (vector<double>::size_type i = 0; i < u.size(); i++)
          u[i] = bn();
     /** eps is the standard deviation of U(0, 1). */
     cout << BDS_test(u, 8, {sqrt(1.0 / 12.0)}) << '\n';

     for (vector<double>::size_type i = 0; i < u.size(); i++)
          u[i] = i % 2 ? 2.0 * bn() : bn();
     /** eps is the standard deviation of the mixture of U(0, 1) and
         U(0, 2) with mixing weights 0.5. */
     cout << BDS_test(u, 8, {sqrt(13.0 / 48.0)});
}
\end{verbatim}}
\noindent produces:
{\footnotesize \begin{verbatim}
0.28868 2 0.27392 0.39207
0.28868 3 0.26732 0.39461
0.28868 4 -0.33474 0.36891
0.28868 5 -0.97089 0.16580
0.28868 6 -1.83736 0.03308
0.28868 7 -2.35252 0.00932
0.28868 8 -2.16494 0.01520

0.52042 2 -3.96242 0.00004
0.52042 3 0.39043 0.34811
0.52042 4 -0.07102 0.47169
0.52042 5 1.30413 0.09609
0.52042 6 1.26937 0.10215
0.52042 7 2.17663 0.01475
0.52042 8 2.04631 0.02036
\end{verbatim}}


\section{Limits of spectral clustering}
\label{ch:ulrike2004}

\textit{Limits of spectral clustering} by Ulrike Von Luxburg. \\
Cited by 102. \textit{Advances in neural information processing systems}.
\newline

\textbf{Main point} is that \begin{inparaenum}[\itshape a\upshape)]
\item normalized spectral clustering is superior than unnormalized spectral clustering.
\end{inparaenum}


\subsection{Realizing the analog sorter}
\input{realizing}

\subsection{Running the simulation}
In order to explore how analog sorting works in larger systems, we use an ODE solver built on the odeint solver in the Python SciPy library~\cite{scipy}.
We first construct an array of variables, an array of the ODEs from (5), and an array of initial values, which we supply to SciPy's odeint. The initial values for $x$ are randomly chosen from a dynamic range using randint from Python's random module. The initial $y$ values are set at 0.9. SciPy's odeint returns the values of $x$ and $y$ at each specified time interval of 0.04 seconds.

The data is used to plot graphs, but also to determine when the $x$ values have settled in at their final value.

% The code here
% https://github.com/mandli/intro-numerical-methods/blob/master/09_ODE_ivp.ipynb
% Would be a good starting point for the simulation.

% https://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.qr.html
% http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.ode.html#scipy.integrate.ode
% http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.odeint.html


\documentclass{article}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{mathrsfs}
\usepackage{amssymb} % for "\mathbb" macro
\usepackage[round]{natbib}
\usepackage{url}
\usepackage{hyperref}
\usepackage[toc,page]{appendix}

\title{Three Factor Seasonal Commodity Price Process}
\author{Jake C. Fowler}
\date{December 2020}

\begin{document}
\newcommand{\+}[1]{\ensuremath{\mathbf{#1}}}

\maketitle

WARNING: THIS DOCUMENT IS CURRENTLY WORK IN PROGRESS

\tableofcontents

\newpage

\section{Introduction}
This paper presents a specific set for parameters for the multi-factor model presented in \cite{Fowler}
such that the model should have similar statistical properties to the three-factor spot price
model presented in \cite{Boogert} the model used in the commercial KyStore gas storage valuation model.

\section{Forward Price SDE}
The starting point is the SDE (stochastic differential equation) for the forward
price process:

\begin{align}
    \label{eq:forward_sde}
    \frac{dF(t, T)^l}{F(t, T)^l}=\sum_{i=1}^{n^l} \sigma_i^l(T)e^{-\alpha_i^l(T-t)}dz_i^l(t) \\
    \nonumber
    \alpha_i^l \in \mathbb{R}_{\ge 0} \\
    \nonumber
    t \in \mathbb{R}_{\ge 0} \\
    \nonumber
    T \in \{ T_0^l, T_1^l, T_2^l, \hdots | T_j^l \ge t  \} \\
    \nonumber
    \sigma_i^l : \mathbb{R}_{\ge 0} \rightarrow \mathbb{R} \\
    \nonumber
    l \in [1, m]
\end{align}

Where $z_i^l(t)$ follow correlated Wiener processes with correlation $\rho_{i, j}^{x, y}$, i.e.

\begin{equation}
    \mathbb{E}[dz_i^x(t)dz_j^y(t)] = \rho_{i, j}^{x, y}dt
\end{equation}

$F(t, T_j)^l$ is the forward price observed at time $t$, for delivery over the time
interval $[T_j, T_{j+1})$ of the $l^{\text{th}}$ of $m$ commodity underlyings.

\section{Three Factor Seasonal Form}
The three factor seasonal parameters clearly has parameter $n=3$. The first factor is
the spot or short-term factor. For this factor there is a constant volatity and mean
reversion, i.e. $\sigma_1(T) = \sigma_{spot}$ and as this is the only factor with
non-zero mean reversion $\alpha_1 = \alpha$. This factor has the heuristic interpretation
as random shocks which move the forward curve in an exponentially decaying (as function of
time to maturity) manner, with the biggest effect being on the spot price. It can also % TODO reference clewlow strickland for SDE of spot price
be interpreted as driving random shocks to the spot price which then mean revert.

As mentioned above the remaining two factors have zero mean reversion, i.e.
$\alpha_2 = \alpha_3 = 0$. TODO seasonal factor

The third factor, the long-term factor, has constant volatility, $\sigma_2 = \sigma_{long}$,
and represents parallel movement which effect the whole forward curve in a maturity
independent manner.

Finally, it is assumed that the three Brownian Motions are independent. Putting this
together, and using the subscripts $spot$, $seas$ and $long$ the Brownian Motions
of the spot, seasonal, and long-term factors we get the following.

\begin{equation}
    \frac{dF(t, T)}{F(t, T)}= \sigma_{spot} e^{-\alpha(T - t)} dz_{spot}(t) + \\
        \sigma_{seas}(T) dz_{seas}(t) + \sigma_{long} dz_{long}(t)
\end{equation}

Refer back to \cite{Fowler} for the statistical properties of the forward and spot
price assumed by just substituting the parameters in \cite{Fowler} with the specific
one given above.

\section{Critique of Three Factor Seasonal Model}
The strength of the three-factor seasonal model is it's parsimony.
Being able to specify the gas price dynamics using only four parameters is of great help
in allowing users to intuitively see what is driving the extrinsic value of gas storage
being valued. Traders can easily adjust the input
parameters based on their view of the market. For example if a trader take a view that
the future summer-winter spread volatility is going to be higher than in the historical
period used to calibrate the parameters they can easily bump up the seasonal volatility
parameter when valuing a potential storage deal. For a non-parsimonious model with many parameters
some of which are completely abstract (for example correlation between factors), such
usage would not be practical.

Another example of the is that risk managers can
create scenario matrices containing storage facility (or portfolio) P\&L based on scenarios
applied to any of the four parameters.

% Quote Taleb and compare to Black-Scholes as a parameterisation?

It also allows for a relatively simple and intuitive way of calibrating
model parameters from historic spot and forward prices. This is of particular importance
for gas storage where there is generally a lack of liquid traded instruments which can be used
to calibrate model volatility and correlation parameters to
\footnote[1]{For many natural gas markets, there are European options traded, for which the
implied volatility can be used to calibrate price dynamics. However, the extrinsic value
of storage is mostly derived from the relative movement of different delivery points
on the forward curve, i.e. by calendar spreads. The European option volatility curve
does convey some information about calendar spread volatility (e.g. a big difference
in implied vol for two forward contracts implies that one contract will move by much
more than another, hence higher calendar spread volatility) but not enough to fully
calibrate the joint price dynamics of the whole forward curve.}.

% TODO cite other Kyos paper
The KyStore product is clearly a popular one and The Author believes that this is at least partially due
to the parsimony of the underlying price process model.

\bigskip
The big downside of this model is that the forward volatility seasonality structure is unrealistic.
It is well known that, adjusting for time-to-maturity effects, the volatility of winter periods
will generally be higher than those in summer. Figure \ref{fig:seasonal_vol} plots the forward
volatility implied by the model, with mean reversion set to zero in order to remove any t
time-to-maturity effect. This chart shows that the model implies two volatility peaks a
year, once in February, as expected, but the other around August.

\begin{figure}
    \includegraphics{vol_seasonality.png}
    \caption{Forward Volatility By Delivery Date}
    \label{fig:seasonal_vol}
\end{figure}

\bibliographystyle{plainnat}
\bibliography{three_factor_seasonal_model}

\end{document}