%% Journal of Open Research Software Latex template -- Created By Stephen Bonner and John Brennan, Durham University, UK.

\documentclass{jors}
\usepackage{amsmath, amssymb}
\usepackage{cite}
\usepackage{graphicx, subcaption}
\graphicspath{ {images/} }

%% Set the header information
\pagestyle{fancy}
\definecolor{mygray}{gray}{0.6}
\renewcommand\headrule{}
\rhead{\footnotesize 3}
\rhead{\textcolor{gray}{UP JORS software Latex paper template version 0.1}}

\newcommand{\supervisordisagreement}[2]{\textcolor{green}{\texttt{{Supervisor disagreement!!! <<<<<<< Lincoln}}} \textcolor{blue}{#1} \textcolor{green}{\texttt{=======}} \textcolor{red}{#2} \textcolor{green}{\texttt{>>>>>>> James}}}
\newcommand{\todo}[1]{\textcolor{green}{\texttt{[TODO: #1]}}}
\newcommand{\note}[1]{\textcolor{green}{\texttt{[NOTE: #1]}}}

\begin{document}

{\bf Software paper for submission to the Journal of Open Research Software} \\

Please submit the completed paper to: editor.jors@ubiquitypress.com

\rule{\textwidth}{1pt}

\section{(1) Overview}

\vspace{0.5cm}

\section{Title}
% Spinsim: a GPU optimized simulator of spin-half and spin-one quantum systems
Spinsim: a GPU optimized python package for simulating spin-half and spin-one quantum systems

\section{Paper Authors}
1. Tritt, Alex;\\
2. Morris, Joshua;\\
3. Hochstetter, Joel;\\
4. Anderson, R. P.;\\
5. Saunderson, James;\\
6. Turner, L. D.;\\

\section{Paper Author Roles and Affiliations}
1. School of Physics \& Astronomy, Monash University, Victoria 3800, Australia.\\
	Primary author of the released packages.\\
2. School of Physics \& Astronomy, Monash University, Victoria 3800, Australia.\\
	Present address: Vienna Center for Quantum Science and Technology (VCQ), Faculty of Physics, University of Vienna, 1010 Vienna, Austria.\\
	Author of first version of code.\\
3. School of Physics \& Astronomy, Monash University, Victoria 3800, Australia.\\
	Present address: Department of Applied Mathematics and Theoretical Physics, Centre for Mathematical Sciences, University of Cambridge, Cambridge, UK.\\
	Optimization and extension to spin-one of first version of code.\\
4. School of Molecular Sciences, La Trobe University, PO box 199, Bendigo, Victoria 3552, Australia.\\
	Present address: Q-CTRL Pty. Ltd.\\
	Original conception of first version, and advice on current version of code.\\
5. Department of Electrical and Computer Systems Engineering, Monash University, Victoria 3800, Australia.\\
	Advice on numerical analysis.\\
6. School of Physics \& Astronomy, Monash University, Victoria 3800, Australia.\\
	Original conception of released version of algorithm.

\section{Abstract}
	The \emph{Spinsim} \emph{python} package simulates spin-half and spin-one quantum mechanical systems following a time dependent Shr\"odinger equation.
	It makes use of \texttt{numba.cuda}, which is an \emph{LLVM} (Low Level Virtual Machine) compiler for \emph{Nvidia Cuda} compatible systems using GPU parallelization.
	Along with other optimizations, this allows for speed improvements from 3 to 4 orders of magnitude while staying just as accurate, compared to industry standard packages. % Mention which ones?
	It is available for installation on \emph{PyPI}, and the source code is available on \emph{github}.
	The initial use-case for the \emph{Spinsim} will be to simulate quantum sensing-based ultracold atom experiments for the Monash University School of Physics \& Astronomy spinor Bose-Einstein condensate (spinor BEC) lab, but we anticipate it will be useful in simulating any range of spin-half or spin-one quantum systems with time dependent Hamiltonians that cannot be solved analytically.
	These appear in the fields of nuclear magnetic resonance (NMR), nuclear quadrupole resonance (NQR) and magnetic resonance imaging (MRI) experiments and quantum sensing, and with the spin-one systems of nitrogen vacancy centres (NVCs), ultracold atoms, and BECs.

\section{Keywords}
Time dependent Schr\"odinger equation; Quantum; Physics; Spin one; Spin half; Integrator; Exponentiator; GPU; Parallel computing; Solver; Python; Numba; Sensing;

\section{Introduction}
\subsection{Motivation}
	Ultracold rubidium atoms have proven their effectiveness in state of the art technologies in quantum sensing~\cite{degen_quantum_2017}, the use of quantum mechanics to make precise measurements of small signals.
	The rotation of these atoms can be modelled as quantum spin systems, which is the quantum mechanical model for objects with angular momentum.
	The simplest spin system, spin-half (i.e.spin quantum number of $ \frac12 $, also referred to as a qubit), is quantized into just two quantum spin levels, and this describes the motion of some fundamental particles such as electrons.
	However, systems more practical for sensing, such as ultracold rubidium atoms, are more accurately described as a spin-one quantum system (i.e.spin quantum number of $ 1 $, also referred to as a quitrit), which is quantized into three quantum spin levels.

	The design of sensing protocols requires many steps of verification, including simulation.
	This is especially important, since running real experiments can be expensive and time consuming, so it is more practical to debug such protocols quickly and cheaply on a computer.
	In general, any design of experiments using spin systems could benefit from a fast, accurate method of simulation.

	% \texttt{AtomicPy}~\cite{morris_qcmonkatomicpy_2018}
	In the past, the spinor Bose Einstein condensate (spinor BEC) lab at Monash University used an in-house, \emph{cython} based script, on which this package, \emph{Spinsim}, is based, and standard differential equation solvers (such as \emph{Mathematica}'s~\cite{wolfram_research_inc_mathematica_2020} function \texttt{NDSolve}) to solve the Schr\"odinger equation for quantum sensing spin systems.
	Spin one systems are sometimes approximated to spin-half for a faster execution time, at the cost of not modelling all effects in the system.
	However, these methods are not completely optimized for our use-case, and therefore come with some issues.

	First, while the execution time for these solvers is acceptable for running a small number of experiments, for certain experiments involving large arrays of independent atom clouds (which require many thousands of simulations to be run), this time accumulates to the order of many hours, or even multiple days.
	Given the recent boom in machine learning, many research computers are now equipped with modern graphics processing units (GPUs).
	Their many cores, which can range from hundreds to tens of thousands in number, allows them to run some highly parallel algorithms much faster than a central processing unit (CPU).
	Examples include calculating color values for many pixels on a screen in the titular graphics processing, or the weights in a large neural network.
	While there do exists parallelized algorithms to solve general differential equations~\cite{lions_resolution_2001}, there are properties specific to the spin system that allow for much more fast and elegant parallelization in our use-case.
	We use these properties (see \emph{\nameref{sec:parallelization}}) to write a parallel, GPU based solver that runs faster than industry standard non-parallel algorithms by up to 4 orders of magnitude.
	% Mention GPU in fewer words

	Second, the Schr\"odinger equation has the geometric property of being norm persevering.
	In other words, the time-evolution operator for a system between two points in time must be unitary.
	As such, numerical solutions to the Schr\"odinger equation should also preserve norms.
	For many numerical methods like those in the Runge Kutta family, the approximations used are in general not norm preserving, so the evaluated quantum state may diverge towards an infinite norm, or converge to zero if run for many iterations.
	To avoid this, we use a geometric integration technique for our solver (see \emph{\nameref{sec:magnus}}).

	Third, our system (and similar spin systems) can be very oscillatory.
	In standard conditions for our application, the expected spin projection of a system that we want to solve for can rotate in physical space (alternatively viewed as a point rotating around an abstract object known as a \emph{Bloch sphere}) at a rate of 700kHz. % Too specific
	Standard integration methods require very small time-steps in order to accurately depict these oscillations.
	For this reason we use a fourth order Magnus expansion~\cite{magnus_exponential_1954,blanes_magnus_2009} based solver for our geometric integrator to make larger time-steps accurate (see \emph{\nameref{sec:magnus}}).
	We also use a dynamically changing rotating frame to slow down the effective rotations in the system, and thus reduce the size of integration steps being done, making each step more accurate (see \emph{\nameref{sec:rotating_frame}}).

	% By finding ways to parallelize the problem of solving quantum spin systems, we can use the many cores of a GPU to our advantage in this context as well.

	\subsection{Recent literature} % Recent relevant software? Recent software with similar functionality
		While we benchmark \emph{Spinsim} only against industry standard integrators, we also note that application specific quantum numerical simulator packages are regularly being released.
		However, none of these packages are designed specific to quantum sensing, and so these releases are not suitable for simulations highly accurate at high time resolutions required in that field.

		In quantum optimal control (QOC), researchers look at how to design robust pulse sequences to control spin systems using machine learning techniques.
		Particularly, this means that simulators used here require automatic differentiation~\cite{griewank_who_2012}, which \emph{Spinsim} does not currently support.
		A main use of QOC is to design control sequences in quantum computers.
		Many of these pulse sequences are trains of hard pulses (which are physically constant bursts of radiation), which, if using the accuracy reducing rotating wave approximation (RWA), can have simplified time evolution operators.
		This paradigm, used in the recent \emph{Qulacs}~\cite{suzuki_qulacs_2021}, \emph{QOpt}~\cite{teske_qopt_2021} and \emph{QSim}~\cite{isakov_simulations_2021}, is not suitable for accurately simulating more complicated (especially when looking from the lab frame of reference) pulse shapes found in quantum sensing.
		However, all of these packages do allow for simulating entanglement (important for quantum computing, but not for all sensing applications), which \emph{Spinsim} does not and, like \emph{Spinsim}, all have versions optimised for parallelisation (although \emph{QSim} does sacrifice accuracy here for the sake of speed again, by using single-precision floats).
		Dalgaard and Motzoi~\cite{dalgaard_fast_2021} have recently proposed Magnus integration techniques for use in QOC, but have not provided any open source implementation to do so.

		% Three qubit such packages were published recently, being \emph{PULSEE}~\cite{candoli_pulsee_2021}, \emph{QOpt}~\cite{teske_qopt_2021} and ~\cite{isakov_simulations_2021}.
		% While primarily focusing on being a complete toolkit for NMR simulations and quantum control optimization respectively, both contain Schr\"odinger equation solvers.
		% Furthermore, neither package can take advantage of GPU parallelization.

		Another application of simulations of spin systems is nuclear magnetic resonance (NMR).
		In this field, the simulator part of the \emph{PULSEE}~\cite{candoli_pulsee_2021} package uses up to the 3rd order Magnus expansion just once to solve the complete system, rather than using it many times as a time stepping method.
		Nested integrals of commutators are numerically evaluated on a CPU using Euler integration.
		This is likely to be slow (due to the Euler-integrated nested integrals of commutators) and inaccurate (due to using the expansion over a large time step), and potentially not even convergent in general (also due to the large time step); although it may suffice for the kinds of problems this NMR simulation package is looking at.

		% Designed for optimising quantum control protocols, the \emph{QOpt} package works in the paradigm of a time series of instructions, rather than the analytic forms used in \emph{Spinsim}.
		% Given the solid grid of time series, this package cannot take advantage of the commutator free Magnus based techniques that \emph{Spinsim} employs.
		% It may be able to model hard pulse sequences accurately (prevalent in quantum control), but will likely not be as accurate when modelling a continuous magnetic environment of a spin system.

		% Finally, also just recently, Dalgaard and Motzoi~\cite{dalgaard_fast_2021} propose the use of Magnus expansion based techniques for simulations in quantum control optimization.
		% However, they do not provide an implementation of their work.

\section{Implementation and architecture}
	% \subsection{Mathematical methods}
	\subsection{Quantum mechanics background}
		The \emph{Spinsim} package solves the time-dependent Schr\"{o}dinger equation
		\begin{align}
			i\hbar\frac{\text{d}\psi(t)}{\text{d}t} &= H(t)\psi(t),\label{eq:schroedinger}
		\end{align}
		where the quantum state $ \psi(t) \in \mathbb{C}^N $ is assumed normalized, and the Hamiltonian $ H(t) \in \mathbb{C}^{N \times N} $ is Hermitian.
		Here $ N $ is the number of levels in the quantum system.
		Often we are considering systems with spin of half ($ N = 2 $ case) or one ($ N = 3 $).
		% Spin-half is the $ N = 2 $ case, and spin-one is $ N = 3 $.
		% \supervisordisagreement{
		% 	Spin-half is the $ N = 2 $ case, and spin-one is $ N = 3 $.
		% }{
		% Spin-half is the $ N = 2 $ case, and spin-one is the $ N = 3 $ case.
		% }
		We set $ \hbar = 1 $, so that the Hamiltonian has physical dimension of angular frequency.
		% Furthermore, we have chosen for the Hamiltonian $ H(t) $ to have the physical dimension of frequency, rather than the more typical dimension of energy. To elaborate, we represent energies $ E $ of the system as their equivalent frequencies $ f $ from the Planck relation

		% \begin{align}
		% 	E &= \hbar \omega\\
		% 	&= 2\pi\hbar f,
		% \end{align}

		% where $ \hbar $ is (reduced) Planck's constant (or equivalently, setting
		% $ \hbar = 1 $). In addition, rather than being represented by standard coordinates in $ \mathbb{C}^{N \times N} $, in \emph{Spinsim} the Hamiltonian $ H(t) $ is instead represented with respect to a choice of basis for the corresponding Lie Algebra, $ \mathfrak{su}(N) $. For example, when set to spin-half mode, the \emph{Spinsim} package solves the time dependent Schr\"odinger equation of the form

		Rather than parametrizing the problem in terms of the matrix elements of $ H(t) $, we consider time varying real coefficients in a linear combination of a basis of fixed operators,
		\begin{align}
			H(t) &= \sum_{j = 1}^{N^2 - 1} \omega_j(t) \mathcal{A}_j.
		\end{align}
		Here we exclude the identity so that the Hamiltonian is traceless, which corresponds to choosing a physically meaningless energy zero point.
		It is well known~\cite[(p74)]{j_j_sakurai_jun_john_modern_2011} that a charged spin-half system with magnetic moment $ \overrightarrow{\mu} $, and gyromagnetic ratio $ \gamma $, in a magnetic field $ \overrightarrow{B}(t) $, has Hamiltonian
		\begin{align}
			H(t) &= -\overrightarrow{B}(t)\cdot \overrightarrow{\mu}\\
			% &= -\gamma \overrightarrow{B}(t)\cdot \overrightarrow{J}\\
			&= -\gamma \left(B_x(t) J_x + B_y(t) J_y + B_z(t) J_z\right)\\
			% &= \left(-\gamma B_x(t) J_x\right) + \left(-\gamma B_y(t) J_y\right) + \left(-\gamma B_z(t) J_z\right)\\
			&= \omega_x(t) J_x + \omega_y(t) J_y + \omega_z(t) J_z.
		\end{align}
		Here $ J_x $, $ J_y $ and $ J_z $ are spin operators, equal to the Pauli matrices~\cite[(p169)]{j_j_sakurai_jun_john_modern_2011} halved.
		% \supervisordisagreement{
		% 	Here $ J_x, J_y $ and $ J_z $ are spin operators, equal to the Pauli matrices halved.
		% }{
		% 	Here $ J_x, J_y $ and $ J_z $ are spin operators
		% 	\begin{align}
		% 		J_x &= \frac12\begin{pmatrix}
		% 			0 & 1 \\
		% 			1 & 0
		% 		\end{pmatrix},
		% 		&J_y &= \frac12\begin{pmatrix}
		% 			0 & -i \\
		% 			i &  0
		% 		\end{pmatrix},
		% 		&\textrm{and }J_z &= \frac12\begin{pmatrix}
		% 			1 &  0 \\
		% 			0 & -1
		% 		\end{pmatrix},\label{eq:spin_half_operators}
		% 	\end{align}
		% 	equal to the Pauli matrices halved.
		% }
		Motivated by this, we define our basis for the spin-half model with $ \mathcal{A}_1 = J_x $, $ \mathcal{A}_2 = J_y $ and $ \mathcal{A}_3 = J_z $.
		Additionally, because of the equivalence of the magnetic field components $ B_j(t) $ and the $ \omega_j(t) $, we henceforth refer to the latter as field functions.

		In the spin-one case there is no universal standard basis of operators $ \mathcal{A}_j $.
		Choices include the Gell-Mann matrices~\cite{gell-mann_symmetries_1962}, and multiple dipole-quadrupole bases~\cite{hamley_spin-nematic_2012, di_dipolequadrupole_2010}.
		In general, we can choose any basis from the 8-dimensional Lie algebra $ \mathfrak{su}(3) $, which is the vector space of traceless Hermitian operators that can generate transformations (from the corresponding Lie group $ SU(3) $) in the spin-one system.
		We focus on a particular subfamily of of spin-one systems for which the Hamiltonian is a linear combination of matrices from a 4-dimensional subspace of $ \mathfrak{su}(3) $, consisting of the spin matrices $ J_x $, $ J_y $ and $ J_z $, (labelled $ L_x $, $ L_y $ and $ L_z $ in Reference~\cite{hamley_spin-nematic_2012}) and a single quadrupole operator $ Q = \text{diag}(1, -2, 1)/3 $,% \todo{Describe this subspace better}

		% This spin-one exponentiator evolves Hamiltonians spanned by $ J_x $, $ J_y $, $ J_z $ and $ Q $ which is sufficient for three level systems in arbitrary bias fields, but with single-photon coupling.
		% An exponentiator capable of evolving an arbitrary spin-one Hamiltonian, for example with different coupling between the lower and upper pairs of states, or with two-photon coupling, is included in the package.

		% With this in mind, we choose to represent the Hamiltonian a linear combination of matrices from a 4-dimensional subspace of $ \mathfrak{su}(3) $, consisting of the spin matrices $ J_x, J_y $ and $ J_z $, and a single quadrupole operator $ Q = \text{diag}(1, -2, 1)/3 $,
		\begin{align}
			H(t) &= \omega_x(t) J_x + \omega_y(t) J_y + \omega_z(t) J_z + \omega_q(t) Q.
		\end{align}

		Note that $ Q $ is proportional to $ J_z^2 $ (up to a change in energy zero point), and $ Q_{zz} $~\cite{hamley_spin-nematic_2012} and $ Q_0 $~\cite{di_dipolequadrupole_2010} from alternative quadrupole bases.
		The $ J_x $, $ J_y $, $ J_z $ and $ Q $ are the only operators necessary to simulate many spin-one quantum systems in arbitrary bias fields, but with single-photon coupling.
		This includes quadratic Zeeman splitting described by the Breit-Rabi formula~\cite{mockler_atomic_1961}, important to experiments in our lab.
		The \emph{Spinsim} simulator can also be configured to solve a general spin-one system by setting the Hamiltonian to an arbitrary point in $ \mathfrak{su}(3) $ using the full quadrupole basis, which extends the possible Hamiltonian to
		\begin{align}
			H(t) =& \omega_x(t) J_x + \omega_y(t) J_y + \omega_z(t) J_z + \omega_q(t) Q\nonumber\\
			&+ \omega_{u1}(t) U_1 + \omega_{u2}(t) U_2 + \omega_{v1}(t) V_1 + \omega_{v2}(t) V_2.
		\end{align}
		Here the additional operators are those defined in~\cite{di_dipolequadrupole_2010}. Note that this is included for completeness, but has not been thoroughly tested, as our lab has no physical context for modelling the $ U_1, U_2, V_1 $ and $ V_2 $ operators.
		% In general ... Lie algebra ... mathfrak ... admits a basis ... .
		% \begin{align}
		% 	\frac{\text{d}}{\text{d}t}\psi(t) = -i 2\pi (f_x(t) J_x + f_y(t) J_y + f_z(t) J_z) \psi(t),
		% \end{align}

		% where $ i^2 = -1 $, $ \psi(t) \in \mathbb{C}^2 $, and the spin-half spin projection operators are given by

		% DONT WRITE OUT, sigma

		% Call omega, no implementation detail
		% The fields $ f $ that represents the time dependent Hamiltonian $ H(t) $, are the collection of energy functions $ f_x(t), f_y(t), f_z(t) $, with time $ t $ and $ f $ with physical dimension of frequency that control the dynamics of the system. The user must define a method that returns a sample of these field functions when a sampling time is input. In physical terms, these functions could represent the $ x,y,z $ components of a magnetic field applied to a magnetically spin-one BEC or NVC.

		% Similarly, when \emph{Spinsim} is set to spin-one mode, it can solve the Schr\"odinger equation of the form

		% \begin{align}
		% 	\frac{\text{d}}{\text{d}t}\psi(t) = -i 2\pi (f_x(t) J_x + f_y(t) J_y + f_z(t) J_z + f_q(t) Q) \psi(t).
		% \end{align}

		% where now $ \psi(t) \in \mathbb{C}^3 $, and the spin-one operators are given by

		% \begin{align}
		% 	J_x &= \frac{1}{\sqrt{2}}\begin{pmatrix}
		% 		0 & 1 & 0 \\
		% 		1 & 0 & 1 \\
		% 		0 & 1 & 0
		% 	\end{pmatrix},&
		% 	J_y &= \frac{1}{\sqrt{2}}\begin{pmatrix}
		% 		0 & -i &  0 \\
		% 		i &  0 & -i \\
		% 		0 &  i &  0
		% 	\end{pmatrix},\nonumber\\
		% 	J_z &= \begin{pmatrix}
		% 		1 & 0 &  0 \\
		% 		0 & 0 &  0 \\
		% 		0 & 0 & -1
		% 	\end{pmatrix},&
		% 	\textrm{and }Q &= \frac{1}{3}\begin{pmatrix}
		% 		1 &  0 & 0 \\
		% 		0 & -2 & 0 \\
		% 		0 &  0 & 1
		% 	\end{pmatrix}.\label{eq:spin_one_operators}
		% \end{align}

		% The matrices $ J_x, J_y, J_z $ are regular spin operators, and $ Q $ is a quadrupole operator. Note that $ Q $ is proportional to $ Q_{zz} $ as defined by Hamley et al~\cite{hamley_spin-nematic_2012}, and $ Q_0 $ as defined by Di et al~\cite{di_dipolequadrupole_2010}.
		% INLINE Q DIAG. WHY WE'VE PICKED THESE FOUR. For ground state spin-one atoms in a magnetic field, ... Breit Rabi ... . COULD BE READILY EXTENDED TO INCLUDE FURTHER ELEMENTS OR TO HIGHER SPINS.

		% Too far out of scope
		% Alternative: integrate Bloch equations 3 coupled real equations
		% fast and parallel function from wavefunctions
		As well as integrating the Schr\"odinger equation, \emph{Spinsim} also has the functionality to calculate the expected spin projection $ \langle \overrightarrow{J}\rangle (t) $ of a system from its state.
		In an experimental setting, one cannot measure the value of the state itself, and must instead measure observables such as spin projection.
		If this is done for an ensemble of systems, the average spin projection over all systems will converge to the expected value as calculated here.
		As an alternative to solving the Schr\"odinger equation, if one is only interested in the dynamics of the expected spin projection, one can instead solve for $ \langle \overrightarrow{J}\rangle (t) $ directly using the Bloch equations~\cite{bloch_nuclear_1946}.
		These are three coupled real equations as compared to three coupled complex equations for the spin-one Schr\"odinger equation.
		However, the Bloch equations cannot be used to compute $ \psi(t) $, and can only fully model spin-half systems (and spin-one systems approximated to spin-half).
		Thus, with \emph{Spinsim} we choose to solve the full Schr\"odinger equation instead.

		% Frequently when one calculates the times series of the quantum state $ \psi(t) $, they are also interested in its expected spin projection. It is given by the vector

		% % Inline
		% \begin{align}
		% 	\left\langle J\right\rangle(t) &= \begin{pmatrix}
		% 		\psi(t)^\dagger J_x \psi(t)\\
		% 		\psi(t)^\dagger J_y \psi(t)\\
		% 		\psi(t)^\dagger J_z \psi(t)
		% 	\end{pmatrix},
		% \end{align}

		% where $ \cdot^\dagger $ is the adjoint operator. The expected spin projection can be interpreted as the average direction that the spin system is oriented in space when many systems of the equivalent state are measured (due to the Heisenberg uncertainty principle, only one component of the exact orientation of a quantum state can be known at any point in time, which is why we need to deal with expected values rather than absolute values). However, it can also be used to model the overall orientation of an ensemble of many quantum systems, like a BEC, for instance. \emph{Spinsim} has the functionality to calculate the expected spin projection of a system from its state.

	\subsection{Unitary time evolution and the Magnus expansion}\label{sec:magnus}
		Since $H(t)$ is Hermitian, it follows from Equation~\eqref{eq:schroedinger} that $\|\psi(t)\|^2 = \|\psi(t_0)\|^2$ for all $t$.
		Therefore it is possible to write $ \psi(t) $ in terms of a unitary transformation $ \mathcal{U}(t, t_0) $ of the state $ \psi(t_0) $, for any time $ t_0 $, that is, $ \psi(t) = \mathcal{U}(t, t_0)\psi(t_0) $.
		It then follows from Equation~\eqref{eq:schroedinger} that $ \mathcal{U}(t, t_0) $ also follows a Schr\"{o}dinger equation,
		\begin{align}
			i\hbar\frac{\text{d}\mathcal{U}(t, t_0)}{\text{d}t} &= H(t)\mathcal{U}(t, t_0).\label{eq:unitary_schroedinger}
		\end{align}

		Thus, to solve Equation~\eqref{eq:schroedinger} for $ \psi(t) $ given a particular $ \psi(t_0) $, one only needs to solve Equation~\eqref{eq:unitary_schroedinger} for $ \mathcal{U}(t, t_0) $.
		% Note that because $ \mathcal{U}(t, t_0) $ is unitary, the calculated value of $ \psi(t) $ using this method will not change in magnitude, conserving probability.
		Since $ \mathcal{U}(t, t_0) $ is unitary, it can be written as a matrix exponential of a anti-Hermitian (also termed skew-Hermitian) matrix.
		If the Hamiltonian is a of constant value $ H(t) = \mathcal{H} $, then this exponential is % \note{Should we use H, i.e.no argument, or H(t'), i.e.arbitrary argument, to represent constancy?} % Check if continuity requirement on unitary
		\begin{align}
			\mathcal{U}(t, t_0) &= \exp(-i (t - t_0) \mathcal{H}).\label{eq:exp_sol_of_constant}
		\end{align}
		% The act of integrating a linear differential equation by exponentiation can be generalised for time dependent Hamiltonians using the Magnus Series~\cite{blanes_magnus_2009}, where
		For a time varying Hamiltonian, the general solution for $ \mathcal{U}(t, t_0) $ is much more complex, because it encapsulates the full solution to the time-dependent Schr\"{o}dinger equation.
		The well known Dyson series~\cite{kalev_integral-free_2020} gives an explicit expression for $ \mathcal{U}(t, t_0) $ in terms of multiple time integrals over nested time commutators of $ H(t') $.
		While the Dyson series has recently been used numerically~\cite{kalev_integral-free_2020}, it is challenging to work with because once truncated, it is in general no longer unitary.

		The Magnus series, in contrast, gives $ \mathcal{U}(t, t_0) $ in terms of an exponential of a series of anti-Hermitian operators, specifically
		\begin{align}
			\mathcal{U}(t, t_0) &= \exp\left(\Omega(t, t_0)\right) = \exp\left(\sum_{m = 1}^\infty \Omega_m(t, t_0)\right).
		\end{align}
		As such, the Magnus series explicitly preserves unitarity when truncated~\cite{magnus_exponential_1954}.
		The terms in the Magnus series are integrals over nested time commutators of the Hermitian matrix $ A(t') $, where, in the case of quantum mechanics, $ A(t') = -iH(t') $.
		For instance, the first three terms are~\cite{blanes_magnus_2009}
		\begin{align}
			\Omega_1(t, t_0) &= \int_0^t\text{d}t_1A(t_1),\\
			\Omega_2(t, t_0) &= \frac12\int_{t_0}^t\text{d}t_1\int_{t_0}^{t_1}\text{d}t_2[A(t_1), A(t_2)],\\
			\Omega_3(t, t_0) &= \frac16\int_{t_0}^t\text{d}t_1\int_{t_0}^{t_1}\text{d}t_2\int_{t_0}^{t_2}\text{d}t_3\left([A(t_1), [A(t_2), A(t_3)]] + [A(t_3), [A(t_2), A(t_1)]]\right),
		\end{align}
		where $ [X, Y] = XY - YX $ is the commutator.
		Note that, when truncated to first order, the Magnus expansion $ \mathcal{U}(t, t_0) = \exp\left(\int_{t_0}^t\text{d}t_1\left(-iH(t_1)\right)\right) $ reduces to Equation~\eqref{eq:exp_sol_of_constant}, with $ H(t) $ approximated by its average value over the interval $ [t_0, t] $.

		% Insert first few omegas here
		% Throw back to constant
		% Here each $ \Omega_m(t) $ are integrals over time commutators of $ H(t') $.
		% $ \Omega_1(t) $, for example, is $ \Omega_1(t) = H_\text{av}t $, where $ H_\text{av} $ is the average value of $ H $ over the interval being integrated over, which is the solution for when $ H $ is constant in time.
		% The terms $ \Omega_m(t) $ for $ m > 1 $ are corrections to this approximation. One can truncate the Magnus series before infinity to make a Mangus expansion, which can given an approximation for $ \mathcal{U}(t, t_0) $.
		% Importantly, any truncation of the Magnus expansion is unitary, meaning that probability will be conserved when used.

		Convergence of the Magnus series for $ \mathcal{U}(t, t_0) $ is not in general guaranteed, but it is under the condition that $ \int_{t_0}^t\text{d}t_1\|H(t_1)\|_2 < \xi \approx 1.08686870\dots $~\cite{blanes_magnus_2009}.
		% The loosest bound discovered for the convergence of a general Magnus series for $ \mathcal{U}(t, t_0) $ is $ \int_{t_0}^t\text{d}t_1\|H(t_1)\| < 1.08686870\dots $~\cite{blanes_magnus_2009}.
		% A Magnus series for $ \mathcal{U}(t, t_0) $ will be guaranteed to converge if $ \int_{t_0}^t\text{d}t_1\|H(t_1)\| < \xi = 1.08686870\dots $~\cite{blanes_magnus_2009}.
		Furthermore, each subsequent term in the expansion increases in complexity rapidly.
		For these reasons, the Magnus expansion is generally used as a time-stepping method rather than a single step to solve the complete system.
		Designing a Magnus-based solver is a trade-off between the number of terms used in each expansion, and the number of time-steps used.
		Time stepping is based on the fact that time evolution operators can be split into a product via
		\begin{align}
			\mathcal{U}(t, t_0) &= \mathcal{U}(t, t_{n - 1})\mathcal{U}(t_{n - 1}, t_{n - 2})\cdots\mathcal{U}(t_2, t_1)\mathcal{U}(t_1, t_0).\label{eq:product_of_time_evolution}
		\end{align}
		Each of the time-evolution operators can be approximated by a Magnus expansion.

		% Each of these parts can be approximated as a Magnus expansion. So, if the Magnus expansion of time evolution operator is inaccurate, it can be broken down into a product of many time evolution operators which each make a much smaller step between times. These components will each have a much more accurate Magnus expansion. % convergence of Magnus series, not simple. Also Dyson. Standard bounds.
		% Numerical solver takes only a few terms of the expansion. However, any long operator can be broken into a product. Cite Magnus papers. Designing a Magnus-based integrator is a trade off.

		Many unitary time-stepping techniques have been developed based on the Magnus expansion~\cite{auer_magnus_2018}.
		Some of these techniques use Gauss-Legendre quadrature sampling and the Baker–Campbell–Hausdorff formula to respectively avoid the integration and time commutators of the Hamiltonian, producing simple expressions that can be used for unitary time-stepping~\cite{blanes_fourth-_2006}.
		% These take advantage of the Baker–Campbell–Hausdorff formula

		In particular, we use the commutator free, fourth order method (CF4) from Reference~\cite{auer_magnus_2018}.
		Suppose we wish to evaluate a time-step of $ \mathcal{U}(t + \delta t, t) $ using the CF4 method.
		The Hamiltonians are sampled at times
		\begin{align}
			t_1 &= t + \frac12 \left(1 - \frac{1}{\sqrt{3}}\right)\delta t\quad\text{and}\\
			t_2 &= t + \frac12 \left(1 + \frac{1}{\sqrt{3}}\right)\delta t,
		\end{align} % maybe not rm d
		based on the second order Gauss-Legendre quadrature
		\begin{align}
			\overline{H}_1 &= \frac{3 + 2\sqrt{3}}{12}H(t_1) + \frac{3 - 2\sqrt{3}}{12}H(t_2)\quad\text{and}\label{eq:cf4_sample_1}\\
			\overline{H}_2 &= \frac{3 - 2\sqrt{3}}{12}H(t_1) + \frac{3 + 2\sqrt{3}}{12}H(t_2).\label{eq:cf4_sample_2}
		\end{align}
		% \, \quad \qquad
		% The time-evolution operator is then approximated by
		Then
		\begin{align}
			\mathcal{U}(t + \delta t, t) = \exp(-i\overline{H}_2\,\delta t)\exp(-i\overline{H}_1\,\delta t) + \mathcal{O}(\delta t^5),\label{eq:cf4_implementation}
		\end{align}% \approx? Unitary? End with sentence
		is used to approximate the time-evolution operator.

	\subsection{Lie-Trotter based exponentiator}
		Evaluating matrix exponentials is a core part of the integration algorithm.
		Rather than exponentiating the Hamiltonian directly as in Equation~\eqref{eq:cf4_implementation}, \emph{Spinsim} works with the field functions $ \omega_j(t) $.
		% as a vector representation of the Hamiltonian of the spin system being simulated, rather than generic matrix elements in an array.
		% This allows for matrix exponentiators that take advantage of this representation.
		% For all implementations of exponentiation, the \emph{Spinsim} computes the matrix exponential
		% % \begin{align}
		% % 	\exp(-iHt) &= E(\omega_x, \omega_y, \omega_z, \omega_q)\\
		% % 	&= \exp(-i (\omega_x J_x + \omega_y J_y + \omega_z J_z + \omega_q Q)).
		% % \end{align}
		% \begin{align}
		% 	\exp(-iHt) &= E(\omega_x, \omega_y, \omega_z, \omega_q)\\
		% 	&= \exp(-i (\omega_x J_x + \omega_y J_y + \omega_z J_z + \omega_q Q)).
		% \end{align}
		% In practice, $ \omega $ is the Lie algebra (spin and quadrature) basis representation of $ \overline{H_1} $ or $ \overline{H_2} $ defined in Equations~\eqref{eq:cf4_sample_1} and~\eqref{eq:cf4_sample_2}, which need to be exponentiated in Equation~\eqref{eq:cf4_implementation}.

		For spin-half, the exponentiator is in an analytic form in $ \omega_x, \omega_y $ and $ \omega_z $. % ... yz, which follows directly from eqref
		For spin-one, an exponentiator based on the Lie-Trotter product formula~\cite{moler_nineteen_2003}
		% Explicitly, this formula is
		\begin{align}
			\exp\left( X + Y\right) &= \lim_{n\to\infty} \left(\exp\left(\frac{X}{n}\right) \exp\left(\frac{Y}{n}\right)\right)^n,\label{eq:lie_trotter}
		\end{align}
		is used.
		% Here $ X,Y\in\mathbb{C}^{N\times N} $.
		An advantage of the Lie-Trotter approach is that $ \exp(-iA_k/n) $ has known analytic forms for the Lie algebra basis elements $ A_k $.
		Hence the unitary time-evolution operator is approximated by $ U = \exp(-iH\,\delta t) = T^n + \mathcal{O}(\delta t^3) $, where
		\begin{align}
			T &= \exp\left(\frac{-i \omega_x J_x}{n}\right) \exp\left(\frac{-i \omega_y J_y}{n}\right) \exp\left(\frac{-i \omega_z J_z}{n}\right) \exp\left(\frac{-i \omega_q Q}{n}\right).
		\end{align}
		In fact, commutation relations between the Lie basis operators and the leapfrog splitting method~\cite{barthel_optimized_2020} allow us to instead write
		\begin{align}
			T &= \exp\left(-i\frac12D_{z,q}\right)\exp(-i\Phi J_\phi)\exp\left(-i\frac12D_{z,q}\right).
		\end{align}
		Here $ z = \omega_z/n $, $q = \omega_q/n $, $ \Phi = \sqrt{\omega_x^2 + \omega_y^2}/n $, $ \phi = \text{arctan}2(\omega_y, \omega_x) $ (where $ \text{arctan}2(y, x) $ is the two argument arctangent~\cite{organick_fortran_1966}), $ D_{z,q} = zJ_z + qQ $ and $ J_\phi = \cos(\phi) J_x + \sin(\phi) J_y $.
		The element-wise analytic form of this is
		\begin{align}
			T &= \begin{pmatrix}
				\left(\cos\left(\frac{\Phi}{2}\right) e^{-iz/2}e^{-iq/6}\right)^2 & \frac{-i}{\sqrt{2}} \sin(\Phi)e^{iq/6}e^{-iz/2}e^{-i\phi} & -\left(\sin\left(\frac{\Phi}{2}\right)e^{iq/6}e^{-i\phi}\right)^2\\
				\frac{-i}{\sqrt{2}} \sin(\Phi)e^{iq/6}e^{-iz/2}e^{i\phi} & \cos(\Phi)e^{i4q} & \frac{-i}{\sqrt{2}} \sin(\Phi)e^{iq/6}e^{iz/2}e^{-i\phi}\\
				-\left(\sin\left(\frac{\Phi}{2}\right)e^{-iq/6}e^{i\phi}\right)^2 & \frac{-i}{\sqrt{2}} \sin(\Phi)e^{iq/6}e^{iz/2}e^{i\phi} & \left(\cos\left(\frac{\Phi}{2}\right) e^{iz/2}e^{-iq/6}\right)^2
			\end{pmatrix}.\label{eq:lie_trotter_4}
		\end{align}
		% The equality holds for the limit for all $ n $ if the operators commute, like it does for real and complex numbers.
		% This tells us that we can approximate a matrix exponential of a linear combination of Hermitian operators by scaling down these operators, taking the analytically simple exponentials of each of the operators $ -iJ_x, -iJ_y, -iJ_z $ and $ -iQ $ separately, multiplying each individual result together, and then raising the combination $ T $ to the power of the original reduction.
		% 2^{-\tau} -> n
		% where X=Jx+iJy and Y=Qz+JZ
		% Dz=Jz+Qzz

		% The exponential $ E(\omega) $ can be approximated as, for large $ 2^\tau $,
		% \begin{align}
		% 	E(\omega) =& \exp\left(-i\omega_x J_x - i\omega_y J_y - i\omega_z J_z - i\omega_q Q\right)\\
		% 	=& \exp\left(2^{-\tau}\left(-i\omega_x J_x - i\omega_y J_y - i\omega_z J_z - i\omega_q Q\right)\right)^{2^\tau}\\
		% 	\approx& \biggl(\exp\left(-i\frac12\left(2^{-\tau} \omega_z J_z + 2^{-\tau}\omega_q Q\right)\right)\nonumber\\
		% 	&\cdot\exp\left(-i\left(2^{-\tau} \omega_\phi J_\phi\right)\right)\nonumber\\
		% 	&\cdot\exp\left(-i\frac12\left(2^{-\tau} \omega_z J_z + 2^{-\tau} \omega_q Q\right)\right)\biggr)^{2^\tau}\\
		% 	=& \begin{pmatrix}
		% 		\left(\cos\left(\frac{\Phi}{2}\right) e^{-iz/2}e^{-iq/6}\right)^2 & \frac{-i}{\sqrt{2}} \sin(\Phi)e^{iq/6}e^{-iz/2}e^{-i\phi} & -\left(\sin\left(\frac{\Phi}{2}\right)e^{iq/6}e^{-i\phi}\right)^2\\
		% 		\frac{-i}{\sqrt{2}} \sin(\Phi)e^{iq/6}e^{-iz/2}e^{i\phi} & \cos(\Phi)e^{i4q} & \frac{-i}{\sqrt{2}} \sin(\Phi)e^{iq/6}e^{iz/2}e^{-i\phi}\\
		% 		-\left(\sin\left(\frac{\Phi}{2}\right)e^{-iq/6}e^{i\phi}\right)^2 & \frac{-i}{\sqrt{2}} \sin(\Phi)e^{iq/6}e^{iz/2}e^{i\phi} & \left(\cos\left(\frac{\Phi}{2}\right) e^{iz/2}e^{-iq/6}\right)^2
		% 	\end{pmatrix}^{2^\tau}\\
		% 	=& T^{2^\tau}.
		% \end{align}
		% % Trotter half
		% \begin{align}
		% 	E(\omega) =& \exp\left(-i\omega_x J_x - i\omega_y J_y - i\omega_z J_z\right)\\
		% 	=& \exp\left(2^{-\tau}\left(-i\omega_x J_x - i\omega_y J_y - i\omega_z J_z\right)\right)^{2^\tau}\\
		% 	\approx& \biggl(\exp\left(-i\frac12 2^{-\tau} \omega_z J_z\right)\exp\left(-i\left(2^{-\tau} \omega_\phi J_\phi\right)\right)\exp\left(-i\frac12 2^{-\tau} \omega_z J_z\right)\biggr)^{2^\tau}\\
		% 	=& \begin{pmatrix}
		% 		\cos\left(\frac{\Phi}{2}\right)e^{-iz} & -i\sin\left(\frac{\Phi}{2}\right) e^{i\phi}\\
		% 		-i\sin\left(\frac{\Phi}{2}\right) e^{-i\phi} & \cos\left(\frac{\Phi}{2}\right)e^{iz}
		% 	\end{pmatrix}^{2^\tau}\\
		% 	=& T^{2^\tau}.
		% \end{align}
		% % Trotter 8
		% \begin{align}
		% 	E(\omega) =& \exp\biggl(-i\omega_x J_x - i\omega_y J_y - i\omega_z J_z - i\omega_q Q\\
		% 	&- i\omega_{u1} U_1 - i\omega_{u2} U_2 - i\omega_{v1} V_1 - i\omega_{v2} V_2\biggr)\\
		% 	& \exp\biggl(2^{-\tau}\biggl(-i\omega_x J_x - i\omega_y J_y - i\omega_z J_z - i\omega_q Q\\
		% 	&- i\omega_{u1} U_1 - i\omega_{u2} U_2 - i\omega_{v1} V_1 - i\omega_{v2} V_2\biggr)\biggr)^{2^\tau}\\
		% 	\approx& \biggl(\exp\left(-i2^{-\tau} \omega_\phi J_\phi\right)\exp\left(-i2^{-\tau} \omega_{u\phi} U_{u\phi}\right)\\
		% 	&\cdot\exp\left(-i2^{-\tau} \omega_{v\phi} V_{v\phi}\right)\exp\left(-i2^{-\tau} \omega_z J_z -i2^{-\tau} \omega_q Q \right)\biggr)^{2^\tau}\\
		% 	=& \biggl(\begin{pmatrix}
		% 		\cos^2\left(\frac{\Phi}{2}\right) & \frac{-i}{\sqrt{2}} \sin(\Phi)e^{-i\phi} & -\left(\sin\left(\frac{\Phi}{2}\right)e^{-i\phi}\right)^2\\
		% 		\frac{-i}{\sqrt{2}} \sin(\Phi)e^{i\phi} & \cos\left(\Phi\right) & \frac{-i}{\sqrt{2}} \sin(\Phi)e^{-i\phi}\\
		% 		-\left(\sin\left(\frac{\Phi}{2}\right)e^{i\phi}\right)^2 & \frac{-i}{\sqrt{2}} \sin(\Phi)e^{i\phi} & \cos^2\left(\frac{\Phi}{2}\right)
		% 	\end{pmatrix}\\
		% 	&\cdot \begin{pmatrix}
		% 		\cos\left(\Phi_u\right) & 0 & -i \sin\left(\Phi_u\right)e^{-i\phi_u}\\
		% 		0 & 1 & 0\\
		% 		-i \sin\left(\Phi_u\right)e^{i\phi_u} & 0 & \cos\left(\Phi_u\right)
		% 	\end{pmatrix}\\
		% 	&\cdot \begin{pmatrix}
		% 		\cos^2\left(\frac{\Phi_v}{2}\right) & \frac{-i}{\sqrt{2}} \sin(\Phi_v)e^{-i\phi_v} & \left(\sin\left(\frac{\Phi_v}{2}\right)e^{-i\phi_v}\right)^2\\
		% 		\frac{-i}{\sqrt{2}} \sin(\Phi_v)e^{i\phi_v} & \cos\left(\Phi_v\right) & \frac{i}{\sqrt{2}} \sin(\Phi_v)e^{-i\phi_v}\\
		% 		\left(\sin\left(\frac{\Phi_v}{2}\right)e^{i\phi_v}\right)^2 & \frac{i}{\sqrt{2}} \sin(\Phi_v)e^{i\phi_v} & \cos^2\left(\frac{\Phi_v}{2}\right)
		% 	\end{pmatrix}\\
		% 	&\cdot \begin{pmatrix}
		% 		e^{-iz - iq} & 0 & 0\\
		% 		0 & e^{i2q} & 0\\
		% 		0 & 0 & e^{iz - iq}
		% 	\end{pmatrix}\biggr)^{2^\tau}\\
		% 	=& T^{2^\tau}
		% \end{align}

		Matrix exponentiation is completed by raising $ T $ to an integer $ n $.
		To approximate the limit in Equation~\eqref{eq:lie_trotter}, $ n $ must be a large number.
		We choose $ n $ to be of the form $ n = 2^\tau $, as raising matrices to powers of two can be done efficiently by iterative squaring.
		% This can be done just by iterative squaring, which requires a small number of operations.
		While there are some loose upper bounds for the $ n $ required to reach a desired accuracy for the matrix exponential~\cite{suzuki_generalized_1976}, using these to choose a value of $ n $ can, in practice, cause floating-point errors from over-squaring.
		Instead, we tested the formula on $ 10^5 $ matrices for different values of $ \tau $, and compared them to results given by \texttt{scipy.linalg.expm()}, from the popular python library \emph{SciPy}~\cite{virtanen_scipy_2020}.
		We found that the error minimized after a number of squares of $ \tau = 24 $.
		Thus, by default, $ n = 2^{24} $.

		In addition to those introduced by the Lie-Trotter approximation, we find errors when manipulating matrices close to the identity.
		Because $ n $ is very large, $ T $ is very close to the identity.
		In particular, the limited precision of floating-point numbers is unable to represent the diagonal of $ T $ and its squares accurately.
		Errors appear in two forms.
		One form is that iterative squaring of matrices close to the identity produces floating-point cancellation errors.
		It can be avoided by working with the difference of the matrices from the identity when iteratively squaring.
		That is, if $ A = I + a $, then instead of calculating the square $ S = I + s = A^2 $, we find $ s = (a + 2I)a $.
		The other form of error is that if we use this method, we must initially calculate $ T - I $ accurately, rather than $ T $.
		This cannot be done by simply subtracting the identity from Equation~\eqref{eq:lie_trotter_4}, as doing so would introduce cancellation error.
		It can be avoided by replacing the exponentials and trigonometric functions along the diagonal with specialized implementations of functions such as $ \text{expm}1(x) = \exp(x) - 1 $, designed for this purpose~\cite{hewlett-packard_hp_1994}.
		The identity is added to the result of the iterative residual squaring and the full exponential is returned.

		% % Choice of n. n is large number, integer, power of 2. Efficiently by iterated squaring. Reference. % Revisit!!!
		% Here $ z = 2^{-\tau}\frac{\omega_z}{2} $, $ q = 2^{-\tau}\frac{\omega_q}{6} $, $ \phi = 2^{-\tau}\sqrt{\omega_x^2 + \omega_y^2} $, and $ \phi = \text{atan}2(\omega_y, \omega_x) $.
		% Once $ T $ is calculated, it is then recursively squared $ \tau $ times to obtain $ E(\omega) $.
		% In practice the calculations here are done by finding the differences of the matrices from the identity to avoid floating-point cancellation errors from subtracting a small number from 1.
		% Three sentences rather than one
		% Errors:
		%	Trotter
		% 	Evaluating T
		% 	Iterated squaring
		% This spin-one exponentiator cannot solve arbitrary spin-one quantum systems, as that would require the ability to exponentiate a point in the full, 8-dimensional Lie algebra of $ \mathfrak{su}(3) $, rather than just the 4-dimensional subspace spanned by the subalgebra $ \mathfrak{su}(2) $ spanned by $ \{J_x, J_y, J_z, Q\} $, and the single quadratic operator $ Q $.
		This spin-one exponentiator evolves Hamiltonians spanned by $ J_x $, $ J_y $, $ J_z $ and $ Q $ which is sufficient for three level systems in arbitrary bias fields, but with single-photon coupling.
		An exponentiator capable of evolving an arbitrary spin-one Hamiltonian, for example with different coupling between the lower and upper pairs of states, or with two-photon coupling, is included in the package.
		When assessing accuracy for spin-one problems, we have used the single-photon exponentiator.

		Note that, the methods for both spin-half and spin-one use analytic forms of matrix exponentials so that $ T $ is unitary by construction; $ \mathcal{U}=T^n $ is then also unitary.
		Simulations in \emph{Spinsim} thus maintain unitarity and so conserve probability even over very large numbers of time-steps.

	\subsection{Discretization and parallelization}\label{sec:parallelization}
		Frequently we wish to sample the state at times $ t_k $ spaced more coarsely than the integration time-step.
		Consider discrete times $ t_k = t_0 + \Delta t\cdot k $ where $ \Delta t $ is the time-step of the time-series, in contrast to $ \delta t $, the time-step of the integration.
		% The unitary operator $ \mathcal{U}_k $
		Denoting $ \psi_k = \psi(t_k) $ and $ \mathcal{U}_k = \mathcal{U}(t_{k}, t_{k-1}) $, the time-series of states $  \psi_k $ and time-evolution operators $ \mathcal{U}_k $ satisfy
		\begin{align}
			\psi_k &= \mathcal{U}_k\psi_{k-1}.\label{eq:integration_compilation}
		\end{align}
		This presents an opportunity for parallelism.
		While $ \psi_k $ depends on $ \psi_{k-1} $, the operators $ \mathcal{U}_k $ depend only on the field functions, and not on previous states $ \psi_{k'} $ or previous operators $ \mathcal{U}_{k'} $.
		Hence, the time-evolution operators $ \mathcal{U}_k $ can be calculated in parallel.
		% It allows \emph{Spinsim} to use GPU parallelization on the level of time sample points, so a speed up is achieved even if just a single simulation is run.

		\emph{Spinsim} splits the full simulation into time intervals $ [t_{k - 1}, t_{k}] $, and calculates time-evolution operators $ \mathcal{U}_k $ for these intervals in parallel on a GPU. % check phrase "massive"
		% within small time intervals $ [t_{k - 1}, t_{k}] $, which are each calculated massively in parallel on a GPU.
		When all the $ \mathcal{U}_k $ are calculated, the CPU then multiplies them together (a comparatively less demanding job than calculating them) using Equation~\eqref{eq:integration_compilation} to determine the output samples $ \psi_k $ of the evolving state.

		% Within each parallel discretization, we ...
		Beyond discretization for parallelization, we discretize our time-evolution operator into individual integration time-steps.
		Each of the $ \mathcal{U}_k $ is further split into products $ u^k_{L-1} \cdots u^k_0 $ of $ L $ time-evolution operators, now separated by the integration time-step $ \delta t = \Delta t/L $.

		To choose an appropriate value of $ \delta t $, users can refer to the accuracy plots in Figures~\ref{fig:benchmark_spin_one_step_error} and~\ref{fig:benchmark_spin_half_step_error} under \emph{\nameref{sec:accuracy}}. % \note{How do I reference a section if they don't have number labels in the finished article? Should I?}
		To choose an appropriate value of $ \Delta t $, users must consider three factors.
		For the most benefit out of GPU parallelization, users should choose $ \Delta t $ so that the number of time samples $ K $ is at least as large as the number of GPU cores available for \emph{Spinsim}.
		Users should also make sure that $ \Delta t $ represents a large enough sample rate for the state $ \psi_k $ or expected spin projection $ \langle \overrightarrow{J}\rangle_k $ to be used in their further analysis.
		Finally, if $ K $ is too large, then the GPU will run out of memory on compilation, which will raise an exception. % (which we found to be $ K = 3\cdot 10^8 $ sample points for the 10GiB \emph{GeForce RTX 3080}).
		We found that the 10GiB \emph{GeForce RTX 3080} could run at full memory capacity with 60 million sample points for spin-one mode and 120 million sample points for spin-half.
		% Comment on number of samples for cores or whatever physics is being done
		% Introduce K if it turns out I use it later

		% \begin{align}
		% 	\mathcal{U}(t_k, t_{k-1}) &= \mathcal{U}(t_k, t_k - \delta t) \cdots \mathcal{U}(t_{k-1} + 2\delta t, t_{k-1} + \delta t) \mathcal{U}(t_{k-1} + \delta t, t_{k-1})\\
		% 	\mathcal{U}_k &= u^k_{L-1} \cdots u^k_0,\quad\text{where}\\
		% 	u^k_{l} &= \mathcal{U}(t_0 + (k - 1)\Delta t + (l + 1)\delta t, t_0 + (k - 1)\Delta t + l\delta t)
		% \end{align} % Just use middle product
		% with $ \delta t $ being the integration level time-step.
		% Note that the sampling time-steps are related to each other by $ \Delta t = L\delta t $.

	\subsection{Dynamically rotating frame}\label{sec:rotating_frame}
		Hamiltonians which have dominant and slowly-varying terms induce rotations around a primary axis, which is usually chosen to be the quantization axis (which is the $ z $ axis when representing spin operators with the Pauli matrices).
		Transforming from the lab (standard) frame of reference into one rotating around this axis by rotation operator $ R_{\omega_r}(t) = \exp(i\omega_r J_z t) $, the resulting system follows a new Hamiltonian
		\begin{align}
			H_r(t) &= R_{\omega_r}(t) (\omega_x(t) J_x + \omega_y(t) J_y) R_{\omega_r}(-t) + (\omega_z(t) - \omega_r)J_z +\omega_q(t) Q
		\end{align}
		It is a common technique in solving quantum mechanical problems to enter rotating frames, and their more abstract counterparts of interaction pictures~\cite[(p336,338)]{j_j_sakurai_jun_john_modern_2011}.
		This is almost always done to enable the rotating wave approximation (RWA), which is an assumption that the oscillatory components of $ H_r(t) $ on average make minor contributions to time evolution, and can be ignored.
		In some cases, this allows one to obtain analytic, but approximate, solutions to the quantum system.
		Note that the RWA is never invoked by \emph{Spinsim}, as doing this would reduce the accuracy of simulation results; although a problem already expressed in a rotating frame can of course be input by the user if desired.
		In fact, because of the speed increase of \emph{Spinsim} compared to alternatives, users who would have previously simulated their systems using the RWA due to speed concerns may now wish to use \emph{Spinsim} to simulate beyond RWA effects that would previously have been missed.

		Instead we take advantage of rotating frames to increase numerical accuracy.
		By our assumption that $ |\omega_r| \approx |\omega_z(t)| \ge |\omega_x(t)|, |\omega_y(t)|, |\omega_q(t)| $ for the duration we have entered the frame, then the rotating frame Hamiltonian will have a smaller norm than the lab frame Hamiltonian.
		This will yield more accurate results when making discrete time-steps by the integrator, as the distances stepped are smaller.

		A different rotating frame is entered by \emph{Spinsim} for each time interval $ [t_{k - 1}, t_k] $.
		This dynamic application allows for the rotating frame to be used even during sweeps of the dominant term $ \omega_z(t) $ where it is only approximately constant locally.
		Here we choose $ \omega_r = \omega_z(t_{k - 1} + \Delta t/2) $ as an approximate average value of $ \omega_z(t) $ over the duration.
		The calculated time-evolution operator $ \mathcal{U}_k^r $ is then transformed back to the lab frame as
		\begin{align}
			\mathcal{U}_k &= R_{\omega_r}(-\Delta t)\mathcal{U}_k^r.\label{eq:exit_rotating_frame}
		\end{align}
		This functionality can be disabled for systems that do not fit these criteria.

		% % By transforming into a frame rotating around this axis, a new Hamiltonian $ H^r $ with reduced norm is formed, making the time evolution operators more accurate.
		% % In numerical analysis, this is known as preconditioning [\emph{Is this right James?}]; in quantum mechanics, as transforming to the interaction picture.
		% % condition number?
		% % Use rotation matrix
		% If the rotating frame option is selected, the time-evolution operators $ \mathcal{U}_k $ are first calculated within a rotating frame of reference as $ \mathcal{U}^r_k $, which, in some situations, reduces the size of the field functions used in the calculation, increasing accuracy.
		% The rotation speed of the rotating frame is calculated locally for each parallel time-step $ \mathcal{U}_k $, and only for rotations around the $ z $ axis.
		% The rotating from field functions $ \omega^r_x, \omega^r_y, \omega^r_z, $ and $ \omega^r_q $ are related to the field function $ \omega $ from the user input via
		% \begin{align}
		% 	\omega^r_x(t) + i\omega^r_y(t) &= e^{-i \omega_r t}(\omega_x(t) + i\omega_y(t)),\\
		% 	\omega^r_z(t) &= \omega_z(t) - \omega_r\quad\text{and}\\
		% 	\omega^r_q(t) &= \omega_q(t),\quad\text{for spin-one.}
		% \end{align}

		% Where $ \omega_r = \omega_z(t_k + \frac12\Delta t) $ is sampled the midpoint value of the fields over the interval $ [t_{k - 1}, t_k] $.
		% This, assuming that a midpoint sample is representative of an average value over the time interval, decreases the magnitude of $ \omega_z(t) $, while leaving the other field components at an equivalent magnitude.
		% The rotation is then applied to obtain the lab frame time-evolution operator $ \mathcal{U}_k $ via
		% \begin{align}
		% 	\mathcal{U}_k &= \exp(-i \omega_r J_z \Delta t) \mathcal{U}^r_k.
		% \end{align}
		% Specifically, this relationship is $ \mathcal{U}_k = \text{diag}\left(\exp\left(-i \frac12 \omega_r \Delta t\right), \exp\left(i \frac12 \omega_r \Delta t\right)\right) $ for spin-half, and $ \mathcal{U}_k = \text{diag}(\exp(-i \omega_r \Delta t), 0, \exp(i \omega_r \Delta t)) $ for spin-one.

		% % Put up the top
		% It is a common technique in solving quantum mechanical problems to enter rotating frames, and their more abstract counterparts of interaction pictures~\cite{j_j_sakurai_jun_john_modern_1994}.
		% This is typically done in to enable the Rotating Wave Approximation (RWA), which is an assumption that the oscillatory components of $ \omega^r_x, \omega^r_y, \omega^r_z, $ and $ \omega^r_q $ on an average make minor contributions to time evolution, and can be ignored.
		% In some cases, this allows for analytic solutions to the approximate quantum system to be obtained.
		% Note that the RWA is \emph{not} invoked in \emph{Spinsim}, as doing this would reduce the accuracy of simulation results, defeating our purpose of using a rotating frame.
		% % Can still benefit from the rotating frame ...
		% % Can still key in RWA Hamiltonians

	% \subsubsection{Magnus-based integration method}
	% 	The integration method used in \emph{Spinsim} is the commutator free 4 (CF4) method from Auer et al~\cite{auer_magnus_2018}, which based on the Magnus expansion. Each of the $ \mathcal{U}_k $ are split into products of time evolution operators between times separated by a smaller time-step, that is,

	% 	\begin{align}
	% 		\mathcal{U}(t_k, t_{k-1}) &= \mathcal{U}(t_k, t_k - \delta t) \cdots \mathcal{U}(t_{k-1} + 2\delta t, t_{k-1} + \delta t) \mathcal{U}(t_{k-1} + \delta t, t_{k-1})\\
	% 		\mathcal{U}_k &= u^k_{L-1} \cdots u^k_0\textrm{, where}\\
	% 		u^k_{L-1} &= \mathcal{U}(t_0 + (k - 1)\Delta t + (l + 1)\delta t, t_0 + (k - 1)\Delta t + l\delta t)
	% 	\end{align}

	% 	with $ \delta t $ being the integration level time-step. Note that the time-steps are related to each other by $ \Delta t = L\delta t $, where $ L\in\mathbb{N} $.

		% The CF4 method is used to calculate each individual $ u^k_l $. Let the fine sample time be given by $ t_I = l\delta t + t_k $. Then as part of the CF4 method, the field functions are sampled at particular times based on the second order Gauss-Legendre quadrature, given by

		% \begin{align}
		% 	t_1 &= t_I + \frac12 \delta t\left(1 - \frac{1}{\sqrt{3}}\right)\textrm{, and}\\
		% 	t_2 &= t_I + \frac12 \delta t\left(1 + \frac{1}{\sqrt{3}}\right).
		% \end{align}

		% Now, let
		% \begin{align}
		% 	f(t_1) &= (f_x(t_1), f_y(t_1), f_z(t_1), f_q(t_1))\textrm{, and}
		% 	f(t_2) &= (f_x(t_2), f_y(t_2), f_z(t_2), f_q(t_2)).
		% \end{align}

		% The integration time evolution operator can then be calculated using

		% \begin{align}
		% 	\omega_1 =& (\omega_{1,x}, \omega_{1,y}, \omega_{1,z}, \omega_{1,q})\\
		% 	=& 2 \pi \delta t \left(\frac{3 + 2 \sqrt{3}}{12} f(t_1) + \frac{3 - 2 \sqrt{3}}{12} f(t_2)\right)\textrm{, and}\\
		% 	\omega_2 =& (\omega_{2,x}, \omega_{2,y}, \omega_{2,z}, \omega_{2,q})\\
		% 	=& 2 \pi \delta t \left(\frac{3 - 2 \sqrt{3}}{12} f(t_1) + \frac{3 + 2 \sqrt{3}}{12} f(t_2)\right)\textrm{, so}\\
		% 	u =& \exp(-i \left( \omega_{2,x} J_x + \omega_{2,y} J_y + \omega_{2,z} J_z + \omega_{2,q} Q\right))\\
		% 	&\cdot\exp(-i \left( \omega_{1,x} J_x + \omega_{1,y} J_y + \omega_{1,z} J_z + \omega_{1,q} Q\right)).
		% \end{align}

% \subsection{Software architecture}
	\subsection{Integrator architecture}
		\begin{figure}[h!]
			\centering
			\includegraphics[scale=0.475]{architecture.png}
			\caption{
				The mathematical and computational breakdown of \emph{Spinsim}.
				The procedural problem of evaluating the state time series $ \psi_k $ is broken into the parallel problem of evaluating time-evolution operators $ \mathcal{U}_k $.
				Each of the $ \mathcal{U}_k $ are found independently on a separate GPU thread.
				The problem is brought into a dynamic rotating frame to increase numerical accuracy.
				The $ U_k $ are broken further into time-stepping operators $ u_k $, each of which are evaluated using the Magnus CF4 method.
				Matrix exponentials are evaluated using a Lie-Trotter based method, including a residual squaring technique to avoid over-squaring.
			}
			\label{fig:architecture}
		\end{figure}
		A visual summary of the structure of \emph{Spinsim} is shown in Figure~\ref{fig:architecture}.
		The integrator in the \emph{Spinsim} package calls a kernel (multithreaded function) to be run on a \emph{Cuda} capable \emph{Nvidia} GPU in parallel, with a different thread being allocated to each of the $ \mathcal{U}_k $.
		Alternatively, the kernel can be run in parallel on a multicore CPU if a compatible GPU is not available.
		The kernel returns when each of the $ \mathcal{U}_k $ have been evaluated.

		Each thread begins with initialization of variables, which includes moving into the rotating frame.
		It then starts a loop to find each of the time-stepping operators $ u^k_l $.
		% The code then loops over each integration time-step $ \delta t $ to calculate the integration time-evolution operators $ u^k_l $.
		Within the loop, the integrator enters a device function to sample the user-provided field functions $ \omega_j(t) $ (device functions are GPU subroutines, and in practice all device functions here are compiled inline for speed).
		After this, it applies the rotating frame transformation $R_{\omega_r}(t)$, before calculating the Gauss-Legendre weightings $ \overline{H}_1 $ and $ \overline{H}_2 $. Next, the matrix exponentials are taken within another device function.
		Finally, $ u^k_l $ is premultiplied to $ \mathcal{U}^r_k $ (which is initialized to the identity), and the loop continues.

		When the loop has finished, $ \mathcal{U}^r_k $ is transformed to $ \mathcal{U}_k $ as in Equation~\eqref{eq:exit_rotating_frame}, and this is returned.
		Once all threads have executed, using Equation~\eqref{eq:integration_compilation}, the state $ \psi_k $ is calculated in a (CPU) \texttt{numba.jit()}ed function from the $ \mathcal{U}_k $ and an initial condition $ \psi_{\text{init}} = \psi(t_0) $.

	\subsection{Compilation of integrator}
		The \emph{Spinsim} integrator is constructed and compiled just-in-time (JIT), using the \emph{Numba}~\cite{lam_numba_2015} python package.
		This is achieved via the \texttt{numba.cuda.jit()} decorator, which compiles python functions into \emph{Nvidia Cuda}~\cite{nickolls_scalable_2008} kernels using an LLVM~\cite{lattner_llvm_2004} (Low Level Virtual Machine) compiler.
		The particular device functions used are not predetermined, but are instead chosen based on user input to decide on a closure.
		This technique has multiple advantages.
		First, the field functions $ \omega_j(t) $ are provided by the user as a plain python method.
		Note that this method must be \texttt{numba.cuda.jit()} compatible.
		This allows users to define $ \omega_j(t) $ in a way that compiles and executes fast, does not put many restrictions on the form of the function, and returns the accurate results of analytic functions (compared to the errors seen in interpolation).
		Compiling the simulator also allows the user to set meta-parameters, and choose the features they want to use in a way that does not require experience with the \texttt{numba.cuda} library.
		This was especially useful for running benchmarks comparing integration methods from previous versions of the software to the current one, CF4.
		The default settings should be optimal for most users, although tuning the values of \emph{Cuda} meta-parameters \texttt{max\_registers} and \texttt{threads\_per\_block} could improve performance for different GPUs.
		Third, JIT compilation also allows the user to select a target device other than \emph{Cuda} for compilation, so the simulator can run, using the same algorithm, on a multicore CPU in parallel instead of a GPU.
		In particular, this allows for compatibility with products running \emph{Apple}'s \emph{MacOS 10.14 Mojave} and later, which are incompatible with \emph{Cuda} devices and software.

		Customization and execution of the simulator is interfaced through an object of class \texttt{spinsim.Simulator}.
		The \emph{Cuda} kernel is defined as per the user’s instructions on construction of the instance, and it is used by calling the method \texttt{spinsim.Simulator.evaluate()}.
		Users are returned an instance of \texttt{spinsim.Results} including the time, state, time-evolution operator, and expected spin projection.
		Note that, to avoid making unnecessary calculations, the expected spin projection is calculated as a lazy parameter if needed rather than returned by the simulator object.

		The \emph{Spinsim} package is designed so that a single simulator instance can be used to execute many simulations, sweeping through parameters while not needing to be recompiled.
		This is done through the \texttt{sweep\_parameters[]} argument to the user-provided field functions $ \omega_j(t) $.
		% The user can set \texttt{sweep\_parameters[]} to determine parameter for the Hamiltonian.
		First the user must instantiate a \texttt{spinsim.Simulator} object.
		They can then set the value for \texttt{sweep\_parameters[]} for a particular simulation each time \texttt{spinsim.Simulator.evaluate()} is called.
		As \texttt{sweep\_parameters[]} is a \texttt{numpy.ndarray}~\cite{harris_array_2020}, one can use this functionality to sweep many parameters using the same simulator object.
		For a use-case example, an MRI experimentalist might want to simulate many spin systems at different locations $ (x, y) $, in a magnetic field gradient $ \omega_z(t) = x - 2 y $.
		To do this they could choose to set \texttt{sweep\_parameters = [x, y]}, and define \texttt{field\_sample[2] = sweep\_parameters[0] - 2*sweep\_parameters[1]}. % \linebreak
		This feature is demonstrated with examples in the documentation.

\section{Quality control}
	\subsection{Evaluation of accuracy}\label{sec:accuracy}
		All accuracy benchmarks were run in \emph{Cuda} mode, on the desktop computer with the \emph{Ryzen 7 5800X} and \emph{GeForce RTX 3080}, which from Figure~\ref{fig:benchmark_device_aggregate} are the fastest CPU and GPU from the devices tested.
		Note that these benchmarks do not take into account the amount of time required to JIT compile simulation code for the first time (order of seconds), as we want to look at this in the limit of running many simulations while sweeping through a parameter that differs in each one.

		Benchmarks were performed using \texttt{neural-sense.sim.benchmark} (where \texttt{neural-sense}~\cite{alexander-tritt-monash_alexander-tritt-monashneural-sense_2020} is the quantum sensing package that \emph{Spinsim} was written for).
		This simulation involves continuously driving transitions in the system for $ 100\,\text{ms} $, while exposing it to a $ 1\,\text{ms} $ pulsed signal that the system should be able to sense.
		The system has the Hamiltonian
		\begin{align}
			H(t) = \omega J_z + 2\Omega\cos(\omega t)J_x + \Omega_p p_\Omega(t - t_p) J_z,\label{eq:neural_pulse}
		\end{align}
		where $ \omega = 2\pi\cdot700\,\text{kHz} $, $ \Omega = 2\pi\cdot1\,\text{kHz} $,$ \Omega_p = 2\pi\cdot70\,\text{Hz} $, $ t_p = 233\,\text{ms} $ and $ p_\Omega(t) $ is a single cycle of a sine wave wth frequency $ \Omega $.

		We first wanted to test the accuracy of the different integration techniques for various integration time-steps.
		Here we wanted to test the advantages of using a Magnus-based integration method.
		Accuracy was calculated by taking the quantum state simulation evaluations of a typical quantum sensing experiment and finding the root mean squared (RMS) error from a baseline simulation run by \texttt{scipy.integrate.ivp\_solve()} as part of the \emph{SciPy} python package, via
		\begin{align}
			\epsilon &= \frac{1}{K}\sqrt{\sum_{k = 0}^{K - 1}\sum_{m_j = -j}^j|\psi_{k, (m_j)} - \psi_{k, (m_j)}^{\textrm{baseline}}|^2},\label{eq:error}
		\end{align}
		where $ j \in \{\frac12, 1\} $ is the spin quantum number of the system.
		This baseline was computed in 2.4 hours (in comparison to order of $ 100\,\text{ms} $ these \emph{Spinsim} tests were executed in), and was also used for comparisons to other software packages.

		Error vs time-step plots are shown in Figures~\ref{fig:benchmark_spin_one_step_error} and~\ref{fig:benchmark_spin_half_step_error}.
		In order to find how long a simulation method takes to complete for a given accuracy, we also measured the execution time for each of the simulations.
		These plots are shown in Figures~\ref{fig:benchmark_spin_one_execution_error} and~\ref{fig:benchmark_spin_half_execution_error}.
		The former sets of plots will be of interest to users choosing an appropriate time-step for simulations, and the latter for estimating the execution time of such simulations.
		% In all of these comparisons, errors above $ 10^{-3} $ were counted as a failed simulation, as the maximum possible error for a quantum state saturates given that it is a point on a unit complex sphere.
		In all of these comparisons, errors above $ 10^{-3} $ were counted as a failed simulation, as quantum states cannot be arbitrarily far away from each other given that they are points on a unit complex sphere.
		Errors below $ 10^{-11} $ were also excluded, as this was the order of magnitude of the errors in the reference simulation.

		The integration techniques tested were the Magnus-based CF4, as well as two Euler-based sampling methods.
		A midpoint Euler method was chosen as the simplest (and fastest for a given integration time-step) possible sampling method, whereas a Heun-Euler sampling method was used as a comparison to previous versions of this code.
		These are respectively labelled as the modified Euler method and the improved Euler method in Reference~\cite[p328]{suli_introduction_2003}.
		We benchmarked these methods both while using and not using the rotating frame option.
		This was done separately for spin-one and spin-half systems, to ensure they both yield accurate results.

		From Figure~\ref{fig:benchmark_spin}, we find that overall, the results that \emph{Spinsim} gives are accurate to those of \emph{SciPy}.
		Figures~\ref{fig:benchmark_spin_one_step_error}, and~\ref{fig:benchmark_spin_half_step_error} show that using the Magnus-based integration method is up to 3 orders of magnitude more accurate when compared the Euler-based methods.
		Also, using the rotating frame increased the accuracy here by 4 orders of magnitude for any individual integration method.
		When comparing speed vs accuracy in Figures~\ref{fig:benchmark_spin_one_execution_error}, and~\ref{fig:benchmark_spin_half_execution_error}, the advantage that CF4 gives in terms of accuracy far outweighs its slower execution speed when compared to midpoint Euler methods.
		% By default \emph{Spinsim} sets the integrator to CF4, and uses the rotating frame.
		% These can be modified using optional arguments when instantiating the \texttt{spinsim.Simulator} object.
		\begin{figure}[h!]
			\begin{subfigure}[b]{0.475\textwidth}
				\includegraphics[scale=0.475]{benchmark_spin_one_step_error.png}
				% \caption{Accuracy of the spin-one options of \emph{Spinsim}.}
				\caption{}
				\label{fig:benchmark_spin_one_step_error}
			\end{subfigure}
			\hfill
			\begin{subfigure}[b]{0.475\textwidth}
				\includegraphics[scale=0.475]{benchmark_spin_one_execution_error.png}
				% \caption{Speed vs accuracy of the spin-one options of \emph{Spinsim}.}
				\caption{}
				\label{fig:benchmark_spin_one_execution_error}
			\end{subfigure}
			\vfill
			\begin{subfigure}[b]{0.475\textwidth}
				\includegraphics[scale=0.475]{benchmark_spin_half_step_error.png}
				% \caption{Accuracy of the spin-half options of \emph{Spinsim}.}
				\caption{}
				\label{fig:benchmark_spin_half_step_error}
			\end{subfigure}
			\hfill
			\begin{subfigure}[b]{0.475\textwidth}
				\includegraphics[scale=0.475]{benchmark_spin_half_execution_error.png}
				% \caption{Speed vs accuracy of the spin-half options of \emph{Spinsim}.}
				\caption{}
				\label{fig:benchmark_spin_half_execution_error}
			\end{subfigure}
			\caption{Speed and accuracy of the spin-one and spin-half options of \emph{Spinsim}.
			A simulation of a typical sensing experiment for our lab was run for every integration time-step, for each of the possible integration techniques (Magnus-based commutator free 4, and two Euler methods).
			In the simulation, transitions are continuously driven in the spin system for a duration of $ 100 \,\text{ms} $, and an additional small signal is injected for $ 1\,\text{ms} $.
			See Equation~\eqref{eq:neural_pulse}.
			Each technique was tested while both using and not using a transformation into a rotating frame.
			Both execution time and error were recorded for each of the simulations.
			Error is RMS error compared to a long running \emph{SciPy} baseline.}
			\label{fig:benchmark_spin}
		\end{figure}
		% \begin{figure}[h!]
		% 	\begin{subfigure}[b]{0.475\textwidth}
		% 		\includegraphics[scale=0.475]{benchmark_spin_half_step_error.png}
		% 		% \caption{Accuracy of the spin-half options of \emph{Spinsim}.}
		% 		\caption{}
		% 		\label{fig:benchmark_spin_half_step_error}
		% 	\end{subfigure}
		% 	\hfill
		% 	\begin{subfigure}[b]{0.475\textwidth}
		% 		\includegraphics[scale=0.475]{benchmark_spin_half_execution_error.png}
		% 		% \caption{Speed vs accuracy of the spin-half options of \emph{Spinsim}.}
		% 		\caption{}
		% 		\label{fig:benchmark_spin_half_execution_error}
		% 	\end{subfigure}
		% 	\caption{Speed and accuracy of the spin-half options of \emph{Spinsim}.
		% 	A simulation of a typical neural sensing experiment was run for every integration time-step, for each of the possible integration techniques (Magnus-based commutator free 4, and two Euler methods).
		% 	In the simulation, transitions are continuously driven in the spin system for a duration of 100ms, and an additional small signal is injected for 1ms.
		% 	Each technique was tested while both using and not using a transformation into a rotating frame.
		% 	Both execution time and error were recorded for each of the simulations.
		% 	Error is RMS error compared to a long running \emph{SciPy} baseline.}
		% 	\label{fig:benchmark_spin_half}
		% \end{figure}

	\subsection{Comparison to alternatives}
		We ran the same error and execution time benchmarks on some alternative packages, listed in Table~\ref{tab:external}, to compare \emph{Spinsim}'s performance to theirs.
		\begin{table}[h!]
			\caption{The software packages used for and verification of \emph{Spinsim}.}
			\label{tab:external}
			\begin{tabular}{l|l|p{6.5cm}}
				\textbf{Software package}								&\textbf{Function}					&\textbf{Details}\\
				\hline
				Spinsim													&\texttt{Simulator()}				&The best performing \emph{Spinsim} configuration, using the CF4 integrator and the rotating frame mode. This was run both on CPU and GPU.
				\\
				\hline
				QuTip~\cite{johansson_qutip_2013}						&\texttt{sesolve()}					&The Schr\"odinger equation solver from the popular \emph{python} quantum mechanics library, \emph{QuTip}.
				This was chosen as a comparison to a specially designed solver used within the physics community for this application.
				Some quantum mechanics simulation packages such as the recent \emph{scqubits}~\cite{groszkowski_scqubits_2021} use \emph{QuTip} for calculating time evolution.
				Like \emph{Spinsim}, \emph{QuTip} allows users to sample from compiled functions, and uses parallelization.\\
				\hline
				Mathematica~\cite{wolfram_research_inc_mathematica_2020}	&\texttt{NDSolve()}					&A generic ODE solver from the \emph{Mathematica} software.
				This was chosen as it has been historically popular with our lab group for simulating quantum sensing experiments.\\
				\hline
				SciPy~\cite{virtanen_scipy_2020}							&\texttt{integrate.ivp\_solve()}	&A generic ODE solver from the popular \emph{python} scientific computing library.
				This was chosen as a comparison to a generic solver from within the python ecosystem.
			\end{tabular}
		\end{table}
		To obtain simulation results of different accuracies, the step sizes of the alternative integrators were limited to a maximum value.
		In some cases, the maximum number of steps was modified in some cases to allow for the smaller step sizes.
		% In each case, the step sizes of the alternative integrators were limited to a maximum value
		Apart from that, the integrator settings were left untouched from the default values, as a representation of what a user would experience using a generic solver for spin system problems.

		Similarly to the internal \emph{Spinsim} benchmarks (see \emph{\nameref{sec:accuracy}}), the expected spin projection was evaluated in each case, but only the states were compared to calculate a relative error via Equation~\eqref{eq:error}.
		Again, we used the longest running \emph{SciPy} simulation as a baseline for comparison, as the accuracy of \emph{Mathematica} plateaus at small time-steps.
		Functions used for sampling were compiled to \emph{LLVM} and \emph{cython} in \emph{Spinsim} and \emph{QuTip} respectively.
		in both cases, the time taken to complete a simulation was measured using a second simulation using the already compiled functions, to represent the use-case of sweeping through many simulations.

		The speed of only one simulation was measured for each benchmark.
		However, it might be possible to increase the average speed of many benchmarks from \emph{Mathematica} and \emph{SciPy} packages by using multithreading to run multiple simulations at a time.
		% When this was attempted using \emph{Mathematica}, the kernels crashed as the 32GiB of RAM was not enough to run them all at once.
		We attempted doing 8 way multithreading (on an 8 core \emph{Ryzen 7 5800X}) with \emph{Mathematica}, but the solver crashed due to insufficient RAM (of $ 32\,\text{GiB} $).
		Multithreading was not attempted using \emph{SciPy}, due to the fact that running the full set of benchmarks of only a single simulation per integration time-step already consumes over 11 hours of computational time.
		Regardless, for a fair comparison, both \emph{Mathematica} and \emph{SciPy} results are plotted with an artificial reduction in execution time by a factor of 8 (dotted line in Figures~\ref{fig:benchmark_spin_one_execution_error} and~\ref{fig:benchmark_spin_half_execution_error}), which is an upper bound for the speed increase that could be obtained by running them parallel on an 8 core processor.
		Simulations from \emph{Spinsim} and \emph{QuTip} automatically run multithreaded, so this comparison is not plotted for these packages.

		From Figure~\ref{fig:benchmark_external}, for any given error tolerance, \emph{Spinsim} is over 3 orders of magnitude faster than \emph{Mathematica} and \emph{QuTip}, and 4 orders of magnitude more accurate than \emph{SciPy}.
		In practice, this means that a 25 minute \emph{SciPy} simulation is reduced to 50ms, and a three week long \emph{SciPy} batch simulation of 1000 separate systems (a realistic situation for testing quantum sensing protocols) would take less than one minute in \emph{Spinsim}.
		Note that while the three week long \emph{SciPy} example could be parallelised over a computer cluster, saving some, \emph{Spinsim} allows users to avoid the need for using clusters and, given that a single \emph{SciPy} simulation still takes 25 minutes to run, the \emph{Spinsim} batch would still be faster.
		\begin{figure}[h!]
			\centering
			\includegraphics[scale=0.475]{benchmark_external_execution_error.png} % 0.9
			\caption{Speed vs accuracy of two alternative integration packages.
			A simulation of a typical sensing experiment for our lab was run for every integration time-step, for each of alternative packages (\texttt{qutip.sesolve()} from \emph{QuTip}, \texttt{NDSolve()} from \emph{Mathematica}, and \texttt{scipy.integrate.ivp\_solve()} from \emph{SciPy}).
			In the simulation, transitions are continuously driven in the spin system for a duration of $ 100\,\text{ms} $, and an additional small signal is injected for $ 1\,\text{ms} $.
			See Equation~\eqref{eq:neural_pulse}.
			Both execution time and error were recorded for each of the simulations.
			Error is RMS error compared to a long running \emph{SciPy} baseline.
			The \emph{Mathematica} and \emph{SciPy} results are also shown with a speed up by a factor of 8 to represent the upper bound of hypothetical parallelization across an 8 core CPU.}
			\label{fig:benchmark_external}
		\end{figure}

	\subsection{Parallelization performance}
		Once the algorithm behind \emph{Spinsim} was developed, we wanted to check its execution speed while running on various devices.
		The main reason for this test was to quantify the speed increase of parallelization by comparing execution speeds on highly parallel devices (being GPUs), and highly procedural devices (being CPUs).
		Speed benchmarks were performed using \texttt{neural-sense.sim.benchmark}, by comparing the evaluation speed of typical spin-one sensing experiments on different devices.
		Results are of these benchmarks are shown in Figure~\ref{fig:benchmark_device_aggregate}.
		The integration code was compiled by \texttt{numba} for multicore CPUs, CPUs running single threaded, and \emph{Nvidia Cuda} compatible GPUs.
		The compiled code was then run on different models of each of these devices.
		These test devices are given in Table~\ref{tab:devices}.

		\begin{table}[h!]
			\caption{Devices used in the parallelization speed test.
			These devices are part of individual computers, which are separated here by horizontal lines.}
			\label{tab:devices}
			\begin{tabular}{l|l|l|l|l}
				\textbf{Name}	&\textbf{Device}&\textbf{RAM (GiB)}	&\textbf{Cores}	&\textbf{Cooling}\\
				\hline
				Core i7-6700	&Intel CPU		&16					&4				&Air\\
				Quadro K620		&Nvidia GPU		&2					&384			&Air\\
				\hline
				Core i7-8750H	&Intel CPU		&16					&6				&Air\\
				GeForce GTX 1070&Nvidia GPU		&8					&2048			&Air\\
				\hline
				Core i7-10850H	&Intel CPU		&32					&6				&Air\\
				Quadro T1000	&Nvidia GPU		&4					&768			&Air\\
				\hline
				Ryzen 9 5900X	&AMD CPU		&32					&12				&Air\\
				GeForce RTX 3070&Nvidia GPU		&8					&5888			&Air\\
				\hline
				Ryzen 7 5800X	&AMD CPU		&32					&8				&Liquid\\
				GeForce RTX 3080&Nvidia GPU		&10					&8704			&Air\\
			\end{tabular}
		\end{table}
		\begin{figure}[htbp!]
			\centering
			\includegraphics[scale=0.6]{benchmark_device_aggregate.png}
			\caption{Evaluation speed of a simulation of a typical spin-one sensing experiment for our lab on both CPUs and GPUs.
			Integration time-step is set to $ 100\,\text{ns} $.
			Transitions are continuously driven in the spin system for a duration of $ 100\,\text{ms} $, and an additional small signal is injected for $ 1\,\text{ms} $.
			See Equation~\eqref{eq:neural_pulse}.
			Evaluation time is determined by an average of 100 similar simulations for each device, where each individual simulation varies in dressing amplitude (transition frequency).}
			\label{fig:benchmark_device_aggregate}
		\end{figure}

		The results in Figure~\ref{fig:benchmark_device_aggregate} show the benefit to using parallelization when solving a spin system problem.
		Moving from the 6 core \emph{Core i7-8750H} CPU to the 12 core \emph{Ryzen 9 5900X} CPU doubles the execution speed, as does moving from the 384 core \emph{Quadro K620} GPU to the 768 core \emph{Quadro T1000} GPU.
		So, in these cases, performance scales in proportion to thread count.
		Moving from a single core processor to a high end GPU increases performance by well over an order of magnitude on three of the five computers used for testing.
		Even the low end \emph{Quadro K620} was an improvement over the \emph{Core i7-6700} used by the same computer.
		Execution speed vs number of cuda cores starts to plateau as the number of cores increases.
		This happens because the time it takes to transfer memory from RAM to VRAM (dedicated graphics memory), which is independent on the number of cores of the GPU, becomes comparable to the execution time of the simulator logic.
		However, there is still a large improvement from using the high end \emph{GeForce RTX 3070} to the \emph{GeForce RTX 3080}, with the latter simulating the experiment almost twice as fast as it would take to run the simulated experiment in the real world.

		Surprisingly, we found that the \emph{Ryzen 7 5800X} 8 core CPU was able to execute the benchmark faster than the \emph{Ryzen 9 5900X} 12 core CPU.
		This can be explained by the fact that the \emph{Ryzen 7 5800X} was cooled by liquid rather than air, meaning it was likely able to boost to a higher core clock, and resist thermal throttling.

		Users can view Figure~\ref{fig:benchmark_device_aggregate} to decide on how much their current or future hardware will be able to take advantage of \emph{Spinsim}'s parallelism.
		% Figure~\ref{fig:benchmark_device_aggregate} can be used by potential \emph{Spinsim} users for finding the relative performance for devices of varying abilities of parallelization.
		Another factor not shown in the plot is that, in practice, running highly code parallel code on a CPU on a personal computer will severely limit the responsiveness of other applications, as it can utilize the entire CPU (as it should).
		In contrast, this does not happen when running a GPU based program, as it requires very little CPU utilization to function.
		This can be convenient when running simulations on a personal laptop or desktop, as other work on the computer does not have to halt while simulations are being run.

	\subsection{Testing}
		During the accuracy tests, it was confirmed that all possible modes of \emph{Spinsim} agree with a baseline \emph{SciPy} simulation, as close as the user wants up to the error of $ 10^{-11} $ of that baseline.
		Therefore, \emph{Spinsim} can be trusted for accuracy.
		The Lie Trotter matrix exponentiator was tested separately from the full system, as well as benchmarked separately against \texttt{scipy.linalg.expm()} from \emph{SciPy}.
		These tests and benchmarks were run as part of the \texttt{neural\_sense} package.
		The simulator has also been adopted by members of our lab, who have given advice on user experience.

		The kernel execution was profiled thoroughly, and changes were made to optimize VRAM and register usage and transfer.
		This was done specifically for the development hardware of the \emph{GeForce GTX 1070}, so one may get some performance increases by changing some GPU specific meta parameters when instantiating the \texttt{spinsim.Simulator} object.

		A good way to confirm that \emph{Spinsim} is functioning properly after an installation is to run the tutorial code provided and compare the outputs.
		Otherwise, one can reproduce the benchmarks shown here using \texttt{neural\_sense.sim.benchmark}.

\section{(2) Availability}
\vspace{0.5cm}
\section{Operating system}
Developed and tested on \emph{Windows 10}.
CPU functionality tested on \emph{MacOS 10.16 Big Sur} (note that \emph{MacOS 10.14 Mojave} and higher is not compatible with \emph{Cuda} hardware and software).
The package (including \emph{Cuda} functionality) is in principle compatible with Linux, but functionality has not been tested.

\section{Programming language}
Python (3.7 or greater)

\section{Additional system requirements}
To use the (default) \emph{Nvidia Cuda} GPU parallelization, one needs to have a \emph{Cuda} compatible \emph{Nvidia} GPU~\cite{noauthor_cuda_2012}.
For \emph{Cuda} mode to function, one also needs to install the \emph{Nvidia Cuda} toolkit~\cite{noauthor_cuda_2013}.
If \emph{Cuda} is not available on the system, the simulator will automatically parallelize over multicore CPUs instead.

\section{Dependencies}
numba (0.50.1 or greater)\\
numpy (1.19.3)\\
matplotlib (for example code, 3.2)\\
neuralsense (for benchmark code)

\section{List of contributors}

1. Alex Tritt\\
	School of Physics \& Astronomy, Monash University, Victoria 3800, Australia.\\
	Primary author of the released packages.\\
2. Joshua Morris\\
	School of Physics \& Astronomy, Monash University, Victoria 3800, Australia.\\
	Present address: Vienna Center for Quantum Science and Technology (VCQ), Faculty of Physics, University of Vienna, 1010 Vienna, Austria.\\
	Author of first version of code.\\
3. Joel Hochstetter\\
	School of Physics \& Astronomy, Monash University, Victoria 3800, Australia.\\
	Present address: Department of Applied Mathematics and Theoretical Physics, Centre for Mathematical Sciences, University of Cambridge, Cambridge, UK.\\
	Optimization and extension to spin-one of first version of code.\\
4. Russell P. Anderson\\
	School of Molecular Sciences, La Trobe University, PO box 199, Bendigo, Victoria 3552, Australia.\\
	Present address: Q-CTRL Pty. Ltd.\\
	Original conception of first version of code, with minor contributions to released packages.\\
5. James Saunderson\\
	Department of Electrical and Computer Systems Engineering, Monash University, Victoria 3800, Australia.\\
	Advice on numerical analysis.\\
6. Lincoln D. Turner\\
	School of Physics \& Astronomy, Monash University, Victoria 3800, Australia.\\
	Original conception of released version of algorithm.

\section{Software location:}

{\bf Archive}

\begin{description}[noitemsep,topsep=0pt]
	\item[Name:] Monash Bridges
	\item[Persistent identifier:] 10.26180/13285460
	\item[Licence:] Apache 2.0
	\item[Publisher:]  Alex Tritt
	\item[Version published:] 1.0.0
	\item[Date published:] \textcolor{blue}{dd/mm/yy}
\end{description}

{\bf Code repository}

\begin{description}[noitemsep,topsep=0pt]
	\item[Name:] GitHub
	\item[Persistent identifier:] https://github.com/alexander-tritt-monash/spinsim
	\item[Licence:] BSD 3 Clause
	\item[Date published:] 18/11/20
\end{description}

\section{Language}

English.

\section{(3) Reuse potential}

	\subsection{Applications}
		The \emph{Spinsim} package will be useful for any research group needing quick, accurate, and/or large numbers of simulations involving spin-half or spin-one systems.
		In particular, the package can simulate a system with complicated (i.e.not just hard pulses), analytic driving fields, with no approximations like the RWA required, at a high time resolution. % Frequency, amplitude and phase modulation; multiple tones; narrow-band interference; Radiation in control Hamiltonian; interesting polarisation
		We stress that many application specific simulators are designed for the paradigm of using the RWA and/or hard pulse approximations~\cite{suzuki_qulacs_2021, teske_qopt_2021, isakov_simulations_2021}, which both sacrifice accuracy for the sake of speed.
		While this is viable for some areas of physics research, when designing quantum sensing protocols, we need to take all effects of the spin system and surrounding lab environment into account.
		Thus, \emph{Spinsim}'s primary use is to fill in this gap of simulators for use of developing new quantum sensing protocols with spin-half and spin-one systems, and so should be useful to many within the field of quantum sensing.
		Moreover, \emph{Spinsim} is fast and accurate enough that there is no need to simplify the problem to compromise accuracy for speed.

		The package was written for the context of testing a particular ultracold atom magnetic sensing protocol design.
		This project aims to be able to measure neural signals using ultracold atoms.
		Electrical pulses made by neurons are currently measured using electrical probes~\cite{mitterdorfer_potassium_2002}, which is intrusive and damages the cells.
		We, and others in the field~\cite{barry_optical_2016, xu_wavelet-based_2016, parashar_axon_2020, webb_high_2021} instead propose to sense the small magnetic fields that these electrical currents produce.
		Rubidium ultracold atom clouds can potentially be made sensitive enough to these small magnetic waveforms that we can use them as sensors.
		The \emph{Spinsim} package was written to simulate possible measurement protocols for this, showing the behaviour of the array of spin-one atoms interacting with the magnetic fields of the neurons, control signals, and the lab environment.

		So far, the package has also been used to produce simulations for four other new and conceptually distinct quantum sensing protocols from members of the Monash University School of Physics \& Astronomy spinor BEC lab.
		Included in this were some honors projects, where \emph{Spinsim} allowed for many quick simulations in the limited time available. % People who have not used quantum computation before have managed to do this

		Another example of spin based magnetic field sensing is the use of nitrogen vacancy centres (NVCs).
		These are spin-one structures found in diamond, doped with Nitrogen atoms.
		This leaves a vacancy in a position adjacent to the Nitrogen atom, which pairs of electrons occupy to obtain the spin-one properties.
		Similar to ultracold atoms, NVCs can be placed and addressed in 2D arrays in order to take many samples in one measurement.
		As work first started on \emph{Spinsim}, Parashar et al~\cite{parashar_axon_2020} published results on simulation experiments of magnetic neural pulse sensing using NVCs.
		This is a field \emph{Spinsim} could be useful in.

		The \emph{Spinsim} package is designed to simulate small dimensional quantum systems, including large arrays of non-interacting spin systems.
		This means that it would not be able to integrate large arrays of entangled states or interacting particles.
		As a result, despite being fast at simulating qubits, it is inappropriate for the package to be used for quantum computing.
		In addition, \emph{Spinsim} is currently designed to integrate the time evolution of pure states only, meaning it is only accurate for the duration that the physical spin system is coherent.
		While ultracold atoms can be coherent for tens of seconds, to the point where decoherence is negligible for most quantum sensing experiments, NVCs have a much shorter coherence time of up to $ 1\,\text{ms} $~\cite{degen_quantum_2017}, so fewer simulations are accurate while ignoring decoherence.
		Not being able to model decoherence also means that \emph{Spinsim} may not be adequate for use in some nuclear magnetic resonance (NMR) applications where relaxation is important, such as those covered by \emph{SpinEvolution}~\cite{veshtort_spinevolution_2006}, for example. % Too much negativity; Long and adiabatic pulse sequences; simulate in the lab frame, end up with bloch-seigert shifts; targeted at individual quanta, separate regime

		With these restrictions in mind, \emph{Spinsim} could be used for some simplified simulations in various areas of NMR.
		There are many atomic nuclei with spins of half (eg protons, Carbon 13) and, and some that have spins of one (eg Lithium 6, Nitrogen 14)~\cite{fuller_nuclear_1976}, which, if relaxation and interactions between systems are not important for the application, \emph{Spinsim} could be used to simulate for spectroscopy experiments, for example.
		The inclusion of a quadrupole operator means that \emph{Spinsim} should be able to simulate nuclear quadrupole resonance (NQR) spectroscopy for spin-one nuclei~\cite{bain_nqr_2004}, such as Nitrogen 14, provided a suitable coordinate system is chosen.
		This technique measures energy level differences between levels split by electric field gradients, rather than static magnetic bias fields.
		Another possible use-case could be for magnetic resonance imaging (MRI) simulation and pulse sequence design.
		MRI uses measures the response of spins of an array of spin-half protons to a spatially varying pulse sequence~\cite{mckinnon_physics_1998}, which essentially just corresponds to many separate \emph{Spinsim} simulations of spins at different positions in space.
		While this package offers some advantages over state of the art simulators in the field~\cite{kose_fast_2019}, with its use of quantum mechanics over classical mechanics, and its absence of rotating wave approximations, its parametrized pulse sequence definitions and geometric integrator, again, the lack of interacting particles and decoherence features are may limit its use in this area.

		The field of Quantum Optimal Control (QOC) requires many simulations in order to optimize a pulse sequence to be robust to the noisy environment of the system, using machine learning techniques~\cite{dalgaard_fast_2021}.
		The ability to automatically differentiate~\cite{griewank_who_2012} the simulator is important for this application.
		This is not currently a feature of \emph{Spinsim}.
		However, it could be added in the future, especially since it might also be helpful for some quantum sensing applications.

	\subsection{Support}
		Documentation for \emph{Spinsim} is available on \href{https://spinsim.readthedocs.io/en/latest/}{\emph{Read the Docs} (https://spinsim.readthedocs.io/en/latest/)}.
		This documentation contains a thorough tutorial on how to use the package, and installation instructions.

		For direct support with the \emph{Spinsim} package, one can open an issue in the \emph{github} repository.
		One can also use this contact to suggest extensions to the package.
		\emph{Spinsim} is planned to be used and maintained by the Monash University School of Physics \& Astronomy spinor BEC lab into the future.

\section{Acknowledgements}

Thank you to the Monash University School of Physics \& Astronomy spinor BEC lab group. In particular, Hamish Taylor, Travis Hartley, Chris Bounds and Sam White, who have been using \emph{Spinsim} to simulate new ideas for quantum sensing protocols, and have given useful feedback of their user experience with the package.

\section{Funding statement}

% \textcolor{blue}{If the software resulted from funded research please give the funder and grant number.}
AT acknowledges support through an Australian Government Research Training Program Scholarship.
JS is the recipient of an Australian Research Council Discovery Early Career Researcher Award (project number DE210101056) funded by the Australian Government.
% JS acknowledges funding from the Australian Research Council Discovery Early Career Researcher Award (project number DE210101056) funded by the Australian Government..
LDT acknowledges funding from the Australian Research Council Linkage Project (project number LP200100082).

\section{Competing interests}

The authors declare that they have no competing interests.

\bibliography{spinsim}{}
\bibliographystyle{vancouver}

\vspace{2cm}

\rule{\textwidth}{1pt}

{ \bf Copyright Notice} \\
Authors who publish with this journal agree to the following terms: \\

Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a  \href{http://creativecommons.org/licenses/by/3.0/}{Creative Commons Attribution License} that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal. \\

Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal. \\

By submitting this paper you agree to the terms of this Copyright Notice, which will apply to this submission if and when it is published by this journal.

\end{document}


    \filetitle{moving}{Apply function to moving window of observations}{tseries/moving}

	\paragraph{Syntax}\label{syntax}

\begin{verbatim}
X = moving(X)
X = moving(X,Range,...)
\end{verbatim}

\paragraph{Input arguments}\label{input-arguments}

\begin{itemize}
\item
  \texttt{X} {[} tseries {]} - Tseries object on whose observations the
  function will be applied.
\item
  \texttt{Range} {[} numeric \textbar{} Inf {]} - Range on which the
  moving function will be applied; \texttt{Inf} means the entire range
  on which the time series is defined.
\end{itemize}

\paragraph{Output arguments}\label{output-arguments}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \texttt{X} {[} tseries {]} - Output time series.
\end{itemize}

\paragraph{Options}\label{options}

\begin{itemize}
\item
  \texttt{'function='} {[} function\_handle \textbar{} \texttt{@mean}
  {]} - Function to be applied to a moving window of observations.
\item
  \texttt{'window='} {[} numeric \textbar{} \emph{\texttt{Inf}} {]} -
  The window of observations where 0 means the current date, -1 means
  one period lag, etc. Inf means that the last n observations (including
  the current one) are used, where n is the frequency of the input data.
\end{itemize}

\paragraph{Description}\label{description}

\paragraph{Example}\label{example}


Muons and electrons identified by the PF algorithm form the starting point for the leptons to be used in analysis.
\subsubsection*{Muons}
Muons are required to have $\pT > 5$ GeV and $|\eta| < 2.4$.
Next, a requirement is made on the mini-isolation of the muon, defined as
\begin{equation}
\mathcal I_{\text{mini}} = \frac{\mathcal I}{\pT},
\end{equation}
where the isolation $\mathcal I$ is taken as the sum of all other PF candidate energies in a cone of size $R=0.4$ around the muon.
The isolation is corrected for contributions from pileup.
The \ttH analysis requires $\mathcal I_{\text{mini}} < 0.25$ to mitigate the contribution of hadronic jets misidentified as muons.
\subsubsection*{Electrons}
Electrons are required to have $\pT > 10$ GeV, $|\eta| < 2.5$, and additionally must not be in the ECAL barrel-endcap gap of $|\eta| = [1.4442, 1.566]$.
A BDT-based electron ID criteria is also employed.
The BDT is trained to distinguish prompt electrons from hadronic jets misidentified as electrons, and is trained with a variety of variables describing the electron's isolation, impact parameter, and kinematics.
The invariant mass of electrons with each photon in the event is required to have a difference of greater than 5 GeV with the mass of the \PZ, in order to reject \Zee events in which one of the electrons is reconstructed as a photon.

Finally, all leptons are required to not be overlapping with the photons in the event, requiring $\Delta R(\text{lepton}, \text{photon}) > 0.2$.