#src # DGSEM with flux differencing

# This tutorial starts with a presentation of the weak formulation of the discontinuous Galerkin
# spectral element method (DGSEM) in order to fix the notation of the used operators.
# Then, the DGSEM formulation with flux differencing (split form DGSEM) and its implementation in
# [Trixi.jl](https://github.com/trixi-framework/Trixi.jl) is shown.

# We start with the one-dimensional conservation law
# ```math
# u_t + f(u)_x = 0, \qquad t\in \mathbb{R}^+, x\in\Omega
# ```
# with the physical flux $f$.

# We split the domain $\Omega$ into elements $K$ with center $x_K$ and size $\Delta x$. With the
# transformation mapping $x(\xi)=x_K + \frac{\Delta x}{2} \xi$ we can transform the reference element
# $[-1,1]$ to every physical element. So, the equation can be restricted to the reference element using the
# determinant of the Jacobian matrix of the transformation mapping
# $J=\frac{\partial x}{\partial \xi}=\frac{\Delta x}{2}$.
# ```math
# J u_t + f(u)_{\xi} = 0, \qquad t\in \mathbb{R}^+, \xi\in [-1,1]
# ```

# ## The weak form of the DGSEM
# We consider the so-called discontinuous Galerkin spectral element method (DGSEM) with collocation.
# It results from choosing a nodal DG ansatz using $N+1$ Gauss-Lobatto nodes $\xi_i$ in $[-1,1]$
# with matching interpolation weights $w_i$, which are used for numerical integration and interpolation with
# the Lagrange polynomial basis $l_i$ of degree $N$. The Lagrange functions are created with those nodes and
# hence fulfil a Kronecker property at the GL nodes.
# The weak formulation of the DGSEM for one element is
# ```math
# J \underline{\dot{u}}(t) = - M^{-1} B \underline{f}^* + M^{-1} D^T M \underline{f}
# ```
# where $\underline{u}=(u_0, u_1, \dots, u_N)^T\in\mathbb{R}^{N+1}$ is the collected pointwise evaluation
# of $u$ at the discretization nodes and $\dot{u} = \partial u / \partial t = u_t$ is the temporal derivative.
# The nodal values of the flux function $f$ results with collocation in $\underline{f}$, since
# $\underline{f}_j=f(\underline{u}_j)$. Moreover, we got the numerical flux $f^*=f^*(u^-, u^+)$.

# We will now have a short overview over the operators we used.

# The **derivative matrix** $D\in\mathbb{R}^{(N+1)\times (N+1)}$ mimics a spatial derivation on a
# discrete level with $\underline{f}_x \approx D \underline{f}$. It is defined by $D_{ij} = l_j'(\xi_i)$.

# The diagonal **mass matrix** $M$ is defined by $M_{ij}=\langle l_j, l_i\rangle_N$ with the
# numerical scalar product $\langle \cdot, \cdot\rangle_N$ defined for functions $f$ and $g$ by
# ```math
# \langle f, g\rangle_N := \int_{-1, N}^1 f(\xi) g(\xi) d\xi := \sum_{k=0}^N f(\xi_k) g(\xi_k) w_k.
# ```
# The multiplication by $M$ matches a discrete integration
# ```math
#   \int_{-1}^1 f(\xi) \underline{l}(\xi) d\xi \approx M \underline{f},
# ```

# The **boundary matrix** $B=\text{diag}([-1, 0,..., 0, 1])$ represents an evaluation of a
# function at the boundaries $\xi_0=-1$ and $\xi_N=1$.

# For these operators the following property holds:
# ```math
#   M D + (M D)^T = B.
# ```
# This is called the summation-by-parts (SBP) property since it mimics integration by parts on a
# discrete level ([Gassner (2013)](https://doi.org/10.1137/120890144)).

# The explicit definitions of the operators and the contruction of the 1D algorithm can be found
# for instance in the tutorial [introduction to DG methods](@ref scalar_linear_advection_1d)
# or in more detail in [Kopriva (2009)](https://link.springer.com/book/10.1007/978-90-481-2261-5).

# This property shows the equivalence between the weak form and the following strong formulation
# of the DGSEM.
# ```math
# \begin{align*}
# J \underline{\dot{u}}(t)
# &= - M^{-1} B \underline{f}^* + M^{-1} D^T M \underline{f}\\[5pt]
# &= - M^{-1} B \underline{f}^* + M^{-1} (B - MD) \underline{f}\\[5pt]
# &= - M^{-1} B (\underline{f}^* - \underline{f}) - D \underline{f}
# \end{align*}
# ```
# More information about the equivalence you can find in [Kopriva, Gassner (2010)](https://doi.org/10.1007/s10915-010-9372-3).

# ## DGSEM with flux differencing
# When using the diagonal SBP property it is possible to rewrite the application of the derivative
# operator $D$ in the calculation of the volume integral into a subcell based finite volume type
# differencing formulation ([Fisher, Carpenter (2013)](https://doi.org/10.1016/j.jcp.2013.06.014)).
# We replace $D \underline{f}$ in the strong form by $2D \underline{f}_{vol}(u^-, u^+)$ with
# the consistent two-point volume flux $f_{vol}$ and receive the DGSEM formulation with flux differencing
# (split form DGSEM) ([Gassner, Winters, Kopriva (2016)](https://doi.org/10.1016/j.jcp.2016.09.013)).

# ```math
# \begin{align*}
# J \underline{\dot{u}}(t) &= - M^{-1} B (\underline{f}^* - \underline{f}) - 2D \underline{f}_{vol}(u^-, u^+)\\[5pt]
# &= - M^{-1} B (\underline{f}^* - \underline{f}_{vol}(\underline{u}, \underline{u})) - 2D \underline{f}_{vol}(u^-, u^+)\\[5pt]
# &= - M^{-1} B \underline{f}_{sur}^* - (2D - M^{-1} B) \underline{f}_{vol}\\[5pt]
# &= - M^{-1} B \underline{f}_{sur}^* - D_{split} \underline{f}_{vol}
# \end{align*}
# ```
# This formulation is in a weak form type formulation and can be implemented by using the derivative
# split matrix $D_{split}=(2D-M^{-1}B)$ and two different fluxes. We divide between the surface
# flux $f=f_{sur}$ used for the numerical flux $f_{sur}^*$ and the already mentioned volume
# flux $f_{vol}$ especially for this formulation.

# This formulation creates a more stable version of DGSEM, because it fulfils entropy stability.
# Moreover it allows the construction of entropy conserving discretizations without relying on
# exact integration. This is achieved when using a two-point entropy conserving flux function as
# volume flux in the volume flux differencing formulation.
# Then, the numerical surface flux can be used to control the dissipation of the discretization and to
# guarantee decreasing entropy, i.e. entropy stability.

# ## Implementation in Trixi
# Now, we have a look at the implementation of DGSEM with flux differencing with [Trixi.jl](https://github.com/trixi-framework/Trixi.jl).
using OrdinaryDiffEq, Trixi

# We implement a simulation for the compressible Euler equations in 2D
# ```math
# \partial_t \begin{pmatrix} \rho \\ \rho v_1 \\ \rho v_2 \\ \rho e \end{pmatrix}
# + \partial_x \begin{pmatrix} \rho v_1 \\ \rho v_1^2 + p \\ \rho v_1 v_2 \\ (\rho e +p) v_1 \end{pmatrix}
# + \partial_y \begin{pmatrix} \rho v_2 \\ \rho v_1 v_2 \\ \rho v_2^2 + p \\ (\rho e +p) v_2 \end{pmatrix}
# = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \end{pmatrix}
# ```
# for an ideal gas with ratio of specific heats $\gamma=1.4$.
# Here, $\rho$ is the density, $v_1$, $v_2$ the velocities, $e$ the specific total energy and
# ```math
# p = (\gamma - 1) \left( \rho e - \frac{1}{2} \rho (v_1^2+v_2^2) \right)
# ```
# the pressure.

gamma = 1.4
equations = CompressibleEulerEquations2D(gamma)

# As our initial condition we will use a weak blast wave from [Hennemann, Gassner (2020)](https://arxiv.org/abs/2008.12044).
# The primitive variables are defined by
# ```math
# \begin{pmatrix} \rho \\ v_1 \\ v_2 \\ p \end{pmatrix}
# = \begin{pmatrix} 1.0 \\ 0.0 \\ 0.0 \\ 1.0 \end{pmatrix} \text{if } \|x\|_2 > 0.5,\;
# \text{and } \begin{pmatrix} \rho \\ v_1 \\ v_2 \\ p \end{pmatrix}
# = \begin{pmatrix} 1.1691 \\ 0.1882 * \cos(\phi) \\ 0.1882 * \sin(\phi) \\ 1.245 \end{pmatrix} \text{else}
# ```
# with $\phi = \tan^{-1}(\frac{x_2}{x_1})$.

# This initial condition is implemented in Trixi under the name [`initial_condition_weak_blast_wave`](@ref).
initial_condition = initial_condition_weak_blast_wave

# In Trixi, flux differencing for the volume integral can be implemented with
# [`VolumeIntegralFluxDifferencing`](@ref) using symmetric two-point volume fluxes.
# First, we set up a simulation with the entropy conserving and kinetic energy preserving
# flux [`flux_ranocha`](@ref) by [Hendrik Ranocha (2018)](https://cuvillier.de/en/shop/publications/7743)
# as surface and volume flux.

# We will confirm the entropy conservation property numerically.

volume_flux = flux_ranocha # = f_vol
solver = DGSEM(polydeg=3, surface_flux=volume_flux,
               volume_integral=VolumeIntegralFluxDifferencing(volume_flux))

# Now, we implement Trixi's `mesh`, `semi` and `ode` in a simple framework. For more information please
# have a look at the documentation, the basic tutorial [introduction to DG methods](@ref scalar_linear_advection_1d)
# or some basic elixirs.
coordinates_min = (-2.0, -2.0)
coordinates_max = ( 2.0,  2.0)
mesh = TreeMesh(coordinates_min, coordinates_max,
                initial_refinement_level=5,
                n_cells_max=10_000,
                periodicity=true)

semi = SemidiscretizationHyperbolic(mesh, equations, initial_condition, solver,
                                    boundary_conditions=boundary_condition_periodic)

## ODE solvers
tspan = (0.0, 0.4)
ode = semidiscretize(semi, tspan);

# To analyse the entropy conservation of the approximation, we will use the analysis calllback
# implemented in Trixi. It provides some information about the approximation including the entropy change.
analysis_callback = AnalysisCallback(semi, interval=100);

# We now run the simulation using `flux_ranocha` for both surface and volume flux.
sol = solve(ode, RDPK3SpFSAL49(), abstol=1.0e-6, reltol=1.0e-6,
            callback=analysis_callback, save_everystep=false);
# A look at the change in entropy $\sum \partial S/\partial U \cdot U_t$ in the analysis callback
# confirms that the flux is entropy conserving since the change is about machine precision.

# We can plot the approximated solution at the time `t=0.4`.
using Plots
plot(sol)

# Now, we can use for instance the dissipative flux [`flux_lax_friedrichs`](@ref) as surface flux
# to get an entropy stable method.
using OrdinaryDiffEq, Trixi

gamma = 1.4
equations = CompressibleEulerEquations2D(gamma)

initial_condition = initial_condition_weak_blast_wave

volume_flux = flux_ranocha # = f_vol
solver = DGSEM(polydeg=3, surface_flux=flux_lax_friedrichs,
               volume_integral=VolumeIntegralFluxDifferencing(volume_flux))

coordinates_min = (-2.0, -2.0)
coordinates_max = ( 2.0,  2.0)
mesh = TreeMesh(coordinates_min, coordinates_max,
                initial_refinement_level=5,
                n_cells_max=10_000,
                periodicity=true)

semi = SemidiscretizationHyperbolic(mesh, equations, initial_condition, solver,
                                    boundary_conditions=boundary_condition_periodic)

## ODE solvers
tspan = (0.0, 0.4)
ode = semidiscretize(semi, tspan);

analysis_callback = AnalysisCallback(semi, interval=100);

# We now run the simulation using the volume flux `flux_ranocha` and surface flux `flux_lax_friedrichs`.
sol = solve(ode, RDPK3SpFSAL49(), abstol=1.0e-6, reltol=1.0e-6,
            callback=analysis_callback, save_everystep=false);
# The change in entropy confirms the expected entropy stability.

using Plots
plot(sol)

# Of course, you can use more than these two fluxes in Trixi. Here, we will give a short list
# of possible fluxes for the compressible Euler equations.
# For the volume flux Trixi provides for example [`flux_ranocha`](@ref), [`flux_shima_etal`](@ref),
# [`flux_chandrashekar`](@ref), [`flux_kennedy_gruber`](@ref).
# As surface flux you can use all volume fluxes and additionally for instance [`flux_lax_friedrichs`](@ref),
# [`flux_hll`](@ref), [`flux_hllc`](@ref).


using PyPlot,LinearAlgebra,Random
"""
```
gen_ellipse(F1:Vector{Float64},F2=Vector{Float64},
	a::Float64,p::Int,npun::Int,ntot::Int,
	tmin::Float64,tmax::Float64)
```

Generate perturbed points in a ellipse given focus F1 and F2 major axis a. It is mandatory to define the number of points `ntot` the of exacts points `p` and the number of point to compose the ellipse `npun`.

`ntot-npun` is the number of random points
`npun-p` is the number of points of the ellipse which are normally-distributed.
`tmin` and `tmax` are parameter to define limits for the figure.

## Example

```julia-repl
julia> gen_ellipse([1.0,2.0],[3.0,5.0],5.0,0,100,1000,-10,10)
```

"""
function gen_ellipse(F1,F2,a,p,npun,ntot,tmin,tmax)
	c = norm(F2-F1,2)
	if a<=c
		error("a<=c, it isn't an elipse")
	end
	b = sqrt(a^2-c^2)
	# let us assume β is the rotation angle from (1,0)
	cosβ = (F2-F1)[1]/norm(F2-F1,2)
	sinβ = sqrt(1.0-cosβ)

	center = 0.5*(F1+F2)
	rng = MersenneTwister(1234)
	x = zeros(ntot)
	y = zeros(ntot)
	θ = shuffle(rng,[0:(2*π)/npun:2*π;])

	for i=1:p
		x[i] = center[1]*cosβ+b*cos(θ[i])*cosβ+center[2]*sinβ+a*sin(θ[i])*sinβ
		y[i] = -center[1]*sinβ -b*cos(θ[i])*sinβ+center[2]*cosβ+a*sin(θ[i])*cosβ
	end
	for i=p+1:npun
		anoise = a+0.1*randn()
		bnoise = b+0.1*randn()
		x[i] = center[1]*cosβ+bnoise*cos(θ[i])*cosβ+center[2]*sinβ+anoise*sin(θ[i])*sinβ
		y[i] = -center[1]*sinβ -bnoise*cos(θ[i])*sinβ+center[2]*cosβ+anoise*sin(θ[i])*cosβ
	end

	x[npun+1:end]= tmin.+(tmax-tmin)*rand(ntot-npun)
	y[npun+1:end]= tmin.+(tmax-tmin)*rand(ntot-npun)
	plot(x,y,".")

	return [x y]
end


using Test, Plots, Parameters

Nx = 500
xm = 10.
x = LinRange(-xm,xm,Nx); y = x

# create some test data
psi = @. 10*exp(-.1*(x^2+y'^2)) + 15*exp(-((x-5)^2+y'^2)) # + randn()+im*randn()
heatmap(x,y,abs2.(psi))

abstract type Basis end
struct Oscillator <: Basis
    ω::Float64
    n::Int64
end
Oscillator(n::Int64) = Oscillator(1.0,n)
function (b::Oscillator)(x)
    @unpack ω,n = b
    T = x * ones(1, n) |> zero
    T[:,1] = @. exp(-(√ω * x)^2 / 2) * (ω / π)^(1 / 4)
    n >= 2 && (T[:,2] = @. sqrt(2 * ω) * x * T[:,1])
    for j ∈ index(b)[3:end]
        m = qnumbers(b)[j]
        T[:,j] = @. sqrt(2 / m) * (√ω * x) * T[:,j-1] - sqrt((m-1) / m) * T[:,j-2]
    end
    return T
end
index(b::Oscillator) = 1:b.n
spectrum(b::Oscillator) = (0:b.n-1)*b.ω
qnumbers(b::Oscillator) = 0:b.n-1

function hermite(x,n)
    @assert n >= 1
    h1 = exp(-x^2 / 2) * (1 / π)^(1 / 4)
    n == 1 && (return h1)
    h2 = sqrt(2) * x * h1
    n == 2 && (return h2)
    h3 = 0.0
    for j ∈ 3:n
        m = j - 1
        h3 = sqrt(2/m) * x * h2 - sqrt((m-1)/m) * h1
        h1 = h2; h2 = h3
    end
    n > 2 && (return h3)
end
hermite(x,n,ω) = hermite(√ω * x,n) * ω ^ ( 1 / 4 )

# test that we can create the basis, or any one of oscillator modes
Nv = 1:20
@time hermite.(x,Nv')
plot(x,hermite.(x,Nv'),legend=false,grid=false)
n = 20
b1 = Oscillator(n)
@time b1(x)
plot(x,b1(x),legend=false,grid=false)

plot(x,hermite.(x,1,[1. .5 .2 .1]))

# filtering cartesian data
function filterH(psi,x,N)
    H = Oscillator(1.0,N)(x)
    T = inv(H'*H)
    F̂ = T*H'*psi*H*T
    psiF = zero(psi)
        for m = 1:N, n = 1:(N+1-m)
            @. psiF += H[:,m]*F̂[m,n]*H[:,n]'
        end
    return psiF
end

n = 30
psiF = filterH(psi,x,n)
heatmap(x,y,abs2.(psiF))

# coversion to polar coords
function polar(psi,x,N)
    Nx = length(x)
    H = Oscillator(1.0,N)(x)
    T = inv(H'*H)
    F̂ = T*H'*psi*H*T
    r = LinRange(0,last(x),Nx/2 |> Int)
    θ = LinRange(0,2*pi,2*Nx)'
    psiP = zero(r*θ)
    for m = 1:N, n = 1:(N + 1 - m)
        @. psiP += hermite(r*cos(θ),n) * F̂[m,n] * hermite(r*sin(θ),m)
    end
    return psiP
end

r = LinRange(0,last(x),Nx/2 |> Int)
θ = LinRange(0,2*pi,2*Nx)'
psiP = polar(psi,x,30)

# plot in polar coordinates
heatmap(θ',r,abs2.(psiP))
dθ = θ[2]-θ[1]
dr = r[2]-r[1]
dx = x[2]-x[1]
psiPth = sum(psiP,dims=2)*dθ
plot(r,psiPth.*r)

# check total integral preserved (< 1%)
sum(psiPth.*r)*dr
sum(psiF)*dx^2

# ======== ideas for cfields
# CField:

#= something like this?
function transform(b::Oscillator)
    x,w = gausshermite(b.n)
    T = b(x)
    return x,w,T
end

xk,wk,Tk = transform(b1)
c = randn(n)
f = Tk*c
@test sum(@. wk*exp(xk^2)*abs2(f)) ≈ sum(abs2.(c))

abstract type CField end
struct YField{N} <: CField
    basis::Basis
    psi::Array{Complex{Float64},N}
end

f1 = YField(b1,randn(b1.n)+im*randn(b1.n))

=#


function readInput(filename::String)::Vector{Int}
	open(filename) do file
		return parse.(Int, split(readline(file), ","))
	end
end

function centroid(vec::Vector{<:Real}; weight_fun::Function)::Int
	min_ = typemax(Int)
	for i in minimum(vec):maximum(vec)
		min_ = min(min_, sum(weight_fun.(abs.(vec .- i))))
	end
	return min_
end

part1(vec) = sum(abs.(vec .- round(Int, mean(vec))))
part2(vec) = centroid(vec, weight_fun=n -> sum(1:n))

vec = readInput("input.txt")
println(part1(vec))
println(part2(vec))


using OrdinaryDiffEq, DiffEqMonteCarlo, Distributions, Test

include("koopman.jl")

function f(du,u,p,t)
  du[1] = dx = p[1]*u[1] - u[1]*u[2]
  du[2] = dy = -3*u[2] + u[1]*u[2]
end

u0 = [1.0;1.0]
tspan = (0.0,10.0)
p = [1.5]
prob = ODEProblem(f,u0,tspan,p)
sol = solve(remake(prob,u0=u0),Tsit5())
cost(sol) = sum(max(x[1]-12,0) for x in sol.u)
u0s = [Uniform(0.25,5.5),Uniform(0.25,5.5)]
ps  = [Uniform(0.5,2.0)]
@time c1, _ = koopman_cost(u0s, ps, cost, prob, Tsit5();saveat=0.1, use_vegas = true)
@time c2 = montecarlo_cost(u0s, ps, cost, prob, Tsit5(); num_monte = 100000, saveat = 0.1)
@show c1, c2

#=function f2(du,u,p,t)
  du[1] = dx = p[1]*u[1] - p[2]*u[1]*u[2]
  du[2] = dy = -p[3]*u[2] + p[4]*u[1]*u[2]
end

u0 = [1.0;1.0]
tspan = (0.0,10.0)
p = [1.5,1.0,3.0,1.0]
prob = ODEProblem(f2,u0,tspan,p)
cost(sol) = sum(max(x[1]-6,0) for x in sol.u)
u0s = [Uniform(0.25,5.5),Uniform(0.25,5.5)]
ps  = [Uniform(0.5,2.0), Uniform(0.5, 1.5), Uniform(2.5, 3.5), Uniform(0.5, 1.5)]
@time c1, _ = koopman_cost(u0s, ps, cost, prob, Tsit5();saveat=0.1, use_vegas = true)
@time c2 = montecarlo_cost(u0s, ps, cost, prob, Tsit5(); num_monte = 100000, saveat = 0.1)
@show c1, c2
=#


"""
mutable struct InverseDistanceSurrogate{X,Y,P,L,U} <: AbstractSurrogate

The square polynomial model can be expressed by 𝐲 = 𝐗β + ϵ, with β = 𝐗ᵗ𝐗⁻¹𝐗ᵗ𝐲
"""
mutable struct SecondOrderPolynomialSurrogate{X,Y,B,L,U} <: AbstractSurrogate
    x::X
    y::Y
    β::B
    lb::L
    ub::U
end

function SecondOrderPolynomialSurrogate(x,y,lb::Number,ub::Number)
    n = length(x)
    d = 1
    X = ones(eltype(x[1]),n,3*d)
    X[:,2] = x
    X[:,3] = x.^2
    β = X\y
    return SecondOrderPolynomialSurrogate(x,y,β,lb,ub)
end

function (sec_ord::SecondOrderPolynomialSurrogate)(val::Number)
    return sec_ord.β[1] + sec_ord.β[2]*val + sec_ord.β[3]*val^2
end

function SecondOrderPolynomialSurrogate(x,y,lb,ub)
    n = length(x)
    d = length(lb)
    X = ones(eltype(x[1]),n,3*d)
    for j = 1:d
        X[:,j+1] =[x[i][j] for i=1:n]
    end
    for j = 1:d-1
        X[:,j+d+1] = [x[i][j]*x[i][j+1] for i = 1:n]
    end
    for j = 1:d
        X[:,j+2*d] = [x[i][j]^2 for i=1:n]
    end
    β = X\y
    return SecondOrderPolynomialSurrogate(x,y,β,lb,ub)
end

function (my_second_ord::SecondOrderPolynomialSurrogate)(val)
    #just create the val vector as X and multiply
    d = length(val)
    X = [[one(eltype(val[1]))]; [val[j] for j =1:d]; [val[j]*val[j+1] for j = 1:d-1]; [val[j]^2 for j = 1:d]]
    return my_second_ord.β'*X
end

function add_point!(my_second::SecondOrderPolynomialSurrogate,x_new,y_new)
    d = length(my_second.lb)
    if d == 1
        #1D
        my_second.x = vcat(my_second.x,x_new)
        my_second.y = vcat(my_second.y,y_new)
        n = length(my_second.x)
        X = ones(eltype(my_second.x[1]),n,3*d)
        X[:,2] = my_second.x
        X[:,3] = my_second.x.^2
        my_second.β = X\my_second.y
    else
        #ND
        my_second.x = vcat(my_second.x,x_new)
        my_second.y = vcat(my_second.y,y_new)
        n = length(my_second.x)
        X = ones(eltype(my_second.x[1]),n,3*d)
        for j = 1:d
            X[:,j+1] =[my_second.x[i][j] for i=1:n]
        end
        for j = 1:d-1
            X[:,j+d+1] = [my_second.x[i][j]*my_second.x[i][j+1] for i = 1:n]
        end
        for j = 1:d
            X[:,j+2*d] = [my_second.x[i][j]^2 for i=1:n]
        end
        my_second.β = X\my_second.y
    end
    nothing
end


"""
simple!(FV,x)
This is the function for Figure 2.1
It also shows up in CI
"""
function simple!(FV, x)
    FV[1] = x[1] * x[1] + x[2] * x[2] - 2.0
    FV[2] = exp(x[1] - 1) + x[2] * x[2] - 2.0
end

function jsimple!(JacV, FV, x)
    JacV[1, 1] = 2.0 * x[1]
    JacV[1, 2] = 2.0 * x[2]
    JacV[2, 1] = exp(x[1] - 1)
    JacV[2, 2] = 2 * x[2]
end

"""
JVsimple(v, FV, x)

Jacobian-vector product for simple!. There is, of course, no reason
to use Newton-Krylov for this problem other than CI or demonstrating
how to call nsoli.jl.
"""
function JVsimple(v, FV, x)
    jvec = zeros(2)
    jvec[1] = 2.0 * x' * v
    jvec[2] = v[1] * exp(x[1] - 1.0) + 2.0 * v[2] * x[2]
    return jvec
end


@testset "basic predicate tests" begin
    pred1 = Predicate((x,y) -> x == y)
    pred2 = Predicate((x,y) -> x != y)
    pred3 = Predicate((x,y) -> all(x .== -y))
    @test pred1(1,1)
    @test !pred1(1,2)
    @test !pred2(1,1)
    @test pred2(1,2)
    @test pred3([1,2],[-1,-2])
    @test !pred3([1,2],[2,3])
end

@testset "is predicate equivalence relation" begin
    pred1 = Predicate((x,y) -> x == y)
    pred2 = Predicate((x,y) -> x != y)
    pred3 = Predicate((x,y) -> all(x .== -y))
    pred4 = Predicate((x,y) -> x == y+3)
    @test check_for_equivalence_relation(pred1, -11:10)
    @test check_for_equivalence_relation(pred2, -7:10)
    @test check_for_equivalence_relation(pred3, [[i,j] for i in -7:10 for j in -2:8])
    @test !check_for_equivalence_relation(pred4, -7:10)
end

@testset "mappings" begin
    m1 = Mapping(x -> [x])
    m2 = Mapping(x -> [-x])
    sol2 = Dict(-4 => 1, -3 => 2, 3 => 2, -2 => 3, 2 => 3, -1 => 4, 1 => 4, 0 => 5)
    m3 = Mapping(x -> [x+3,x-3])
    m5 = Mapping(x -> [x-3])
    sol3 = Dict(-4 => 1, -1 => 1, 2 => 1, -3 => 2, 0 => 2, 3 => 2, -2 => 3, 1 => 3)
    @test find_classes(m1, 1:5) == Dict(zip(1:5,collect(1:5)))
    @test find_classes(m2, -4:3) == sol2
    @test find_classes(m3, -4:3) == sol3
    @test !(find_classes(m5, -4:3) == sol3)
end


export Spindle

"""
`Spindle()` returns the Moser spindle graph. This is a seven-vertex
unit distance graph with chromatic number equal to 4.
"""
function Spindle()
    G = IntGraph(7)
    edges = [
        (1, 2),
        (1, 3),
        (2, 3),
        (2, 4),
        (3, 4),
        (1, 5),
        (1, 6),
        (5, 6),
        (5, 7),
        (6, 7),
        (4, 7) ]
    add_edges!(G,edges)

    d = Dict{Int,Vector}()
    a = sqrt(3)/2

    pts = [ 0 1/2 -1/2 0 ; 0 a a 2a ]

    theta = acos(5/6)/2
    R = [ cos(theta) -sin(theta); sin(theta) cos(theta) ]

    p1 = R*pts
    for k=1:4
        d[k] = p1[:,k]
    end

    p2 = R'*pts
    for k=5:7
        d[k] = p2[:,k-3]
    end
    embed(G,d)
    SimpleGraphs.name(G,"Moser Spindle")
    return G
end


# ---
# title: 1524. Number of Sub-arrays With Odd Sum
# id: problem1524
# author: Tian Jun
# date: 2020-10-31
# difficulty: Medium
# categories: Array, Math
# link: <https://leetcode.com/problems/number-of-sub-arrays-with-odd-sum/description/>
# hidden: true
# ---
#
# Given an array of integers `arr`. Return _the number of sub-arrays_ with
# **odd** sum.
#
# As the answer may grow large, the answer  **must be**  computed modulo `10^9 +
# 7`.
#
#
#
# **Example 1:**
#
#
#
#     Input: arr = [1,3,5]
#     Output: 4
#     Explanation: All sub-arrays are [[1],[1,3],[1,3,5],[3],[3,5],[5]]
#     All sub-arrays sum are [1,4,9,3,8,5].
#     Odd sums are [1,9,3,5] so the answer is 4.
#
#
# **Example 2:**
#
#
#
#     Input: arr = [2,4,6]
#     Output: 0
#     Explanation: All sub-arrays are [[2],[2,4],[2,4,6],[4],[4,6],[6]]
#     All sub-arrays sum are [2,6,12,4,10,6].
#     All sub-arrays have even sum and the answer is 0.
#
#
# **Example 3:**
#
#
#
#     Input: arr = [1,2,3,4,5,6,7]
#     Output: 16
#
#
# **Example 4:**
#
#
#
#     Input: arr = [100,100,99,99]
#     Output: 4
#
#
# **Example 5:**
#
#
#
#     Input: arr = [7]
#     Output: 1
#
#
#
#
# **Constraints:**
#
#   * `1 <= arr.length <= 10^5`
#   * `1 <= arr[i] <= 100`
#
#
## @lc code=start
using LeetCode

## add your code here:
## @lc code=end


using MLBase
using Base.Test

# standardize

X = rand(5, 8)

(Y, t) = standardize(X; center=false, scale=false)
@test isa(t, Standardize)
@test isempty(t.mean)
@test isempty(t.scale)
@test isequal(X, Y)
@test_approx_eq transform(t, X[:,1]) Y[:,1]

(Y, t) = standardize(X; center=false, scale=true)
@test isa(t, Standardize)
@test isempty(t.mean)
@test length(t.scale) == 5
s = sqrt(sum(abs2(X), 2) ./ (8 - 1))
@test_approx_eq Y X ./ s
@test_approx_eq transform(t, X[:,1]) Y[:,1]

(Y, t) = standardize(X; center=true, scale=false)
@test isa(t, Standardize)
@test length(t.mean) == 5
@test isempty(t.scale)
@test_approx_eq Y X .- mean(X, 2)
@test_approx_eq transform(t, X[:,1]) Y[:,1]

(Y, t) = standardize(X; center=true, scale=true)
@test isa(t, Standardize)
@test length(t.mean) == 5
@test length(t.scale) == 5
@test_approx_eq Y (X .- mean(X, 2)) ./ std(X, 2)
@test_approx_eq transform(t, X[:,1]) Y[:,1]


##Univariate kerneldensity and regression
using Distributions
Random.seed!(2017);

linreg(x, y) = hcat(fill!(similar(x), 1), x) \ y

x=rand(Normal(10), 500)
xeval=range(minimum(x), stop=maximum(x), length=100)
h = bwlscv(x, gaussiankernel)
@test h>0
denvalues=kerneldensity(x, xeval=xeval)
@test all(denvalues .>= 0)

y=2 .* x.^2 .+ rand(Normal(0,5),500)
b0, b1 = linreg(x, y)
regfit = b0 .+ b1 .* x
yfit0=localconstant(x, y, xeval=x)
@test sum(abs2, regfit .- y) > sum(abs2, yfit0.-y)

yfit1=locallinear(x, y, xeval=x)
@test sum(abs2, regfit .- y) > sum(abs2, yfit1.-y)

yfit0=npr(x, y, xeval=x, reg=localconstant)
@test sum(abs2, regfit .- y) > sum(abs2, yfit0.-y)

yfit1=npr(x, y, xeval=x, reg=locallinear)
@test sum(abs2, regfit .- y) > sum(abs2, yfit1.-y)

#confidence band
yfit1=npr(x, y, xeval=xeval, reg=locallinear)
cb=bootstrapCB(x, y, xeval=xeval)
@test mean(vec(cb[1,:]) .<= yfit1 .<= vec(cb[2,:])) > .8

#multivariate density estimation
x = rand(Normal(10), 500, 3)
denvalues = kerneldensity(x, h = [1.0, 1.0, 1.0])
@test all(denvalues .>= 0)

#multivariate regression
x = rand(Normal(10), 500, 3)
y = x * ones(3) .+ x.^2 *ones(3)
yfit1 = localconstant(x, y, xeval=x)
regfit = x*inv(x'*x)*x'*y
@test sum(abs2, regfit .- y) > sum(abs2, yfit1.-y)

###Bounded gamma kernel density and regression
x = rand(Gamma(4,2), 500)
xeval = range(0.01, stop=20, length=100)
h = bwlscv(x, gammakernel)
@test h>0
denvalues = kerneldensity(x, xeval=xeval, kernel=gammakernel, lb=0.0)
@test all(denvalues .> 0)

y=2 .* x.^2 + x.*rand(Normal(0, 5), 500)
b0, b1 = linreg(x, y)
regfit = b0 .+ b1 .* x
yfit0=npr(x, y, xeval=x, reg=localconstant, kernel=gammakernel,lb=0.0)
@test sum(abs2, regfit .- y) > sum(abs2, yfit0.-y)

yfit1=npr(x, y, xeval=x, reg=locallinear, kernel=gammakernel, lb=0.0)
@test sum(abs2, regfit .- y) > sum(abs2, yfit1.-y)

#bounded beta kernel density and regression
x = rand(Beta(4,2), 500) * 10
xeval = range(0, stop=10, length=100)
h = bwlscv(x./10, betakernel)
@test h>0
denvalues=kerneldensity(x, xeval=xeval, kernel=betakernel,h=h, lb=0.0,ub=10.0)
@test all(denvalues .> 0)

##Bounded local regression
y=2 .* x.^2 + x.*rand(Normal(0, 5), 500)
b0, b1 = linreg(x, y)
regfit = b0 .+ b1 .* x
yfit0=npr(x, y, xeval=x, reg=localconstant, kernel=betakernel,lb=0.0, ub=10.0)
@test sum(abs2, regfit .- y) > sum(abs2, yfit0.-y)

yfit1=npr(x, y, xeval=x, reg=locallinear, kernel=betakernel, lb=0.0,ub=10.0)
@test sum(abs2, regfit .- y) > sum(abs2, yfit1.-y)

yfit1=npr(x, y, xeval=xeval, reg=locallinear, kernel=betakernel, lb=0.0,ub=10.0)
cb=bootstrapCB(x, y, xeval=xeval,reg=locallinear, kernel=betakernel, lb=0.0,ub=10.0)
@test mean(vec(cb[1,:]) .<= yfit1 .<= vec(cb[2,:])) > .8


#=

By listing the first six prime numbers: 2, 3, 5, 7, 11, and 13, we can see that the 6th prime is 13.
What is the 10 001st prime number?

=#

is_prime(n) = !any(n%i == 0 for i in 2:BigInt(floor(sqrt(n))))

function nth_prime(n)
    counter = 1
    i = 2
    while true
        if is_prime(i)
            #println(counter, ", ", i)
            if counter == n
                #println()
                return i
            end
            counter += 1
        end
        i += 1
    end
end

nth_prime(6)

nth_prime(10001)


### A Pluto.jl notebook ###
# v0.18.4

using Markdown
using InteractiveUtils

# ╔═╡ 1eb34f10-b4b3-11ec-19f5-c7f26108d374
begin
	using LinearAlgebra
	using StatsPlots
md"""
# HW 11

Eric Nguyen
"""
end

# ╔═╡ 6c4b3bf7-26e9-4d6f-9305-65fbc851818f
let
md"""
### Problem 1

Use the properties of determinants to prove that ``\underline{\underline{A}} \,\underline{\underline{B}} = \underline{\underline{0}}`` implies that either ``|\underline{\underline{A}}| = 0`` or ``|\underline{\underline{B}}| = 0``, or both, where ``\underline{\underline{0}}`` is the matrix of all zeros.
Demonstrate this with the matrices

$\underline{\underline{A}} = \begin{bmatrix} 1 & 1 \\ 2 & 2 \end{bmatrix} \qquad\text{and}\qquad \underline{\underline{B}} = \begin{bmatrix} a & b \\ -a & -b \end{bmatrix}$

where ``a`` and ``b`` can be any complex numbers.
Which matrix has zero determinant?
"""
let
	A = [1 1; 2 2]
	a = rand(Complex{Float64})
	b = rand(Complex{Float64})

	B = [a b; -a -b]
	det(A), det(B), eigvals(A), eigvals(B)
end
md"""
### Problem 1

Given that ``|\underline{\underline{A}} \,\underline{\underline{B}}| = 0`` and using the sixth property of determinants as listed in the course notes, ``|\underline{\underline{A}} \,\underline{\underline{B}}| = |\underline{\underline{A}}| |\underline{\underline{B}}|``, we get

$|\underline{\underline{A}} \,\underline{\underline{B}}| = |\underline{\underline{A}}| |\underline{\underline{B}}| = 0$

which implies that at least ``|\underline{\underline{A}}| = 0`` or ``|\underline{\underline{B}}| = 0``.
The determinant ``|\underline{\underline{A}}|`` is

$|\underline{\underline{A}}| = (1)(2) - (1)(2) = 2 - 2 = 0$

so ``\underline{\underline{A}}`` is the matrix with the zero determinant.
"""
end

# ╔═╡ 2743185f-b0d7-4195-8e8e-a1bfcf132934
let
md"""
### Problem 2

Follow the procedure we went through in class to find the symmetry axes of the conic section ``6x^2 + 12xy + y^2 = 16``, and find the angle they make with the ``x,y`` axes.
What kind of curve is this?
A plot would be helpful.
You can do this with MATHEMATICA if you want, but the necessary algebra is rather simple.
"""
md"""
### Problem 2

$\underline{\underline{A}} = \begin{bmatrix} 6 & 6 \\ 6 & 1 \end{bmatrix} \implies \underline{\underline{R}} = \frac{1}{\sqrt{13}} \begin{bmatrix} -2 & 3 \\ 3 & 2 \end{bmatrix} \implies \underline{\underline{R}}^T \underline{\underline{A}} \,\underline{\underline{R}} = \begin{bmatrix} -3 & 0 \\ 0 & 10 \end{bmatrix}$
"""
end

# ╔═╡ 982fab25-1414-4789-bc16-343ab29b4514
let
	A = [6 6; 6 1]
	v1 = eigvecs(A)[:,1]
	v2 = eigvecs(A)[:,2]
	R = [v1 v2]
	R' * A * R
end

# ╔═╡ 7e53c83c-ead7-484f-8762-4a0956ba7210
md"""
This produces the curve

$-3x^2 + 10y^2 = 16$

which is a hyperbola
"""

# ╔═╡ c8977146-1409-4eb1-bede-13436fd7214b
let
	plot(x -> sqrt(16 + 3x^2) / 10, color=:black, legend=false)
	plot!(x -> -sqrt(16 + 3x^2) / 10, color=:black, legend=false)
end

# ╔═╡ 201fb421-39b1-47db-8dec-609706461dca
let
md"""
### Problem 3

Find the eigenvalues and eigenvectors for the matrix

$\underline{\underline{\sigma}}_y = \begin{bmatrix} 0 & -1 \\ i & 0 \end{bmatrix}$

one of the three **Pauli matrices**.
Do this by hand, not with MATHEMATICA.
Normalize the eigenvectors and show that they are orthogonal.
"""
md"""
### Problem 3

First, we find the characteristic polynomial.

$\det(\underline{\underline{\sigma}}_y - \lambda \,\underline{\underline{I}}) = \begin{vmatrix} -\lambda & -i \\ i & -\lambda \end{vmatrix} = \lambda^2 + i^2 = 0 \implies \lambda = ±1$

The eigenvalues are

$\lambda^{(1)} = -1 \qquad \lambda^{(2)} = 1$

For ``\lambda^{(1)} = -1`` we have

$\begin{align*}
-v_1 - i v_2 &= 0 \\
i v_1 - v_2 &= 0
\end{align*}$

$\implies v_1 = 1, \, v_2 = i$

giving the normalized eigenvector

$\underline{v} = \frac{1}{\sqrt{2}} \begin{bmatrix} 1 \\ i \end{bmatrix}$

For ``\lambda^{(2)} = 1`` we have

$\begin{align*}
u_1 - i u_2 &= 0 \\
i u_1 - u_2 &= 0
\end{align*}$

$\implies u_1 = 1, \, u_2 = i$

giving the normalized eigenvector

$\underline{u} = \frac{1}{\sqrt{2}} \begin{bmatrix} 1 \\ -i \end{bmatrix}$

We can show that the eigenvectors are orthogonal:

$⟨v \vert u⟩ = v_i u_i^* = \left(\frac{1}{\sqrt{2}}\right)^2 + \left(\frac{i}{\sqrt{2}}\right)^2 = \frac{1}{\sqrt{2}} - \frac{1}{\sqrt{2}} = 0$
"""
end

# ╔═╡ e12e34c7-90db-4f31-8792-ce2a39acc435
let
md"""
### Problem 4

Find the eigenvalues, two of which equal each other, of the real symmetric matrix

$\underline{\underline{A}} = \begin{bmatrix} 13 & 4 & -2 \\ 4 & 13 & -2 \\ -2 & -2 & 10 \end{bmatrix}$

Construct the three eigenvectors by hand, not with MATHEMATICA.
You will have more freedom than you would have thought.
Do you see how to use this freedom to make all three eigenvectors mutually orthogonal?
"""
md"""
### Problem 4

We first calculate the characteristic equation

$\det(\underline{\underline{A}} - \lambda \, \underline{\underline{I}}) = \begin{vmatrix} 13 - \lambda & 4 & -2 \\ 4 & 13 - \lambda & -2 \\ -2 & -2 & 10 - \lambda \end{vmatrix} = -(\lambda - 18)(\lambda - 9)^2$

So the eigenvalues are

$\lambda^{(1)} = 9 \qquad \lambda^{(2)} = 18$

For ``\lambda^{(1)} = 9`` we have

$\begin{align*}
4v_1 + 4v_2 - 2v_3 &= 0 \\
4v_1 + 4v_2 - 2v_3 &= 0 \\
-2v_1 - 2v_2 + v_3 &= 0
\end{align*}$

$\implies v_1 = 1, v_2 = 0, v_3 = 2 \qquad \text{or} \qquad v_1 = -1, v_2 = 1, v_3 = 0$

which gives the normalized eigenvectors

$\underline{v} = \frac{1}{\sqrt{5}} \begin{bmatrix} 1 \\ 0 \\ 2 \end{bmatrix} \qquad \underline{u} = \frac{1}{\sqrt{2}} \begin{bmatrix} -1 \\ 1 \\ 0 \end{bmatrix}$

For ``\lambda^{(2)} = 18`` we have

$\begin{align*}
-5w_1 + 4w_2 - 2w_3 &= 0 \\
4w_1 - 5w_2 - 2w_3 &= 0 \\
-2w_1 - 2w_2 - 8w_3 &= 0
\end{align*}$

$\implies w_1 = -2, w_2 = -2, w_3 = 1$

which gives the normalized eigenvector

$\underline{w} = \frac{1}{3} \begin{bmatrix} -2 \\ -2 \\ 1 \end{bmatrix}$
"""
end

# ╔═╡ 1aaa8364-b72e-4128-b26a-82d4d4601634
let
md"""
### Problem 5

Find the eigenfrequencies and eigenmodes for the mechanical system

`|~~~k~~~[ m ]~~~k~~~[ m ]~~~k~~~[ m ]~~~k~~~|`

Make a plot that shows the motions of each of the three masses, for the three sets of initial conditions where the masses start at rest with position given by each of the three eigenvectors.
Briefly describe the motions of the three masses, for each of the eigenmodes.
"""
md"""
### Problem 5

The coupled differential equations can be written as

$\begin{align*}
mẍ_1 &= -kx_1 + k(x_2 - x_1) = -2kx_1 + kx_2 \\
mẍ_2 &= -k(x_2 - x_1) + k(x_3 - x_2) = kx_1 - 2kx_2 + kx_3 \\
mẍ_3 &= -k(x_3 - x_2) + k(-x_3) = kx_2 - 2kx_3
\end{align*}$

We can rewrite them using ``\omega_0 = \sqrt{k / m}``,

$\begin{align*}
ẍ_1 &= -2{\omega_0}^2 x_1 + {\omega_0}^2 x_2 \\
ẍ_2 &= {\omega_0}^2 x_1 - 2{\omega_0}^2 x_2 + {\omega_0}^2 x_3 \\
ẍ_3 &= {\omega_0}^2 x_2 - 2{\omega_0}^2 x_3
\end{align*}$

In terms of vectors and matrices, we can write this as

$\underline{ẍ}(t) = -{\omega_0}^2 \underline{\underline{Ω}} \,\underline{x}(t)$

where

$\underline{x} = \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} \qquad Ω = \begin{bmatrix} -2 & 1 & 0 \\ 1 & -2 & 1 \\ 0 & 1 & -2 \end{bmatrix}$

with the help of MATHEMATICA, we find the eigenfrequencies to be

$\omega_1 = \sqrt{(2 - \sqrt{2}) \omega_0}, \, \omega_2 = \sqrt{2\omega_0}, \, \omega_3 = \sqrt{(2 + \sqrt{2}) \omega_0}$

and the normalized eigenmodes to be

$\underline{a}^{(1)} = \frac{1}{2} \begin{bmatrix} 1 \\ -\sqrt{2} \\ 1 \end{bmatrix} \qquad \underline{a}^{(2)} = \frac{1}{\sqrt{2}} \begin{bmatrix} -1 \\ 0 \\ 1 \end{bmatrix} \qquad \underline{a}^{(3)} = \frac{1}{2} \begin{bmatrix} 1 \\ \sqrt{2} \\ 1 \end{bmatrix}$
"""
end

# ╔═╡ 7387c353-51b7-402b-a500-adec0a34b059
let
	Ω = [-2 1 0; 1 -2 1; 0 1 -2]
	eigen(Ω)
end

# ╔═╡ b40838d0-8391-4f1e-a3fc-a4be028b7bc7
(2 - sqrt(2))

# ╔═╡ ec26fefb-18d1-4ac7-b0c1-0d3c71ae1305
begin
	ω₁ = sqrt(2 - sqrt(2))
	ω₂ = sqrt(2)
	ω₃ = sqrt(2 + sqrt(2))

	a1 = (1/2) .* [1; -sqrt(2); 1]
	a2 = (1/sqrt(2)) .* [-1; 0; 1]
	a3 = (1/2) .* [1; sqrt(2); 1]

	s1(t, A, ϕ) = A .* a1 .* cos(ω₁ * t + ϕ)
	s2(t, A, ϕ) = A .* a2 .* cos(ω₂ * t + ϕ)
	s3(t, A, ϕ) = A .* a3 .* cos(ω₃ * t + ϕ)
end;

# ╔═╡ 355540ec-db9b-4ce5-a5d2-0c10497da934
let
	x1(t) = s1(t, 1, 0)[1]
	x2(t) = s1(t, 1, 0)[2]
	x3(t) = s1(t, 1, 0)[3]
	plot(x1, label="x₁")
	plot!(x2, label="x₂")
	plot!(x3, label="x₃")
	title!("Eigenmode 1")
end

# ╔═╡ de44b0c7-eee4-49ed-9264-40a5339e7d6c
md"""
Eigenmode 1: x1 and x3 oscillate with each other, opposite to x2
"""

# ╔═╡ 6f073f29-39b5-496e-b644-45051c81ab54
let
	x1(t) = s2(t, 1, 0)[1]
	x2(t) = s2(t, 1, 0)[2]
	x3(t) = s2(t, 1, 0)[3]
	plot(x1, label="x₁")
	plot!(x2, label="x₂")
	plot!(x3, label="x₃")
	title!("Eigenmode 2")
end

# ╔═╡ 5500a012-48fd-4227-a70f-515425c5122a
md"""
Eigenmode 2: x1 and x3 oscillate against each other while x2 remains in place
"""

# ╔═╡ e0a8d8ea-e5bd-48ca-b44f-4b6625181a17
let
	x1(t) = s3(t, 1, 0)[1]
	x2(t) = s3(t, 1, 0)[2]
	x3(t) = s3(t, 1, 0)[3]
	plot(x1, label="x₁")
	plot!(x2, label="x₂")
	plot!(x3, label="x₃")
	title!("Eigenmode 3")
end

# ╔═╡ cc32c2a5-d377-4a77-8477-b3acc63cca3b
md"""
Eigenmode 3: x1, x2, and x3 all oscillate with each other, however x2 oscillation is amplified
"""

# ╔═╡ 00000000-0000-0000-0000-000000000001
PLUTO_PROJECT_TOML_CONTENTS = """
[deps]
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
StatsPlots = "f3b207a7-027a-5e70-b257-86293d7955fd"

[compat]
StatsPlots = "~0.14.33"
"""

# ╔═╡ 00000000-0000-0000-0000-000000000002
PLUTO_MANIFEST_TOML_CONTENTS = """
# This file is machine-generated - editing it directly is not advised

julia_version = "1.8.0-beta1"
manifest_format = "2.0"
project_hash = "94b1ff0197986fb39c41ec4fbe1e3c63bdbc66ff"

[[deps.AbstractFFTs]]
deps = ["ChainRulesCore", "LinearAlgebra"]
git-tree-sha1 = "6f1d9bc1c08f9f4a8fa92e3ea3cb50153a1b40d4"
uuid = "621f4979-c628-5d54-868e-fcf4e3e8185c"
version = "1.1.0"

[[deps.Adapt]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "af92965fb30777147966f58acb05da51c5616b5f"
uuid = "79e6a3ab-5dfb-504d-930d-738a2a938a0e"
version = "3.3.3"

[[deps.ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"
version = "1.1.1"

[[deps.Arpack]]
deps = ["Arpack_jll", "Libdl", "LinearAlgebra", "Logging"]
git-tree-sha1 = "91ca22c4b8437da89b030f08d71db55a379ce958"
uuid = "7d9fca2a-8960-54d3-9f78-7d1dccf2cb97"
version = "0.5.3"

[[deps.Arpack_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "OpenBLAS_jll", "Pkg"]
git-tree-sha1 = "5ba6c757e8feccf03a1554dfaf3e26b3cfc7fd5e"
uuid = "68821587-b530-5797-8361-c406ea357684"
version = "3.5.1+1"

[[deps.Artifacts]]
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"

[[deps.AxisAlgorithms]]
deps = ["LinearAlgebra", "Random", "SparseArrays", "WoodburyMatrices"]
git-tree-sha1 = "66771c8d21c8ff5e3a93379480a2307ac36863f7"
uuid = "13072b0f-2c55-5437-9ae7-d433b7a33950"
version = "1.0.1"

[[deps.Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[deps.Bzip2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "19a35467a82e236ff51bc17a3a44b69ef35185a2"
uuid = "6e34b625-4abd-537c-b88f-471c36dfa7a0"
version = "1.0.8+0"

[[deps.Cairo_jll]]
deps = ["Artifacts", "Bzip2_jll", "Fontconfig_jll", "FreeType2_jll", "Glib_jll", "JLLWrappers", "LZO_jll", "Libdl", "Pixman_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll", "Zlib_jll", "libpng_jll"]
git-tree-sha1 = "4b859a208b2397a7a623a03449e4636bdb17bcf2"
uuid = "83423d85-b0ee-5818-9007-b63ccbeb887a"
version = "1.16.1+1"

[[deps.Calculus]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "f641eb0a4f00c343bbc32346e1217b86f3ce9dad"
uuid = "49dc2e85-a5d0-5ad3-a950-438e2897f1b9"
version = "0.5.1"

[[deps.ChainRulesCore]]
deps = ["Compat", "LinearAlgebra", "SparseArrays"]
git-tree-sha1 = "9950387274246d08af38f6eef8cb5480862a435f"
uuid = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
version = "1.14.0"

[[deps.ChangesOfVariables]]
deps = ["ChainRulesCore", "LinearAlgebra", "Test"]
git-tree-sha1 = "bf98fa45a0a4cee295de98d4c1462be26345b9a1"
uuid = "9e997f8a-9a97-42d5-a9f1-ce6bfc15e2c0"
version = "0.1.2"

[[deps.Clustering]]
deps = ["Distances", "LinearAlgebra", "NearestNeighbors", "Printf", "SparseArrays", "Statistics", "StatsBase"]
git-tree-sha1 = "75479b7df4167267d75294d14b58244695beb2ac"
uuid = "aaaa29a8-35af-508c-8bc3-b662a17a0fe5"
version = "0.14.2"

[[deps.ColorSchemes]]
deps = ["ColorTypes", "Colors", "FixedPointNumbers", "Random"]
git-tree-sha1 = "12fc73e5e0af68ad3137b886e3f7c1eacfca2640"
uuid = "35d6a980-a343-548e-a6ea-1d62b119f2f4"
version = "3.17.1"

[[deps.ColorTypes]]
deps = ["FixedPointNumbers", "Random"]
git-tree-sha1 = "024fe24d83e4a5bf5fc80501a314ce0d1aa35597"
uuid = "3da002f7-5984-5a60-b8a6-cbb66c0b333f"
version = "0.11.0"

[[deps.Colors]]
deps = ["ColorTypes", "FixedPointNumbers", "Reexport"]
git-tree-sha1 = "417b0ed7b8b838aa6ca0a87aadf1bb9eb111ce40"
uuid = "5ae59095-9a9b-59fe-a467-6f913c188581"
version = "0.12.8"

[[deps.Compat]]
deps = ["Base64", "Dates", "DelimitedFiles", "Distributed", "InteractiveUtils", "LibGit2", "Libdl", "LinearAlgebra", "Markdown", "Mmap", "Pkg", "Printf", "REPL", "Random", "SHA", "Serialization", "SharedArrays", "Sockets", "SparseArrays", "Statistics", "Test", "UUIDs", "Unicode"]
git-tree-sha1 = "96b0bc6c52df76506efc8a441c6cf1adcb1babc4"
uuid = "34da2185-b29b-5c13-b0c7-acf172513d20"
version = "3.42.0"

[[deps.CompilerSupportLibraries_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"
version = "0.5.0+0"

[[deps.Contour]]
deps = ["StaticArrays"]
git-tree-sha1 = "9f02045d934dc030edad45944ea80dbd1f0ebea7"
uuid = "d38c429a-6771-53c6-b99e-75d170b6e991"
version = "0.5.7"

[[deps.DataAPI]]
git-tree-sha1 = "cc70b17275652eb47bc9e5f81635981f13cea5c8"
uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a"
version = "1.9.0"

[[deps.DataStructures]]
deps = ["Compat", "InteractiveUtils", "OrderedCollections"]
git-tree-sha1 = "3daef5523dd2e769dad2365274f760ff5f282c7d"
uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
version = "0.18.11"

[[deps.DataValueInterfaces]]
git-tree-sha1 = "bfc1187b79289637fa0ef6d4436ebdfe6905cbd6"
uuid = "e2d170a0-9d28-54be-80f0-106bbe20a464"
version = "1.0.0"

[[deps.DataValues]]
deps = ["DataValueInterfaces", "Dates"]
git-tree-sha1 = "d88a19299eba280a6d062e135a43f00323ae70bf"
uuid = "e7dc6d0d-1eca-5fa6-8ad6-5aecde8b7ea5"
version = "0.4.13"

[[deps.Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[deps.DelimitedFiles]]
deps = ["Mmap"]
uuid = "8bb1440f-4735-579b-a4ab-409b98df4dab"

[[deps.DensityInterface]]
deps = ["InverseFunctions", "Test"]
git-tree-sha1 = "80c3e8639e3353e5d2912fb3a1916b8455e2494b"
uuid = "b429d917-457f-4dbc-8f4c-0cc954292b1d"
version = "0.4.0"

[[deps.Distances]]
deps = ["LinearAlgebra", "SparseArrays", "Statistics", "StatsAPI"]
git-tree-sha1 = "3258d0659f812acde79e8a74b11f17ac06d0ca04"
uuid = "b4f34e82-e78d-54a5-968a-f98e89d6e8f7"
version = "0.10.7"

[[deps.Distributed]]
deps = ["Random", "Serialization", "Sockets"]
uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b"

[[deps.Distributions]]
deps = ["ChainRulesCore", "DensityInterface", "FillArrays", "LinearAlgebra", "PDMats", "Printf", "QuadGK", "Random", "SparseArrays", "SpecialFunctions", "Statistics", "StatsBase", "StatsFuns", "Test"]
git-tree-sha1 = "5a4168170ede913a2cd679e53c2123cb4b889795"
uuid = "31c24e10-a181-5473-b8eb-7969acd0382f"
version = "0.25.53"

[[deps.DocStringExtensions]]
deps = ["LibGit2"]
git-tree-sha1 = "b19534d1895d702889b219c382a6e18010797f0b"
uuid = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae"
version = "0.8.6"

[[deps.Downloads]]
deps = ["ArgTools", "FileWatching", "LibCURL", "NetworkOptions"]
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"
version = "1.6.0"

[[deps.DualNumbers]]
deps = ["Calculus", "NaNMath", "SpecialFunctions"]
git-tree-sha1 = "5837a837389fccf076445fce071c8ddaea35a566"
uuid = "fa6b7ba4-c1ee-5f82-b5fc-ecf0adba8f74"
version = "0.6.8"

[[deps.EarCut_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "3f3a2501fa7236e9b911e0f7a588c657e822bb6d"
uuid = "5ae413db-bbd1-5e63-b57d-d24a61df00f5"
version = "2.2.3+0"

[[deps.Expat_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "bad72f730e9e91c08d9427d5e8db95478a3c323d"
uuid = "2e619515-83b5-522b-bb60-26c02a35a201"
version = "2.4.8+0"

[[deps.FFMPEG]]
deps = ["FFMPEG_jll"]
git-tree-sha1 = "b57e3acbe22f8484b4b5ff66a7499717fe1a9cc8"
uuid = "c87230d0-a227-11e9-1b43-d7ebe4e7570a"
version = "0.4.1"

[[deps.FFMPEG_jll]]
deps = ["Artifacts", "Bzip2_jll", "FreeType2_jll", "FriBidi_jll", "JLLWrappers", "LAME_jll", "Libdl", "Ogg_jll", "OpenSSL_jll", "Opus_jll", "Pkg", "Zlib_jll", "libass_jll", "libfdk_aac_jll", "libvorbis_jll", "x264_jll", "x265_jll"]
git-tree-sha1 = "d8a578692e3077ac998b50c0217dfd67f21d1e5f"
uuid = "b22a6f82-2f65-5046-a5b2-351ab43fb4e5"
version = "4.4.0+0"

[[deps.FFTW]]
deps = ["AbstractFFTs", "FFTW_jll", "LinearAlgebra", "MKL_jll", "Preferences", "Reexport"]
git-tree-sha1 = "505876577b5481e50d089c1c68899dfb6faebc62"
uuid = "7a1cc6ca-52ef-59f5-83cd-3a7055c09341"
version = "1.4.6"

[[deps.FFTW_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "c6033cc3892d0ef5bb9cd29b7f2f0331ea5184ea"
uuid = "f5851436-0d7a-5f13-b9de-f02708fd171a"
version = "3.3.10+0"

[[deps.FileWatching]]
uuid = "7b1f6079-737a-58dc-b8bc-7a2ca5c1b5ee"

[[deps.FillArrays]]
deps = ["LinearAlgebra", "Random", "SparseArrays", "Statistics"]
git-tree-sha1 = "246621d23d1f43e3b9c368bf3b72b2331a27c286"
uuid = "1a297f60-69ca-5386-bcde-b61e274b549b"
version = "0.13.2"

[[deps.FixedPointNumbers]]
deps = ["Statistics"]
git-tree-sha1 = "335bfdceacc84c5cdf16aadc768aa5ddfc5383cc"
uuid = "53c48c17-4a7d-5ca2-90c5-79b7896eea93"
version = "0.8.4"

[[deps.Fontconfig_jll]]
deps = ["Artifacts", "Bzip2_jll", "Expat_jll", "FreeType2_jll", "JLLWrappers", "Libdl", "Libuuid_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "21efd19106a55620a188615da6d3d06cd7f6ee03"
uuid = "a3f928ae-7b40-5064-980b-68af3947d34b"
version = "2.13.93+0"

[[deps.Formatting]]
deps = ["Printf"]
git-tree-sha1 = "8339d61043228fdd3eb658d86c926cb282ae72a8"
uuid = "59287772-0a20-5a39-b81b-1366585eb4c0"
version = "0.4.2"

[[deps.FreeType2_jll]]
deps = ["Artifacts", "Bzip2_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "87eb71354d8ec1a96d4a7636bd57a7347dde3ef9"
uuid = "d7e528f0-a631-5988-bf34-fe36492bcfd7"
version = "2.10.4+0"

[[deps.FriBidi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "aa31987c2ba8704e23c6c8ba8a4f769d5d7e4f91"
uuid = "559328eb-81f9-559d-9380-de523a88c83c"
version = "1.0.10+0"

[[deps.GLFW_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libglvnd_jll", "Pkg", "Xorg_libXcursor_jll", "Xorg_libXi_jll", "Xorg_libXinerama_jll", "Xorg_libXrandr_jll"]
git-tree-sha1 = "51d2dfe8e590fbd74e7a842cf6d13d8a2f45dc01"
uuid = "0656b61e-2033-5cc2-a64a-77c0f6c09b89"
version = "3.3.6+0"

[[deps.GR]]
deps = ["Base64", "DelimitedFiles", "GR_jll", "HTTP", "JSON", "Libdl", "LinearAlgebra", "Pkg", "Printf", "Random", "RelocatableFolders", "Serialization", "Sockets", "Test", "UUIDs"]
git-tree-sha1 = "af237c08bda486b74318c8070adb96efa6952530"
uuid = "28b8d3ca-fb5f-59d9-8090-bfdbd6d07a71"
version = "0.64.2"

[[deps.GR_jll]]
deps = ["Artifacts", "Bzip2_jll", "Cairo_jll", "FFMPEG_jll", "Fontconfig_jll", "GLFW_jll", "JLLWrappers", "JpegTurbo_jll", "Libdl", "Libtiff_jll", "Pixman_jll", "Pkg", "Qt5Base_jll", "Zlib_jll", "libpng_jll"]
git-tree-sha1 = "cd6efcf9dc746b06709df14e462f0a3fe0786b1e"
uuid = "d2c73de3-f751-5644-a686-071e5b155ba9"
version = "0.64.2+0"

[[deps.GeometryBasics]]
deps = ["EarCut_jll", "IterTools", "LinearAlgebra", "StaticArrays", "StructArrays", "Tables"]
git-tree-sha1 = "83ea630384a13fc4f002b77690bc0afeb4255ac9"
uuid = "5c1252a2-5f33-56bf-86c9-59e7332b4326"
version = "0.4.2"

[[deps.Gettext_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "XML2_jll"]
git-tree-sha1 = "9b02998aba7bf074d14de89f9d37ca24a1a0b046"
uuid = "78b55507-aeef-58d4-861c-77aaff3498b1"
version = "0.21.0+0"

[[deps.Glib_jll]]
deps = ["Artifacts", "Gettext_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Libiconv_jll", "Libmount_jll", "PCRE_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "a32d672ac2c967f3deb8a81d828afc739c838a06"
uuid = "7746bdde-850d-59dc-9ae8-88ece973131d"
version = "2.68.3+2"

[[deps.Graphite2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "344bf40dcab1073aca04aa0df4fb092f920e4011"
uuid = "3b182d85-2403-5c21-9c21-1e1f0cc25472"
version = "1.3.14+0"

[[deps.Grisu]]
git-tree-sha1 = "53bb909d1151e57e2484c3d1b53e19552b887fb2"
uuid = "42e2da0e-8278-4e71-bc24-59509adca0fe"
version = "1.0.2"

[[deps.HTTP]]
deps = ["Base64", "Dates", "IniFile", "Logging", "MbedTLS", "NetworkOptions", "Sockets", "URIs"]
git-tree-sha1 = "0fa77022fe4b511826b39c894c90daf5fce3334a"
uuid = "cd3eb016-35fb-5094-929b-558a96fad6f3"
version = "0.9.17"

[[deps.HarfBuzz_jll]]
deps = ["Artifacts", "Cairo_jll", "Fontconfig_jll", "FreeType2_jll", "Glib_jll", "Graphite2_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Pkg"]
git-tree-sha1 = "129acf094d168394e80ee1dc4bc06ec835e510a3"
uuid = "2e76f6c2-a576-52d4-95c1-20adfe4de566"
version = "2.8.1+1"

[[deps.HypergeometricFunctions]]
deps = ["DualNumbers", "LinearAlgebra", "SpecialFunctions", "Test"]
git-tree-sha1 = "65e4589030ef3c44d3b90bdc5aac462b4bb05567"
uuid = "34004b35-14d8-5ef3-9330-4cdb6864b03a"
version = "0.3.8"

[[deps.IniFile]]
git-tree-sha1 = "f550e6e32074c939295eb5ea6de31849ac2c9625"
uuid = "83e8ac13-25f8-5344-8a64-a9f2b223428f"
version = "0.5.1"

[[deps.IntelOpenMP_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "d979e54b71da82f3a65b62553da4fc3d18c9004c"
uuid = "1d5cc7b8-4909-519e-a0f8-d0f5ad9712d0"
version = "2018.0.3+2"

[[deps.InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[deps.Interpolations]]
deps = ["AxisAlgorithms", "ChainRulesCore", "LinearAlgebra", "OffsetArrays", "Random", "Ratios", "Requires", "SharedArrays", "SparseArrays", "StaticArrays", "WoodburyMatrices"]
git-tree-sha1 = "b15fc0a95c564ca2e0a7ae12c1f095ca848ceb31"
uuid = "a98d9a8b-a2ab-59e6-89dd-64a1c18fca59"
version = "0.13.5"

[[deps.InverseFunctions]]
deps = ["Test"]
git-tree-sha1 = "91b5dcf362c5add98049e6c29ee756910b03051d"
uuid = "3587e190-3f89-42d0-90ee-14403ec27112"
version = "0.1.3"

[[deps.IrrationalConstants]]
git-tree-sha1 = "7fd44fd4ff43fc60815f8e764c0f352b83c49151"
uuid = "92d709cd-6900-40b7-9082-c6be49f344b6"
version = "0.1.1"

[[deps.IterTools]]
git-tree-sha1 = "fa6287a4469f5e048d763df38279ee729fbd44e5"
uuid = "c8e1da08-722c-5040-9ed9-7db0dc04731e"
version = "1.4.0"

[[deps.IteratorInterfaceExtensions]]
git-tree-sha1 = "a3f24677c21f5bbe9d2a714f95dcd58337fb2856"
uuid = "82899510-4779-5014-852e-03e436cf321d"
version = "1.0.0"

[[deps.JLLWrappers]]
deps = ["Preferences"]
git-tree-sha1 = "abc9885a7ca2052a736a600f7fa66209f96506e1"
uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210"
version = "1.4.1"

[[deps.JSON]]
deps = ["Dates", "Mmap", "Parsers", "Unicode"]
git-tree-sha1 = "3c837543ddb02250ef42f4738347454f95079d4e"
uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
version = "0.21.3"

[[deps.JpegTurbo_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b53380851c6e6664204efb2e62cd24fa5c47e4ba"
uuid = "aacddb02-875f-59d6-b918-886e6ef4fbf8"
version = "2.1.2+0"

[[deps.KernelDensity]]
deps = ["Distributions", "DocStringExtensions", "FFTW", "Interpolations", "StatsBase"]
git-tree-sha1 = "591e8dc09ad18386189610acafb970032c519707"
uuid = "5ab0869b-81aa-558d-bb23-cbf5423bbe9b"
version = "0.6.3"

[[deps.LAME_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "f6250b16881adf048549549fba48b1161acdac8c"
uuid = "c1c5ebd0-6772-5130-a774-d5fcae4a789d"
version = "3.100.1+0"

[[deps.LERC_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "bf36f528eec6634efc60d7ec062008f171071434"
uuid = "88015f11-f218-50d7-93a8-a6af411a945d"
version = "3.0.0+1"

[[deps.LZO_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "e5b909bcf985c5e2605737d2ce278ed791b89be6"
uuid = "dd4b983a-f0e5-5f8d-a1b7-129d4a5fb1ac"
version = "2.10.1+0"

[[deps.LaTeXStrings]]
git-tree-sha1 = "f2355693d6778a178ade15952b7ac47a4ff97996"
uuid = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f"
version = "1.3.0"

[[deps.Latexify]]
deps = ["Formatting", "InteractiveUtils", "LaTeXStrings", "MacroTools", "Markdown", "Printf", "Requires"]
git-tree-sha1 = "6f14549f7760d84b2db7a9b10b88cd3cc3025730"
uuid = "23fbe1c1-3f47-55db-b15f-69d7ec21a316"
version = "0.15.14"

[[deps.LazyArtifacts]]
deps = ["Artifacts", "Pkg"]
uuid = "4af54fe1-eca0-43a8-85a7-787d91b784e3"

[[deps.LibCURL]]
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"
version = "0.6.3"

[[deps.LibCURL_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"
version = "7.81.0+0"

[[deps.LibGit2]]
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"

[[deps.LibSSH2_jll]]
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"
version = "1.10.2+0"

[[deps.Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[deps.Libffi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "0b4a5d71f3e5200a7dff793393e09dfc2d874290"
uuid = "e9f186c6-92d2-5b65-8a66-fee21dc1b490"
version = "3.2.2+1"

[[deps.Libgcrypt_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgpg_error_jll", "Pkg"]
git-tree-sha1 = "64613c82a59c120435c067c2b809fc61cf5166ae"
uuid = "d4300ac3-e22c-5743-9152-c294e39db1e4"
version = "1.8.7+0"

[[deps.Libglvnd_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll", "Xorg_libXext_jll"]
git-tree-sha1 = "7739f837d6447403596a75d19ed01fd08d6f56bf"
uuid = "7e76a0d4-f3c7-5321-8279-8d96eeed0f29"
version = "1.3.0+3"

[[deps.Libgpg_error_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "c333716e46366857753e273ce6a69ee0945a6db9"
uuid = "7add5ba3-2f88-524e-9cd5-f83b8a55f7b8"
version = "1.42.0+0"

[[deps.Libiconv_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "42b62845d70a619f063a7da093d995ec8e15e778"
uuid = "94ce4f54-9a6c-5748-9c1c-f9c7231a4531"
version = "1.16.1+1"

[[deps.Libmount_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "9c30530bf0effd46e15e0fdcf2b8636e78cbbd73"
uuid = "4b2f31a3-9ecc-558c-b454-b3730dcb73e9"
version = "2.35.0+0"

[[deps.Libtiff_jll]]
deps = ["Artifacts", "JLLWrappers", "JpegTurbo_jll", "LERC_jll", "Libdl", "Pkg", "Zlib_jll", "Zstd_jll"]
git-tree-sha1 = "c9551dd26e31ab17b86cbd00c2ede019c08758eb"
uuid = "89763e89-9b03-5906-acba-b20f662cd828"
version = "4.3.0+1"

[[deps.Libuuid_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "7f3efec06033682db852f8b3bc3c1d2b0a0ab066"
uuid = "38a345b3-de98-5d2b-a5d3-14cd9215e700"
version = "2.36.0+0"

[[deps.LinearAlgebra]]
deps = ["Libdl", "libblastrampoline_jll"]
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[[deps.LogExpFunctions]]
deps = ["ChainRulesCore", "ChangesOfVariables", "DocStringExtensions", "InverseFunctions", "IrrationalConstants", "LinearAlgebra"]
git-tree-sha1 = "58f25e56b706f95125dcb796f39e1fb01d913a71"
uuid = "2ab3a3ac-af41-5b50-aa03-7779005ae688"
version = "0.3.10"

[[deps.Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[deps.MKL_jll]]
deps = ["Artifacts", "IntelOpenMP_jll", "JLLWrappers", "LazyArtifacts", "Libdl", "Pkg"]
git-tree-sha1 = "e595b205efd49508358f7dc670a940c790204629"
uuid = "856f044c-d86e-5d09-b602-aeab76dc8ba7"
version = "2022.0.0+0"

[[deps.MacroTools]]
deps = ["Markdown", "Random"]
git-tree-sha1 = "3d3e902b31198a27340d0bf00d6ac452866021cf"
uuid = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"
version = "0.5.9"

[[deps.Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[deps.MbedTLS]]
deps = ["Dates", "MbedTLS_jll", "Random", "Sockets"]
git-tree-sha1 = "1c38e51c3d08ef2278062ebceade0e46cefc96fe"
uuid = "739be429-bea8-5141-9913-cc70e7f3736d"
version = "1.0.3"

[[deps.MbedTLS_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"
version = "2.28.0+0"

[[deps.Measures]]
git-tree-sha1 = "e498ddeee6f9fdb4551ce855a46f54dbd900245f"
uuid = "442fdcdd-2543-5da2-b0f3-8c86c306513e"
version = "0.3.1"

[[deps.Missings]]
deps = ["DataAPI"]
git-tree-sha1 = "bf210ce90b6c9eed32d25dbcae1ebc565df2687f"
uuid = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28"
version = "1.0.2"

[[deps.Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"

[[deps.MozillaCACerts_jll]]
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"
version = "2022.2.1"

[[deps.MultivariateStats]]
deps = ["Arpack", "LinearAlgebra", "SparseArrays", "Statistics", "StatsAPI", "StatsBase"]
git-tree-sha1 = "7008a3412d823e29d370ddc77411d593bd8a3d03"
uuid = "6f286f6a-111f-5878-ab1e-185364afe411"
version = "0.9.1"

[[deps.NaNMath]]
git-tree-sha1 = "737a5957f387b17e74d4ad2f440eb330b39a62c5"
uuid = "77ba4419-2d1f-58cd-9bb1-8ffee604a2e3"
version = "1.0.0"

[[deps.NearestNeighbors]]
deps = ["Distances", "StaticArrays"]
git-tree-sha1 = "ded92de95031d4a8c61dfb6ba9adb6f1d8016ddd"
uuid = "b8a86587-4115-5ab1-83bc-aa920d37bbce"
version = "0.4.10"

[[deps.NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"
version = "1.2.0"

[[deps.Observables]]
git-tree-sha1 = "fe29afdef3d0c4a8286128d4e45cc50621b1e43d"
uuid = "510215fc-4207-5dde-b226-833fc4488ee2"
version = "0.4.0"

[[deps.OffsetArrays]]
deps = ["Adapt"]
git-tree-sha1 = "043017e0bdeff61cfbb7afeb558ab29536bbb5ed"
uuid = "6fe1bfb0-de20-5000-8ca7-80f57d26f881"
version = "1.10.8"

[[deps.Ogg_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "887579a3eb005446d514ab7aeac5d1d027658b8f"
uuid = "e7412a2a-1a6e-54c0-be00-318e2571c051"
version = "1.3.5+1"

[[deps.OpenBLAS_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "Libdl"]
uuid = "4536629a-c528-5b80-bd46-f80d51c5b363"
version = "0.3.17+2"

[[deps.OpenLibm_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "05823500-19ac-5b8b-9628-191a04bc5112"
version = "0.8.1+0"

[[deps.OpenSSL_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "ab05aa4cc89736e95915b01e7279e61b1bfe33b8"
uuid = "458c3c95-2e84-50aa-8efc-19380b2a3a95"
version = "1.1.14+0"

[[deps.OpenSpecFun_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "13652491f6856acfd2db29360e1bbcd4565d04f1"
uuid = "efe28fd5-8261-553b-a9e1-b2916fc3738e"
version = "0.5.5+0"

[[deps.Opus_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "51a08fb14ec28da2ec7a927c4337e4332c2a4720"
uuid = "91d4177d-7536-5919-b921-800302f37372"
version = "1.3.2+0"

[[deps.OrderedCollections]]
git-tree-sha1 = "85f8e6578bf1f9ee0d11e7bb1b1456435479d47c"
uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
version = "1.4.1"

[[deps.PCRE_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b2a7af664e098055a7529ad1a900ded962bca488"
uuid = "2f80f16e-611a-54ab-bc61-aa92de5b98fc"
version = "8.44.0+0"

[[deps.PDMats]]
deps = ["LinearAlgebra", "SparseArrays", "SuiteSparse"]
git-tree-sha1 = "e8185b83b9fc56eb6456200e873ce598ebc7f262"
uuid = "90014a1f-27ba-587c-ab20-58faa44d9150"
version = "0.11.7"

[[deps.Parsers]]
deps = ["Dates"]
git-tree-sha1 = "621f4f3b4977325b9128d5fae7a8b4829a0c2222"
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
version = "2.2.4"

[[deps.Pixman_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b4f5d02549a10e20780a24fce72bea96b6329e29"
uuid = "30392449-352a-5448-841d-b1acce4e97dc"
version = "0.40.1+0"

[[deps.Pkg]]
deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
version = "1.8.0"

[[deps.PlotThemes]]
deps = ["PlotUtils", "Requires", "Statistics"]
git-tree-sha1 = "a3a964ce9dc7898193536002a6dd892b1b5a6f1d"
uuid = "ccf2f8ad-2431-5c83-bf29-c5338b663b6a"
version = "2.0.1"

[[deps.PlotUtils]]
deps = ["ColorSchemes", "Colors", "Dates", "Printf", "Random", "Reexport", "Statistics"]
git-tree-sha1 = "bb16469fd5224100e422f0b027d26c5a25de1200"
uuid = "995b91a9-d308-5afd-9ec6-746e21dbc043"
version = "1.2.0"

[[deps.Plots]]
deps = ["Base64", "Contour", "Dates", "Downloads", "FFMPEG", "FixedPointNumbers", "GR", "GeometryBasics", "JSON", "Latexify", "LinearAlgebra", "Measures", "NaNMath", "Pkg", "PlotThemes", "PlotUtils", "Printf", "REPL", "Random", "RecipesBase", "RecipesPipeline", "Reexport", "Requires", "Scratch", "Showoff", "SparseArrays", "Statistics", "StatsBase", "UUIDs", "UnicodeFun", "Unzip"]
git-tree-sha1 = "edec0846433f1c1941032385588fd57380b62b59"
uuid = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
version = "1.27.4"

[[deps.Preferences]]
deps = ["TOML"]
git-tree-sha1 = "d3538e7f8a790dc8903519090857ef8e1283eecd"
uuid = "21216c6a-2e73-6563-6e65-726566657250"
version = "1.2.5"

[[deps.Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[deps.Qt5Base_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "Fontconfig_jll", "Glib_jll", "JLLWrappers", "Libdl", "Libglvnd_jll", "OpenSSL_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libxcb_jll", "Xorg_xcb_util_image_jll", "Xorg_xcb_util_keysyms_jll", "Xorg_xcb_util_renderutil_jll", "Xorg_xcb_util_wm_jll", "Zlib_jll", "xkbcommon_jll"]
git-tree-sha1 = "ad368663a5e20dbb8d6dc2fddeefe4dae0781ae8"
uuid = "ea2cea3b-5b76-57ae-a6ef-0a8af62496e1"
version = "5.15.3+0"

[[deps.QuadGK]]
deps = ["DataStructures", "LinearAlgebra"]
git-tree-sha1 = "78aadffb3efd2155af139781b8a8df1ef279ea39"
uuid = "1fd47b50-473d-5c70-9696-f719f8f3bcdc"
version = "2.4.2"

[[deps.REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[[deps.Random]]
deps = ["SHA", "Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[deps.Ratios]]
deps = ["Requires"]
git-tree-sha1 = "dc84268fe0e3335a62e315a3a7cf2afa7178a734"
uuid = "c84ed2f1-dad5-54f0-aa8e-dbefe2724439"
version = "0.4.3"

[[deps.RecipesBase]]
git-tree-sha1 = "6bf3f380ff52ce0832ddd3a2a7b9538ed1bcca7d"
uuid = "3cdcf5f2-1ef4-517c-9805-6587b60abb01"
version = "1.2.1"

[[deps.RecipesPipeline]]
deps = ["Dates", "NaNMath", "PlotUtils", "RecipesBase"]
git-tree-sha1 = "dc1e451e15d90347a7decc4221842a022b011714"
uuid = "01d81517-befc-4cb6-b9ec-a95719d0359c"
version = "0.5.2"

[[deps.Reexport]]
git-tree-sha1 = "45e428421666073eab6f2da5c9d310d99bb12f9b"
uuid = "189a3867-3050-52da-a836-e630ba90ab69"
version = "1.2.2"

[[deps.RelocatableFolders]]
deps = ["SHA", "Scratch"]
git-tree-sha1 = "cdbd3b1338c72ce29d9584fdbe9e9b70eeb5adca"
uuid = "05181044-ff0b-4ac5-8273-598c1e38db00"
version = "0.1.3"

[[deps.Requires]]
deps = ["UUIDs"]
git-tree-sha1 = "838a3a4188e2ded87a4f9f184b4b0d78a1e91cb7"
uuid = "ae029012-a4dd-5104-9daa-d747884805df"
version = "1.3.0"

[[deps.Rmath]]
deps = ["Random", "Rmath_jll"]
git-tree-sha1 = "bf3188feca147ce108c76ad82c2792c57abe7b1f"
uuid = "79098fc4-a85e-5d69-aa6a-4863f24498fa"
version = "0.7.0"

[[deps.Rmath_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "68db32dff12bb6127bac73c209881191bf0efbb7"
uuid = "f50d1b31-88e8-58de-be2c-1cc44531875f"
version = "0.3.0+0"

[[deps.SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"
version = "0.7.0"

[[deps.Scratch]]
deps = ["Dates"]
git-tree-sha1 = "0b4b7f1393cff97c33891da2a0bf69c6ed241fda"
uuid = "6c6a2e73-6563-6170-7368-637461726353"
version = "1.1.0"

[[deps.SentinelArrays]]
deps = ["Dates", "Random"]
git-tree-sha1 = "6a2f7d70512d205ca8c7ee31bfa9f142fe74310c"
uuid = "91c51154-3ec4-41a3-a24f-3f23e20d615c"
version = "1.3.12"

[[deps.Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[deps.SharedArrays]]
deps = ["Distributed", "Mmap", "Random", "Serialization"]
uuid = "1a1011a3-84de-559e-8e89-a11a2f7dc383"

[[deps.Showoff]]
deps = ["Dates", "Grisu"]
git-tree-sha1 = "91eddf657aca81df9ae6ceb20b959ae5653ad1de"
uuid = "992d4aef-0814-514b-bc4d-f2e9a6c4116f"
version = "1.0.3"

[[deps.Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"

[[deps.SortingAlgorithms]]
deps = ["DataStructures"]
git-tree-sha1 = "b3363d7460f7d098ca0912c69b082f75625d7508"
uuid = "a2af1166-a08f-5f64-846c-94a0d3cef48c"
version = "1.0.1"

[[deps.SparseArrays]]
deps = ["LinearAlgebra", "Random"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"

[[deps.SpecialFunctions]]
deps = ["ChainRulesCore", "IrrationalConstants", "LogExpFunctions", "OpenLibm_jll", "OpenSpecFun_jll"]
git-tree-sha1 = "5ba658aeecaaf96923dce0da9e703bd1fe7666f9"
uuid = "276daf66-3868-5448-9aa4-cd146d93841b"
version = "2.1.4"

[[deps.StaticArrays]]
deps = ["LinearAlgebra", "Random", "Statistics"]
git-tree-sha1 = "4f6ec5d99a28e1a749559ef7dd518663c5eca3d5"
uuid = "90137ffa-7385-5640-81b9-e52037218182"
version = "1.4.3"

[[deps.Statistics]]
deps = ["LinearAlgebra", "SparseArrays"]
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[[deps.StatsAPI]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "c3d8ba7f3fa0625b062b82853a7d5229cb728b6b"
uuid = "82ae8749-77ed-4fe6-ae5f-f523153014b0"
version = "1.2.1"

[[deps.StatsBase]]
deps = ["DataAPI", "DataStructures", "LinearAlgebra", "LogExpFunctions", "Missings", "Printf", "Random", "SortingAlgorithms", "SparseArrays", "Statistics", "StatsAPI"]
git-tree-sha1 = "8977b17906b0a1cc74ab2e3a05faa16cf08a8291"
uuid = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
version = "0.33.16"

[[deps.StatsFuns]]
deps = ["ChainRulesCore", "HypergeometricFunctions", "InverseFunctions", "IrrationalConstants", "LogExpFunctions", "Reexport", "Rmath", "SpecialFunctions"]
git-tree-sha1 = "72e6abd6fc9ef0fa62a159713c83b7637a14b2b8"
uuid = "4c63d2b9-4356-54db-8cca-17b64c39e42c"
version = "0.9.17"

[[deps.StatsPlots]]
deps = ["AbstractFFTs", "Clustering", "DataStructures", "DataValues", "Distributions", "Interpolations", "KernelDensity", "LinearAlgebra", "MultivariateStats", "Observables", "Plots", "RecipesBase", "RecipesPipeline", "Reexport", "StatsBase", "TableOperations", "Tables", "Widgets"]
git-tree-sha1 = "4d9c69d65f1b270ad092de0abe13e859b8c55cad"
uuid = "f3b207a7-027a-5e70-b257-86293d7955fd"
version = "0.14.33"

[[deps.StructArrays]]
deps = ["Adapt", "DataAPI", "StaticArrays", "Tables"]
git-tree-sha1 = "57617b34fa34f91d536eb265df67c2d4519b8b98"
uuid = "09ab397b-f2b6-538f-b94a-2f83cf4a842a"
version = "0.6.5"

[[deps.SuiteSparse]]
deps = ["Libdl", "LinearAlgebra", "Serialization", "SparseArrays"]
uuid = "4607b0f0-06f3-5cda-b6b1-a6196a1729e9"

[[deps.TOML]]
deps = ["Dates"]
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"
version = "1.0.0"

[[deps.TableOperations]]
deps = ["SentinelArrays", "Tables", "Test"]
git-tree-sha1 = "e383c87cf2a1dc41fa30c093b2a19877c83e1bc1"
uuid = "ab02a1b2-a7df-11e8-156e-fb1833f50b87"
version = "1.2.0"

[[deps.TableTraits]]
deps = ["IteratorInterfaceExtensions"]
git-tree-sha1 = "c06b2f539df1c6efa794486abfb6ed2022561a39"
uuid = "3783bdb8-4a98-5b6b-af9a-565f29a5fe9c"
version = "1.0.1"

[[deps.Tables]]
deps = ["DataAPI", "DataValueInterfaces", "IteratorInterfaceExtensions", "LinearAlgebra", "OrderedCollections", "TableTraits", "Test"]
git-tree-sha1 = "5ce79ce186cc678bbb5c5681ca3379d1ddae11a1"
uuid = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
version = "1.7.0"

[[deps.Tar]]
deps = ["ArgTools", "SHA"]
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"
version = "1.10.0"

[[deps.Test]]
deps = ["InteractiveUtils", "Logging", "Random", "Serialization"]
uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[[deps.URIs]]
git-tree-sha1 = "97bbe755a53fe859669cd907f2d96aee8d2c1355"
uuid = "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4"
version = "1.3.0"

[[deps.UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[[deps.Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"

[[deps.UnicodeFun]]
deps = ["REPL"]
git-tree-sha1 = "53915e50200959667e78a92a418594b428dffddf"
uuid = "1cfade01-22cf-5700-b092-accc4b62d6e1"
version = "0.4.1"

[[deps.Unzip]]
git-tree-sha1 = "34db80951901073501137bdbc3d5a8e7bbd06670"
uuid = "41fe7b60-77ed-43a1-b4f0-825fd5a5650d"
version = "0.1.2"

[[deps.Wayland_jll]]
deps = ["Artifacts", "Expat_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Pkg", "XML2_jll"]
git-tree-sha1 = "3e61f0b86f90dacb0bc0e73a0c5a83f6a8636e23"
uuid = "a2964d1f-97da-50d4-b82a-358c7fce9d89"
version = "1.19.0+0"

[[deps.Wayland_protocols_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4528479aa01ee1b3b4cd0e6faef0e04cf16466da"
uuid = "2381bf8a-dfd0-557d-9999-79630e7b1b91"
version = "1.25.0+0"

[[deps.Widgets]]
deps = ["Colors", "Dates", "Observables", "OrderedCollections"]
git-tree-sha1 = "505c31f585405fc375d99d02588f6ceaba791241"
uuid = "cc8bc4a8-27d6-5769-a93b-9d913e69aa62"
version = "0.6.5"

[[deps.WoodburyMatrices]]
deps = ["LinearAlgebra", "SparseArrays"]
git-tree-sha1 = "de67fa59e33ad156a590055375a30b23c40299d3"
uuid = "efce3f68-66dc-5838-9240-27a6d6f5f9b6"
version = "0.5.5"

[[deps.XML2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "1acf5bdf07aa0907e0a37d3718bb88d4b687b74a"
uuid = "02c8fc9c-b97f-50b9-bbe4-9be30ff0a78a"
version = "2.9.12+0"

[[deps.XSLT_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgcrypt_jll", "Libgpg_error_jll", "Libiconv_jll", "Pkg", "XML2_jll", "Zlib_jll"]
git-tree-sha1 = "91844873c4085240b95e795f692c4cec4d805f8a"
uuid = "aed1982a-8fda-507f-9586-7b0439959a61"
version = "1.1.34+0"

[[deps.Xorg_libX11_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll", "Xorg_xtrans_jll"]
git-tree-sha1 = "5be649d550f3f4b95308bf0183b82e2582876527"
uuid = "4f6342f7-b3d2-589e-9d20-edeb45f2b2bc"
version = "1.6.9+4"

[[deps.Xorg_libXau_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4e490d5c960c314f33885790ed410ff3a94ce67e"
uuid = "0c0b7dd1-d40b-584c-a123-a41640f87eec"
version = "1.0.9+4"

[[deps.Xorg_libXcursor_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXfixes_jll", "Xorg_libXrender_jll"]
git-tree-sha1 = "12e0eb3bc634fa2080c1c37fccf56f7c22989afd"
uuid = "935fb764-8cf2-53bf-bb30-45bb1f8bf724"
version = "1.2.0+4"

[[deps.Xorg_libXdmcp_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4fe47bd2247248125c428978740e18a681372dd4"
uuid = "a3789734-cfe1-5b06-b2d0-1dd0d9d62d05"
version = "1.1.3+4"

[[deps.Xorg_libXext_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "b7c0aa8c376b31e4852b360222848637f481f8c3"
uuid = "1082639a-0dae-5f34-9b06-72781eeb8cb3"
version = "1.3.4+4"

[[deps.Xorg_libXfixes_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "0e0dc7431e7a0587559f9294aeec269471c991a4"
uuid = "d091e8ba-531a-589c-9de9-94069b037ed8"
version = "5.0.3+4"

[[deps.Xorg_libXi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll", "Xorg_libXfixes_jll"]
git-tree-sha1 = "89b52bc2160aadc84d707093930ef0bffa641246"
uuid = "a51aa0fd-4e3c-5386-b890-e753decda492"
version = "1.7.10+4"

[[deps.Xorg_libXinerama_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll"]
git-tree-sha1 = "26be8b1c342929259317d8b9f7b53bf2bb73b123"
uuid = "d1454406-59df-5ea1-beac-c340f2130bc3"
version = "1.1.4+4"

[[deps.Xorg_libXrandr_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll"]
git-tree-sha1 = "34cea83cb726fb58f325887bf0612c6b3fb17631"
uuid = "ec84b674-ba8e-5d96-8ba1-2a689ba10484"
version = "1.5.2+4"

[[deps.Xorg_libXrender_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "19560f30fd49f4d4efbe7002a1037f8c43d43b96"
uuid = "ea2f1a96-1ddc-540d-b46f-429655e07cfa"
version = "0.9.10+4"

[[deps.Xorg_libpthread_stubs_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "6783737e45d3c59a4a4c4091f5f88cdcf0908cbb"
uuid = "14d82f49-176c-5ed1-bb49-ad3f5cbd8c74"
version = "0.1.0+3"

[[deps.Xorg_libxcb_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "XSLT_jll", "Xorg_libXau_jll", "Xorg_libXdmcp_jll", "Xorg_libpthread_stubs_jll"]
git-tree-sha1 = "daf17f441228e7a3833846cd048892861cff16d6"
uuid = "c7cfdc94-dc32-55de-ac96-5a1b8d977c5b"
version = "1.13.0+3"

[[deps.Xorg_libxkbfile_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "926af861744212db0eb001d9e40b5d16292080b2"
uuid = "cc61e674-0454-545c-8b26-ed2c68acab7a"
version = "1.1.0+4"

[[deps.Xorg_xcb_util_image_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "0fab0a40349ba1cba2c1da699243396ff8e94b97"
uuid = "12413925-8142-5f55-bb0e-6d7ca50bb09b"
version = "0.4.0+1"

[[deps.Xorg_xcb_util_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll"]
git-tree-sha1 = "e7fd7b2881fa2eaa72717420894d3938177862d1"
uuid = "2def613f-5ad1-5310-b15b-b15d46f528f5"
version = "0.4.0+1"

[[deps.Xorg_xcb_util_keysyms_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "d1151e2c45a544f32441a567d1690e701ec89b00"
uuid = "975044d2-76e6-5fbe-bf08-97ce7c6574c7"
version = "0.4.0+1"

[[deps.Xorg_xcb_util_renderutil_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "dfd7a8f38d4613b6a575253b3174dd991ca6183e"
uuid = "0d47668e-0667-5a69-a72c-f761630bfb7e"
version = "0.3.9+1"

[[deps.Xorg_xcb_util_wm_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "e78d10aab01a4a154142c5006ed44fd9e8e31b67"
uuid = "c22f9ab0-d5fe-5066-847c-f4bb1cd4e361"
version = "0.4.1+1"

[[deps.Xorg_xkbcomp_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxkbfile_jll"]
git-tree-sha1 = "4bcbf660f6c2e714f87e960a171b119d06ee163b"
uuid = "35661453-b289-5fab-8a00-3d9160c6a3a4"
version = "1.4.2+4"

[[deps.Xorg_xkeyboard_config_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xkbcomp_jll"]
git-tree-sha1 = "5c8424f8a67c3f2209646d4425f3d415fee5931d"
uuid = "33bec58e-1273-512f-9401-5d533626f822"
version = "2.27.0+4"

[[deps.Xorg_xtrans_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "79c31e7844f6ecf779705fbc12146eb190b7d845"
uuid = "c5fb5394-a638-5e4d-96e5-b29de1b5cf10"
version = "1.4.0+3"

[[deps.Zlib_jll]]
deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"
version = "1.2.12+1"

[[deps.Zstd_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "e45044cd873ded54b6a5bac0eb5c971392cf1927"
uuid = "3161d3a3-bdf6-5164-811a-617609db77b4"
version = "1.5.2+0"

[[deps.libass_jll]]
deps = ["Artifacts", "Bzip2_jll", "FreeType2_jll", "FriBidi_jll", "HarfBuzz_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "5982a94fcba20f02f42ace44b9894ee2b140fe47"
uuid = "0ac62f75-1d6f-5e53-bd7c-93b484bb37c0"
version = "0.15.1+0"

[[deps.libblastrampoline_jll]]
deps = ["Artifacts", "Libdl", "OpenBLAS_jll"]
uuid = "8e850b90-86db-534c-a0d3-1478176c7d93"
version = "5.0.1+0"

[[deps.libfdk_aac_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "daacc84a041563f965be61859a36e17c4e4fcd55"
uuid = "f638f0a6-7fb0-5443-88ba-1cc74229b280"
version = "2.0.2+0"

[[deps.libpng_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "94d180a6d2b5e55e447e2d27a29ed04fe79eb30c"
uuid = "b53b4c65-9356-5827-b1ea-8c7a1a84506f"
version = "1.6.38+0"

[[deps.libvorbis_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Ogg_jll", "Pkg"]
git-tree-sha1 = "b910cb81ef3fe6e78bf6acee440bda86fd6ae00c"
uuid = "f27f6e37-5d2b-51aa-960f-b287f2bc3b7a"
version = "1.3.7+1"

[[deps.nghttp2_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"
version = "1.41.0+1"

[[deps.p7zip_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"
version = "16.2.1+1"

[[deps.x264_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4fea590b89e6ec504593146bf8b988b2c00922b2"
uuid = "1270edf5-f2f9-52d2-97e9-ab00b5d0237a"
version = "2021.5.5+0"

[[deps.x265_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "ee567a171cce03570d77ad3a43e90218e38937a9"
uuid = "dfaa095f-4041-5dcd-9319-2fabd8486b76"
version = "3.5.0+0"

[[deps.xkbcommon_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Wayland_jll", "Wayland_protocols_jll", "Xorg_libxcb_jll", "Xorg_xkeyboard_config_jll"]
git-tree-sha1 = "ece2350174195bb31de1a63bea3a41ae1aa593b6"
uuid = "d8fb68d0-12a3-5cfd-a85a-d49703b185fd"
version = "0.9.1+5"
"""

# ╔═╡ Cell order:
# ╟─1eb34f10-b4b3-11ec-19f5-c7f26108d374
# ╟─6c4b3bf7-26e9-4d6f-9305-65fbc851818f
# ╟─2743185f-b0d7-4195-8e8e-a1bfcf132934
# ╠═982fab25-1414-4789-bc16-343ab29b4514
# ╟─7e53c83c-ead7-484f-8762-4a0956ba7210
# ╟─c8977146-1409-4eb1-bede-13436fd7214b
# ╟─201fb421-39b1-47db-8dec-609706461dca
# ╟─e12e34c7-90db-4f31-8792-ce2a39acc435
# ╟─1aaa8364-b72e-4128-b26a-82d4d4601634
# ╠═7387c353-51b7-402b-a500-adec0a34b059
# ╠═b40838d0-8391-4f1e-a3fc-a4be028b7bc7
# ╟─ec26fefb-18d1-4ac7-b0c1-0d3c71ae1305
# ╟─355540ec-db9b-4ce5-a5d2-0c10497da934
# ╟─de44b0c7-eee4-49ed-9264-40a5339e7d6c
# ╟─6f073f29-39b5-496e-b644-45051c81ab54
# ╟─5500a012-48fd-4227-a70f-515425c5122a
# ╟─e0a8d8ea-e5bd-48ca-b44f-4b6625181a17
# ╟─cc32c2a5-d377-4a77-8477-b3acc63cca3b
# ╟─00000000-0000-0000-0000-000000000001
# ╟─00000000-0000-0000-0000-000000000002


#=
day7:
- Julia version: 1.7.0
- Author: Paul.Mealor
- Date: 2021-12-07
=#

using AoC, Test

parsePositions(line) = parse.(Int, split(line, ","))
@test parsePositions("5,34,3") == [5,34,3]

totalDistance(candidate, positions) = sum(p -> abs(candidate - p), positions)
@test totalDistance(4,[4,4]) == 0
@test totalDistance(1,[16,1,2,0,4,2,7,1,2,14]) == 41
@test totalDistance(2,[16,1,2,0,4,2,7,1,2,14]) == 37
@test totalDistance(3,[16,1,2,0,4,2,7,1,2,14]) == 39
@test totalDistance(10,[16,1,2,0,4,2,7,1,2,14]) == 71

totalDistance2(candidate, positions) = sum(positions) do p
    distance = abs(candidate - p)
    return distance * (distance + 1) ÷ 2
end
@test totalDistance2(4,[4,4]) == 0
@test totalDistance2(5,[16,1,2,0,4,2,7,1,2,14]) == 168
@test totalDistance2(2,[16,1,2,0,4,2,7,1,2,14]) == 206

function bestPosition(positions)
    bestDistance = typemax(eltype(positions))
    result = -1
    for candidate = min(positions...):max(positions...)
        thisDistance = totalDistance(candidate, positions)
        if thisDistance < bestDistance
            bestDistance = thisDistance
            result = candidate
        end
    end
    return result
end

function bestPosition2(positions)
    bestDistance = typemax(eltype(positions))
    result = -1
    for candidate = min(positions...):max(positions...)
        thisDistance = totalDistance2(candidate, positions)
        if thisDistance < bestDistance
            bestDistance = thisDistance
            result = candidate
        end
    end
    return result
end

@test bestPosition([16,1,2,0,4,2,7,1,2,14]) == 2
@test bestPosition2([16,1,2,0,4,2,7,1,2,14]) == 5

function part1(line)
    positions = parsePositions(line)
    position = bestPosition(positions)
    return totalDistance(position, positions)
end

@test part1(firstExampleLine(7,1)) == 37

function part2(line)
    positions = parsePositions(line)
    position = bestPosition2(positions)
    return totalDistance2(position, positions)
end

@test part2(firstExampleLine(7,1)) == 168

firstLine(7) |> l -> @time part1(l) |> show
firstLine(7) |> l -> @time part1(l) |> show
firstLine(7) |> l -> @time part2(l) |> show
firstLine(7) |> l -> @time part2(l) |> show


function mandelbrot_set(xmin,xmax,ymin,ymax,width,height,maxiter)
    r1 = float_range(xmin, xmax, width)
    r2 = float_range(ymin, ymax, height)

    cartesian =  Array{Float64, 3}(undef, 3, height, width)

    for y = 1:height, x = 1:width
	point = mandel(ComplexF64(r1[x], r2[y]), maxiter)
	if point == maxiter
		point = 0
	else
		point /= convert(Float64, maxiter)

	end

 	red_color, blue_color, green_color = 1.0, 1.0, 1.0
    	if point < 0.01
            red_color, blue_color, green_color = 1.0, 1.0, 1.0
    	else
            red_color = point * 0.80
            blue_color = point * 0.80
            green_color = point * 0.15
    	end

	cartesian[1,y,x] = red_color
	cartesian[2,y,x] = blue_color
	cartesian[3,y,x] = green_color
    end

    return cartesian
end

# the mandelbrot iteration function
function mandel(c, maxiter::Int64)
    z=0+0im
    for n = 1:maxiter
        if abs(z) > 2
            return n-1
        end
        z = z^2 + c
    end
    return maxiter
end

function float_range(xmin, xmax, width)
	r = Array{Float64, 1}(undef, width)

	delta = abs(xmax - xmin)/(width - 1)

	for i = 1:width
		r[i] = xmin + (i-1)*delta
	end

	return r
end


using LinearAlgebra
using CSV
using DataFrames

# selecting the index combination from given range
function getIndexes(i,j,d)
    if i == j
        return [i]
    elseif i > j && i > 2
        return 0
    else
        niz1 = []
        niz2 = collect(i:j-1)
        for k in 1:d
            if !(k in niz2)
                push!(niz1,k)
            end
        end
        return length(niz1) == 0 ? 0 : niz1
    end
end

#selecting columns from indexes as matrix
function getSelectedColumns(selectedColumns,X)
    if selectedColumns != 0
        localX = ones(size(X)[1],1)
        for i in selectedColumns
            localX = [localX X[:,i]]
        end
        return localX
    else
        return 0
    end
end

#load data
Diabete_Data = CSV.read("diabetes.data",DataFrame)

n,d = size(Diabete_Data)
d = d-1

# compute the optimal theta

X = zeros(n,d)
y = zeros(n,1)

for i=1:d
    X[:,i] = Diabete_Data[:,i]
end

y = Diabete_Data[:,end]

# for storing MSEs
savedMSE = []

#for  storing top contender colums
savedIndexes = []

#  age
#  sex
#  bmi
#  map	   mean arterial pressure
#  s1  tc :   total cholesterol
#  s2  ldl    low density lipoprotein
#  s3  hdl    high density lipoprotein
#  s4  tch
#  s5  ltg
#  s6  glu

# string representation of said colums
nameByIndex = ["AGE" "SEX" "BMI" "MAP" "TC" "LDL" "HDL" "TCH" "LTG" "GLU"]

#Calculating all possible solutions
for i in 1:d-1
    for j in 1:d
        localX = []
        temp = getIndexes(i,j,d)
        selectedColumns = temp == 0 ? continue : temp
        localX = getSelectedColumns(selectedColumns,X)
        localX = localX == 0 ? continue : localX
        theta_opt = localX\y
        # achieved loss
        MSE = norm(localX*theta_opt-y,2)^2/n
        push!(savedMSE,MSE)
        push!(savedIndexes,selectedColumns)
    end
end

# Number of top results
numOfBests = 10

#Finding top results
for i in 1:numOfBests
   bestIndex = argmin(savedMSE)
    bestMSE = savedMSE[bestIndex]
    leastColums = savedIndexes[bestIndex]
    println("\n [$i] BEST SOLUTION")
    println("Best MSE of: $(bestMSE) with ")
    for j in leastColums
        println("$(nameByIndex[j]) ")
    end
    println("parameters as most impactful")
    deleteat!(savedMSE,bestIndex)
    deleteat!(savedIndexes,bestIndex)
end


using LinearAlgebra
P = [   0   1   0   0   0;
        1/4 0   1/4 1/4 1/4;
        0   1/2 0   0   1/2;
        0   1/2 0   0   1/2;
        0   0   0   0   1]
T = P[1:4,1:4]
p0 = [1 0 0 0]
for n in 1:10
    println(first(p0*sum([T^k for k in 0:n])*ones(4)))
end
println("Using inverse: ", first(p0*inv(I-T)*ones(4)))
println("Eigenvalues of T: ", sort(eigvals(T)))


@with_kw struct DGaussianPulse <: AbstractWaveform
    fc::Float64 = 0.0
    half_bandwidth::Float64
    half_width::Float64 = 1 / (2 * sqrt(log(2))) * 1 / (π * half_bandwidth)
    time_delay::Float64 = 6 * half_width
end
(s::DGaussianPulse)(t) = -(t - s.time_delay) / s.half_width * exp(-(t - s.time_delay)^2 / s.half_width^2)
@with_kw struct GaussianPulse <: AbstractWaveform
    fc::Float64 = 0.0
    half_bandwidth::Float64
    half_width::Float64 = 1 / (2 * sqrt(log(2))) * 1 / (π * half_bandwidth)
    time_delay::Float64 = 6 * half_width
end

(s::GaussianPulse)(t) = cos(2π * s.fc * (t - s.time_delay)) * exp(-(t - s.time_delay)^2 / s.half_width^2)

const GaussianWaveForm = Union{GaussianPulse,DGaussianPulse}

@recipe function f(s::GaussianWaveForm; nsamples = 20, spectrum = true)
    if spectrum
        xlabel := "frequency"
        xrange := [-s.fc - 10 * s.half_bandwidth, s.fc + 10 * s.half_bandwidth]
    else
        xrange := [s.time_delay - 3s.half_width, s.time_delay + 3s.half_width]
    end
    δt = 1 / (s.fc + s.half_bandwidth) / nsamples
    t_samples = range(s.time_delay - 20s.half_width, s.time_delay + 20s.half_width, step = δt)
    if spectrum
        return fourier(t_samples, s.(t_samples))
    else
        return (t_samples, s.(t_samples))
    end
end


eta = 0.9

function f1(x)
    (1.0 / 2.0) * (x[1]^2 + eta * x[2]^2)
end

function g1(x, storage)
    storage[1] = x[1]
    storage[2] = eta * x[2]
end

function h1(x, storage)
    storage[1, 1] = 1.0
    storage[1, 2] = 0.0
    storage[2, 1] = 0.0
    storage[2, 2] = eta
end

results = optimize(f1, g1, h1, [127.0, 921.0])
@assert results.converged
@assert norm(results.minimum - [0.0, 0.0]) < 0.01

results = optimize(f1, g1, [127.0, 921.0])
@assert results.converged
@assert norm(results.minimum - [0.0, 0.0]) < 0.01

results = optimize(f1, [127.0, 921.0])
@assert results.converged
@assert norm(results.minimum - [0.0, 0.0]) < 0.01


# # Variational problems
#
# In this example, we will numerically simulate an entropy-regularised Wasserstein gradient flow
# approximating the Fokker-Planck and porous medium equations.
#
# The connection between Wasserstein gradient flows and (non)-linear PDEs is due to Jordan, Kinderlehrer and Otto [^JKO98], and
# an easy-to-read overview of the topic is provided in Section 9.3 [^PC19]
#
# [^JKO98]: Jordan, Richard, David Kinderlehrer, and Felix Otto. "The variational formulation of the Fokker--Planck equation." SIAM journal on mathematical analysis 29.1 (1998): 1-17.
# [^PC19]: Peyré, Gabriel, and Marco Cuturi. "Computational optimal transport: With applications to data science." Foundations and Trends® in Machine Learning 11.5-6 (2019): 355-607.
#
# ## Fokker-Planck equation as a $W_2$ gradient flow
# For a potential function $\Psi$ and noise level $\sigma^2$, the Fokker-Planck equation (FPE) is
# ```math
# \partial_t \rho_t = \nabla \cdot (\rho_t \nabla \Psi) + \frac{\sigma^2}{2} \Delta \rho_t,
# ```
# and we take no-flux (Neumann) boundary conditions.
#
# This describes the evolution of a massless particle undergoing both diffusion (with diffusivity $\sigma^2$) and drift (along potential $\Psi$) according to the stochastic differential equation
# ```math
# dX_t = -\nabla \Psi(X_t) dt + \sigma dB_t.
# ```
# The result of Jordan, Kinderlehrer and Otto (commonly referred to as the JKO theorem) states that
# $\rho_t$ evolves following the 2-Wasserstein gradient flow of the Gibbs free energy functional
# ```math
#   F(\rho) = \int \Psi d\rho + \int \log(\rho) d\rho.
# ```
#
# ## Implicit schemes for gradient flows
# In an Euclidean space, the gradient flow of a functional $F$ is simply the solution of an ordinary differential equation
# ```math
#  \dfrac{dx(t)}{dt} = -\nabla F(x(t)).
# ```
# Of course, there is a requirement that $F$ is smooth. A more general formulation of a gradient flow that allows
# $F$ to be non-smooth is the implicit scheme
# ```math
#   x_{t+\tau} = \operatorname{argmin}_x \frac{1}{2} \| x - x_t \|_2^2 + \tau F(x).
# ```
# As the timestep $\tau$ shrinks, $x_t$ becomes a better and better approximation to the gradient flow of $F$.
#
# ## Wasserstein gradient flow
# In the context of the JKO theorem, we seek $\rho_t$ that is the gradient flow of $F$ with
# respect to the 2-Wasserstein distance. This can be achieved by choosing the $W_2$ metric in the implicit step:
# ```math
#   \rho_{t + \tau} = \operatorname{argmin}_{\rho} d_{W_2}^2(\rho_{t}, \rho) + \tau F(\rho).
# ```
# Finally, a numerical scheme for computing this gradient flow can be developed by using the entropic regularisation
# of optimal transport on a discretised domain
# ```math
#   \rho_{t + \tau} = \operatorname{argmin}_{\rho} \operatorname{OT}_\varepsilon(\rho_{t}, \rho) + \tau F(\rho),
# ```
# where
# ```math
#   \operatorname{OT}_\varepsilon(\alpha, \beta) = \min_{\gamma \in \Pi(\alpha, \beta)} \sum_{i,j} \frac{1}{2} \| x_i - x_j \|_2^2 \gamma_{ij} + \varepsilon \sum_{i, j} \gamma_{ij} \log(\gamma_{ij}).
# ```
# Each step of this problem is a minimisation problem with respect to $\rho$.
# Since we use entropic optimal transport which is differentiable, this can be solved using gradient-based methods.

# ## Problem setup
#
using OptimalTransport
using Distances
using LogExpFunctions
using Optim
using Plots
using StatsBase
using ReverseDiff

using LinearAlgebra
using Logging

# Here, we set up the computational domain that we work on - we discretize the interval $[-1, 1]$.
# The natural boundary conditions to use will be Neumann (zero flux), see e.g. [^Santam2017]
#
# [^Santam2017]: Santambrogio, Filippo. "{Euclidean, metric, and Wasserstein} gradient flows: an overview." Bulletin of Mathematical Sciences 7.1 (2017): 87-154.

support = range(-1, 1; length=64)
C = pairwise(SqEuclidean(), support');

# Now we set up various functionals that we will use.
#
# We define the generalised entropy (Equation (4.4) of [^Peyre2015]) as follows. For $m = 1$ this is just the "regular" entropy, and $m = 2$ this is squared $L_2$.
#
# [^Peyre2015]: Peyré, Gabriel. "Entropic approximation of Wasserstein gradient flows." SIAM Journal on Imaging Sciences 8.4 (2015): 2323-2351.
function E(ρ; m=1)
    if m == 1
        return sum(xlogx.(ρ)) - sum(ρ)
    elseif m > 1
        return dot(ρ, @. (ρ^(m - 1) - m) / (m - 1))
    end
end;

# Now define $\psi(x)$ to be a potential energy function that has two potential wells at $x = ± 0.5$.
ψ(x) = 10 * (x - 0.5)^2 * (x + 0.5)^2;
plot(support, ψ.(support); color="black", label="Scalar potential")

# Having defined $\psi$, this induces a potential energy functional $\Psi$ on probability distributions $\rho$:
# ```math
#    \Psi(\rho) = \int \psi(x) \rho(x) dx = \langle \psi, \rho \rangle .
# ```
Ψ = ψ.(support);

# Define the time step $\tau$ and entropic regularisation level $\varepsilon$, and form the associated Gibbs kernel $K = e^{-C/\varepsilon}$.
τ = 0.05
ε = 0.01
K = @. exp(-C / ε);

# We define the (non-smooth) initial condition $\rho_0$ in terms of step functions.
H(x) = x > 0
ρ0 = @. H(support + 0.25) - H(support - 0.25)
ρ0 = ρ0 / sum(ρ0)
plot(support, ρ0; label="Initial condition ρ0", color="blue")

# `G_fpe` is the objective function for the implicit step scheme
# ```math
# G_\mathrm{fpe}(\rho) = \operatorname{OT}_\varepsilon(\rho_{t}, \rho) + \tau F(\rho),
# ```
# and we seek to minimise in $\rho$.
function G_fpe(ρ, ρ0, τ, ε, C)
    return sinkhorn2(ρ, ρ0, C, ε; regularization=true, maxiter=250) + τ * (dot(Ψ, ρ) + E(ρ))
end;

# `step` solves the implicit step problem to produce $\rho_{t + \tau}$ from $\rho_t$.
function step(ρ0, τ, ε, C, G)
    ## only print error messages
    obj = u -> G(softmax(u), ρ0, τ, ε, C)
    opt = with_logger(SimpleLogger(stderr, Logging.Error)) do
        optimize(
            obj,
            ones(size(ρ0)),
            LBFGS(),
            Optim.Options(; iterations=50, g_tol=1e-6);
            autodiff=:forward,
        )
    end
    return softmax(Optim.minimizer(opt))
end
# Now we simulate `N = 10` iterates of the gradient flow and plot the result.

N = 10
ρ = similar(ρ0, size(ρ0, 1), N)
ρ[:, 1] = ρ0
for i in 2:N
    @info i
    ρ[:, i] = step(ρ[:, i - 1], τ, ε, C, G_fpe)
end
colors = range(colorant"red"; stop=colorant"blue", length=N)
plot(
    support,
    ρ;
    title=raw"$F(\rho) = \langle \psi, \rho \rangle + \langle \rho, \log(\rho) \rangle$",
    palette=colors,
    legend=nothing,
)

# ## Porous medium equation
#
# The porous medium equation (PME) is the nonlinear PDE
# ```math
# \partial_t \rho = \nabla \cdot (\rho \nabla \Psi) + \Delta \rho^m,
# ```
# again with Neumann boundary conditions. The value of $m$ in the PME corresponds to picking $m$ in the generalised entropy functional.
# Now, we will solve the PME with $m = 2$ as a Wasserstein gradient flow.
#
function G_pme(ρ, ρ0, τ, ε, C)
    return sinkhorn2(ρ, ρ0, C, ε; regularization=true, maxiter=250) +
           τ * (dot(Ψ, ρ) + E(ρ; m=2))
end;

# set up as previously
N = 10
ρ = similar(ρ0, size(ρ0, 1), N)
ρ[:, 1] = ρ0
for i in 2:N
    ρ[:, i] = step(ρ[:, i - 1], τ, ε, C, G_pme)
end
plot(
    support,
    ρ;
    title=raw"$F(\rho) = \langle \psi, \rho \rangle + \langle \rho, \rho - 1\rangle$",
    palette=colors,
    legend=nothing,
)

# ## Exploiting duality
#
# The previous examples solved the minimisation problem for the implicit gradient flow step directly, involving automatic differentiation through the Sinkhorn iterations used to compute $\operatorname{OT}_\varepsilon(\rho_t, \rho)$ each time a gradient needs to be evaluated.
# While this is straightforward to implement, it is computationally costly.
# An alternative approach for convex variational problems is to proceed via the [dual problem](https://en.wikipedia.org/wiki/Duality_(optimization)).
# The benefit of proceeding via the dual problem is that the part of the dual minimisation problem corresponding to the (entropy-regularised) optimal transport loss is typically available in closed form. This is in contrast to the primal problem, where evaluation of the objective and its gradients requires potentially many Sinkhorn iterations.
#
# Consider a general convex and unconstrained problem. Under (usually satisfied) conditions for strong duality to hold, we have
# ```math
# \begin{aligned}
# &\min_{\rho} \operatorname{OT}_{\varepsilon}(\rho_0, \rho) + \mathcal{F}(\rho)  \\
# &= \min_{\rho} \sup_{u}\left[\langle \rho, u \rangle - \operatorname{OT}^*_{\varepsilon}(\rho_0, u)\right] + \mathcal{F}(\rho)  \\
# &= \sup_{u} \min_{\rho} \langle \rho, u \rangle - \operatorname{OT}^*_{\varepsilon}(\rho_0, u) + \mathcal{F}(\rho) \\
# &= \sup_{u} - \operatorname{OT}^*_{\varepsilon}(\rho_0, u) + \min_{\rho} \langle \rho, u \rangle + \mathcal{F}(\rho) \\
# &= \sup_{u} - \operatorname{OT}^*_{\varepsilon}(\rho_0, u) - \sup_{\rho} \langle \rho, -u \rangle - \mathcal{F}(\rho) \\
# &= \sup_{u} - \operatorname{OT}^*_{\varepsilon}(\rho_0, u) - \mathcal{F}^*(-u).
# \end{aligned}
# ```
# Thus, the dual problem is
# ```math
# \min_{u} \operatorname{OT}^*_{\varepsilon}(\rho_0, u) + \mathcal{F}^*(-u).
# ```
#
# The upshot here is that $u \mapsto \operatorname{OT}^*_{\varepsilon}(\rho_0, u)$ and its gradient is available in closed form. This is a known result in the literature [^CP18].
#
# [^CP18]: Cuturi, Marco, and Gabriel Peyré. “Semi-Dual Regularized Optimal Transport.” ArXiv: Learning, 2018.
#
# The formulas we state below are lifted from statements in [^Z21].
#
# [^Z21]: Zhang, Stephen Y. “A Unified Framework for Non-Negative Matrix and Tensor Factorisations with a Smoothed Wasserstein Loss.” ArXiv: Machine Learning, 2021.
#
# ```math
# \begin{aligned}
# \operatorname{OT}^*_{\varepsilon}(\rho_0, u) &= -\varepsilon \left\langle \rho_0, \log\left( \dfrac{\rho_0}{K e^{u/\varepsilon}} \right) - 1\right\rangle, \\
# \nabla_u \operatorname{OT}^*_{\varepsilon}(\rho_0, u) &= K^\top \left( \dfrac{\rho_0}{K e^{u/\varepsilon}} \right) \odot e^{u/\varepsilon}.
# \end{aligned}
# ```
# At optimality, we can recover the primal optimal point $\rho^\star$ from the dual optimal point $u^\star$ following the formula
# ```math
# \rho^\star = e^{u^\star/\varepsilon} \odot K^\top \dfrac{\rho_0}{K e^{u^\star/\varepsilon}}.
# ```
#
# When $\mathcal{F}^*(\cdot)$ is also available in closed form (this is not always the case), the dual problem has a closed form objective and can generally be solved much more efficiently than the primal problem.
#
# In the setting of the Fokker-Planck and porous medium equations, the function $\mathcal{F}$ can be identified with
#
# ```math
# \mathcal{F}(\rho) = \tau \left[ \langle \psi, \rho \rangle + E_m(\rho) \right].
# ```
#
# A straightforward computation shows that
# ```math
# \mathcal{F}^*(u) = \tau E_m^*\left( \frac{u}{\tau}-\psi \right),
# ```
# where
# ```math
#     E_m^*(u) = \begin{cases}
#     \langle e^u, \mathbf{1} \rangle, & m = 1 \\
#     \sum_i \left[ \left( u_i + \frac{m}{m-1} \right) \left( \frac{m-1}{m} u_i + 1 \right)^{\frac{1}{m-1}} - \frac{1}{m-1} \left( \frac{m-1}{m} u_i + 1 \right)^{\frac{m}{m-1}} \right], & m > 1.
#     \end{cases}
# ```
# In particular, for $m = 2$ we have a simpler formula
# ```math
# E_2^*(u) = \left\| \frac{u}{2} + 1 \right\|_2^2
# ```
#
# We now implement $E_m^*$ for $m = 1, 2$.
E_dual(u, m::Val{1}) = sum(exp.(u))
function E_dual(u, m::Val{2})
    return dot(u / 2 .+ 1, u / 2 .+ 1)
end;
#
# So, the dual problem we are dealing with reads
# ```math
# \min_{u} \operatorname{OT}^*_{\varepsilon}(\rho_0, u) + \tau E_m^*\left( \frac{-u}{\tau}-\psi \right),
# ```
# and we can thus set up `G_dual_fpe`, the dual objective.
#
function G_dual_fpe(u, ρ0, τ, ε, K)
    return OptimalTransport.Dual.ot_entropic_semidual(ρ0, u, ε, K) +
           τ * E_dual(-u / τ - Ψ, Val(1))
end;
#
# Now we set up `step` as previously, except this time we need to convert from the optimal dual variable $u^\star$ to the primal variable $\rho^\star$. In the code, this is handled by `getprimal_ot_entropic_semidual`. We use `ReverseDiff` in this problem.
#
function step(ρ0, τ, ε, K, G)
    obj = u -> G(u, ρ0, τ, ε, K)
    opt = optimize(
        obj,
        (∇, u) -> ReverseDiff.gradient!(∇, obj, u),
        zeros(size(ρ0)),
        LBFGS(),
        Optim.Options(; iterations=250, g_tol=1e-6),
    )
    return OptimalTransport.Dual.getprimal_ot_entropic_semidual(
        ρ0, Optim.minimizer(opt), ε, K
    )
end;
#
# Now we can solve the dual problem as previously, and we note that the dual formulation is solved an order of magnitude faster than the primal formulation.
#
ρ = similar(ρ0, size(ρ0, 1), N)
ρ[:, 1] = ρ0
for i in 2:N
    ρ[:, i] = step(ρ[:, i - 1], τ, ε, K, G_dual_fpe)
end
colors = range(colorant"red"; stop=colorant"blue", length=N)
plot(
    support,
    ρ;
    title=raw"$F(\rho) = \langle \psi, \rho \rangle + \langle \rho, \log(\rho) \rangle$",
    palette=colors,
    legend=nothing,
)

# Setting `m = 2`, we can simulate instead the porous medium equation.
#
function G_dual_pme(u, ρ0, τ, ε, K)
    return OptimalTransport.Dual.ot_entropic_semidual(ρ0, u, ε, K) +
           τ * E_dual(-u / τ - Ψ, Val(2))
end
ρ = similar(ρ0, size(ρ0, 1), N)
ρ[:, 1] = ρ0
for i in 2:N
    @info i
    ρ[:, i] = step(ρ[:, i - 1], τ, ε, K, G_dual_pme)
end
colors = range(colorant"red"; stop=colorant"blue", length=N)
plot(
    support,
    ρ;
    title=raw"$F(\rho) = \langle \psi, \rho \rangle + \langle \rho, \rho - 1\rangle$",
    palette=colors,
    legend=nothing,
)