## Laplacian

function _imfilter_inbounds!(r::AbstractResource, out, A::AbstractArray, L::Laplacian, border::NoPad, inds)
    TT = eltype(out) # accumtype(eltype(out), eltype(A))
    n = 2*length(L.offsets)
    R = CartesianIndices(inds)
    @inbounds for I in R
        tmp = convert(TT, - n * A[I])
        for J in L.offsets
            tmp += A[I+J]
            tmp += A[I-J]
        end
        out[I] = tmp
    end
    out
end

## imgradients
"""
```julia
    imgradients(img, kernelfun=KernelFactors.ando3, border="replicate") -> gimg1, gimg2, ...
```
Estimate the gradient of `img` in the direction of the first and second dimension
at all points of the image, using a kernel specified by `kernelfun`.

# Output

The gradient is returned as a tuple-of-arrays, one for each dimension of the
input; `gimg1` corresponds to the derivative with respect to the first
dimension, `gimg2` to the second, and so on.

# Details

To appreciate the difference between various gradient estimation methods
it is helpful to distinguish between: (1) a continuous scalar-valued
*analogue* image ``f_\\textrm{A}(x_1,x_2)``, where ``x_1,x_2 \\in
\\mathbb{R}``, and (2) its discrete *digital* realization
``f_\\textrm{D}(x_1',x_2')``, where ``x_1',x_2' \\in \\mathbb{N}``, ``1
\\le x_1' \\le M`` and ``1 \\le x_2' \\le N``.

## Analogue image

The gradient of a continuous analogue image ``f_{\\textrm{A}}(x_1,x_2)`` at
location ``(x_1,x_2)`` is defined as the vector
```math
\\nabla \\mathbf{f}_{\\textrm{A}}(x_1,x_2) = \\frac{\\partial
f_{\\textrm{A}}(x_1,x_2)}{\\partial x_1} \\mathbf{e}_{1} +
\\frac{\\partial f_{\\textrm{A}}(x_1,x_2)}{\\partial x_2} \\mathbf{e}_{2},
```
where ``\\mathbf{e}_{d}`` ``(d = 1,2)`` is the unit
vector in the ``x_d``-direction. The gradient points in the direction of
maximum rate of change of ``f_{\\textrm{A}}`` at the coordinates
``(x_1,x_2)``. The gradient can be used to compute the derivative of a
function in an arbitrary direction. In particular, the derivative of
``f_{\\textrm{A}}`` in the direction of a unit vector ``\\mathbf{u}`` is
given by ``\\nabla_{\\mathbf{u}}f_\\textrm{A}(x_1,x_2) = \\nabla
\\mathbf{f}_{\\textrm{A}}(x_1,x_2) \\cdot \\mathbf{u}``, where
``\\cdot`` denotes the dot product.

## Digital image

In practice, we acquire a digital image ``f_\\textrm{D}(x_1',x_2')`` where
the light intensity is known only at a discrete set of locations. This
means that the required partial derivatives are undefined and need to be
approximated using discrete difference formulae [1].

A straightforward way to approximate the partial derivatives is to use
central-difference formulae
```math
 \\frac{\\partial f_{\\textrm{D}}(x_1',x_2')}{\\partial x_1'}  \\approx
        \\frac{f_{\\textrm{D}}(x_1'+1,x_2') - f_{\\textrm{D}}(x_1'-1,x_2') }{2}
```
and
```math
 \\frac{\\partial f_{\\textrm{D}}(x_1',x_2')}{\\partial x_2'}   \\approx
         \\frac{f_{\\textrm{D}}(x_1',x_2'+1) - f_{\\textrm{D}}(x_1',x_2'+1)}{2}.
```
However, the central-difference formulae are very sensitive to noise.
When working with noisy image data,
one can obtain a better approximation of the partial
derivatives by using a suitable weighted combination of the neighboring
image intensities. The weighted combination can be represented as a
*discrete convolution* operation between the image and a
*kernel* which characterizes the requisite weights. In particular,
if ``h_{x_d}`` (``d = 1,2)`` represents a ``2r+1 \\times 2r+1`` kernel, then
```math
 \\frac{\\partial f_{\\textrm{D}}(x_1',x_2')}{\\partial x_d'}  \\approx
\\sum_{i = -r}^r \\sum_{j = -r}^r
f_\\textrm{D}(x_1'-i,x_2'-j)
  h_{x_d}(i,j).
```
The kernel is frequently also called a *mask* or *convolution matrix*.

### Weighting schemes and approximation error

The choice of weights determines the magnitude of the approximation
error and whether the finite-difference scheme is *isotropic*. A
finite-difference scheme is isotropic if the approximation error does
not depend on the orientation of the coordinate system and
*anisotropic* if the approximation error has a directional bias [2].
With a continuous analogue image the magnitude of the gradient would be
invariant upon rotation of the coordinate system, but in practice one
cannot obtain perfect isotropy with a finite set of discrete points.
Hence a finite-difference scheme is typically considered isotropic if
the leading error term in the approximation does not have preferred
directions.

Most finite-difference schemes that are used in image processing are
based on ``3 \\times 3`` kernels, and as noted by [7], many can also be
parametrized by a single parameter ``\\alpha`` as follows:

```math
\\mathbf{H}_{x_{1}} =
\\frac{1}{4 + 2\\alpha}
\\begin{bmatrix}
-1 & -\\alpha & -1 \\\\
0 & 0 & 0 \\\\
 1 & \\alpha & 1
\\end{bmatrix}
\\quad
\\text{and}
\\quad
\\mathbf{H}_{x_{2}} =
\\frac{1}{2 + 4\\alpha}
\\begin{bmatrix}
-1 & 0 & 1 \\\\
-\\alpha & 0 & \\alpha \\\\
 -1 & 0 & 1
\\end{bmatrix},
```
where
```math
\\alpha =
\\begin{cases}
0,  & \\text{Simple Finite Difference}; \\\\
1, &  \\text{Prewitt}; \\\\
2, &  \\text{Sobel}; \\\\
2.4351, &  \\text{Ando}; \\\\
\\frac{10}{3}, &  \\text{Scharr}; \\\\
4, &  \\text{Bickley}.
\\end{cases}
```

## Separable kernel

A kernel is called *separable* if it can be expressed as the
convolution of two one-dimensional filters. With a matrix representation
of the kernel, separability means that the kernel matrix can be written
as an outer product of two vectors. Separable kernels offer
computational advantages since instead of performing a two-dimensional
convolution one can perform a sequence of one-dimensional convolutions.

# Options
You can specify your choice of the finite-difference scheme via the `kernelfun`
parameter. You can also indicate how to deal with the pixels on the border
of the image with the `border` parameter.

## Choices for `kernelfun`
In general `kernelfun` can be any function which satisfies the following
interface:
```julia
    kernelfun(extended::NTuple{N,Bool}, d) -> kern_d,
```
where `kern_d` is the kernel for producing the derivative with respect to
the ``d``th dimension of an ``N``-dimensional array. The parameter `extended[i]` is true
if the image is of size > 1 along dimension ``i``. The parameter `kern_d` may be
provided as a dense or factored kernel, with factored representations
recommended when the kernel is separable.

Some valid `kernelfun` options are described below.

### `KernelFactors.prewitt`

With the *prewit* option [3] the computation of the gradient is based on
the kernels
```math
\\begin{aligned}
\\mathbf{H}_{x_1} & = \\frac{1}{6}
    \\begin{bmatrix}
    -1 & -1 & -1 \\\\
    0 & 0 & 0 \\\\
    1 & 1 & 1
    \\end{bmatrix}
&
\\mathbf{H}_{x_2} & =  \\frac{1}{6}
    \\begin{bmatrix}
    -1 & 0 & 1 \\\\
    -1 & 0 & 1 \\\\
    -1 & 0 & 1
    \\end{bmatrix} \\\\
& = \\frac{1}{6}
    \\begin{bmatrix}
    1 \\\\
    1  \\\\
    1
    \\end{bmatrix}
    \\begin{bmatrix}
    -1 & 0 & 1
    \\end{bmatrix}
&
& = \\frac{1}{6}
    \\begin{bmatrix}
    -1 \\\\
    0  \\\\
    1
    \\end{bmatrix}
    \\begin{bmatrix}
    1 & 1 & 1
    \\end{bmatrix}.
\\end{aligned}
```
See also: [`KernelFactors.prewitt`](@ref) and [`Kernel.prewitt`](@ref)

### `KernelFactors.sobel`

The *sobel* option [4] designates the kernels
```math
\\begin{aligned}
\\mathbf{H}_{x_1} & = \\frac{1}{8}
    \\begin{bmatrix}
    -1 & -2 & -1 \\\\
     0 & 0 & 0 \\\\
     1 & 2 & 1
    \\end{bmatrix}
&
\\mathbf{H}_{x_2} & = \\frac{1}{8}
    \\begin{bmatrix}
    -1 & 0 & 1 \\\\
    -2 & 0 & 2 \\\\
    -1 & 0 & 1
    \\end{bmatrix} \\\\
& = \\frac{1}{8}
    \\begin{bmatrix}
    -1 \\\\
    0  \\\\
    1
    \\end{bmatrix}
    \\begin{bmatrix}
    1 & 2 & 1
    \\end{bmatrix}
&
& = \\frac{1}{8}
    \\begin{bmatrix}
    1 \\\\
    2  \\\\
    1
    \\end{bmatrix}
    \\begin{bmatrix}
    -1 & 0 & 1
    \\end{bmatrix}.
\\end{aligned}
```
See also:  [`KernelFactors.sobel`](@ref) and [`Kernel.sobel`](@ref)

### `KernelFactors.ando3`
The *ando3* option [5] specifies the kernels
```math
\\begin{aligned}
\\mathbf{H}_{x_1} &  =
    \\begin{bmatrix}
    -0.112737 & -0.274526 & -0.112737 \\\\
     0 & 0 & 0 \\\\
     0.112737 & 0.274526 & 0.112737
    \\end{bmatrix}
&
\\mathbf{H}_{x_2}  & =
    \\begin{bmatrix}
    -0.112737 & 0 & 0.112737 \\\\
    -0.274526 & 0 & 0.274526 \\\\
    -0.112737 & 0 & 0.112737
    \\end{bmatrix} \\\\
&  = \\begin{bmatrix}
    -1 \\\\
    0  \\\\
    1
    \\end{bmatrix}
    \\begin{bmatrix}
    0.112737 & 0.274526 & 0.112737
    \\end{bmatrix}
&
&  = \\begin{bmatrix}
    0.112737 \\\\
    0.274526  \\\\
    0.112737
    \\end{bmatrix}
    \\begin{bmatrix}
    -1 & 0 & 1
    \\end{bmatrix}.
\\end{aligned}
```
See also:  [`KernelFactors.ando3`](@ref), and
[`Kernel.ando3`](@ref);  [`KernelFactors.ando4`](@ref), and
[`Kernel.ando4`](@ref); [`KernelFactors.ando5`](@ref), and
[`Kernel.ando5`](@ref)

### `KernelFactors.scharr`

The *scharr* option [6] designates the kernels
```math
\\begin{aligned}
\\mathbf{H}_{x_{1}} & =
\\frac{1}{32}
\\begin{bmatrix}
-3 & -10 & -3 \\\\
0 & 0 & 0 \\\\
 3 & 10 & 3
\\end{bmatrix}
&
\\mathbf{H}_{x_{2}} & =
\\frac{1}{32}
\\begin{bmatrix}
-3 & 0 & 3 \\\\
-10 & 0 & 10\\\\
-3 & 0 & 3
\\end{bmatrix} \\\\
& = \\frac{1}{32}
\\begin{bmatrix}
    -1 \\\\
    0  \\\\
    1
\\end{bmatrix}
\\begin{bmatrix}
    3 & 10 & 3
\\end{bmatrix}
&
& = \\frac{1}{32}
\\begin{bmatrix}
    3 \\\\
    10  \\\\
    3
\\end{bmatrix}
\\begin{bmatrix}
    -1 & 0 & 1
\\end{bmatrix}.
\\end{aligned}
```
See also:  [`KernelFactors.scharr`](@ref) and [`Kernel.scharr`](@ref)

### `KernelFactors.bickley`

The *bickley* option [7,8] designates the kernels
```math
\\begin{aligned}
\\mathbf{H}_{x_1} & = \\frac{1}{12}
    \\begin{bmatrix}
        -1 & -4 & -1 \\\\
         0 & 0 & 0 \\\\
         1 & 4 & 1
    \\end{bmatrix}
&
\\mathbf{H}_{x_2} & = \\frac{1}{12}
    \\begin{bmatrix}
        -1 & 0 & 1 \\\\
        -4 & 0 & 4 \\\\
        -1 & 0 & 1
    \\end{bmatrix} \\\\
& = \\frac{1}{12}
    \\begin{bmatrix}
        -1 \\\\
        0  \\\\
        1
    \\end{bmatrix}
    \\begin{bmatrix}
        1 & 4 & 1
    \\end{bmatrix}
&
&  = \\frac{1}{12}
   \\begin{bmatrix}
        1 \\\\
        4  \\\\
        1
   \\end{bmatrix}
   \\begin{bmatrix}
        -1 & 0 & 1
   \\end{bmatrix}.
\\end{aligned}
```
See also:  [`KernelFactors.bickley`](@ref) and [`Kernel.bickley`](@ref)

## Choices for `border`
At the image edge, `border` is used to specify the padding which will be used
to extrapolate the image beyond its original bounds. As an indicative example
of each option the results of the padding are illustrated on an image consisting of
a row of six pixels which are specified alphabetically: ``\\boxed{a \\, b \\, c \\, d \\, e \\, f}``.
We show the effects of padding only on the left and right border, but analogous
consequences hold for the top and bottom border.

### `"replicate"`

The border pixels extend beyond the image boundaries.
```math
\\boxed{
\\begin{array}{l|c|r}
  a\\, a\\, a\\, a  &  a \\, b \\, c \\, d \\, e \\, f & f \\, f \\, f \\, f
\\end{array}
}
```
See also: [`Pad`](@ref), [`padarray`](@ref), [`Inner`](@ref) and
[`NoPad`](@ref)

### `"circular"`

The border pixels wrap around. For instance, indexing beyond the left border
returns values starting from the right border.
```math
\\boxed{
\\begin{array}{l|c|r}
  c\\, d\\, e\\, f  &  a \\, b \\, c \\, d \\, e \\, f & a \\, b \\, c \\, d
\\end{array}
}
```
See also: [`Pad`](@ref), [`padarray`](@ref), [`Inner`](@ref) and
[`NoPad`](@ref)

### `"symmetric"`
The border pixels reflect relative to a position between pixels. That is, the
border pixel is omitted when mirroring.
```math
\\boxed{
\\begin{array}{l|c|r}
  e\\, d\\, c\\, b  &  a \\, b \\, c \\, d \\, e \\, f & e \\, d \\, c \\, b
\\end{array}
}
```
See also: [`Pad`](@ref), [`padarray`](@ref), [`Inner`](@ref) and
[`NoPad`](@ref)

### `"reflect"`
The border pixels reflect relative to the edge itself.
```math
\\boxed{
\\begin{array}{l|c|r}
  d\\, c\\, b\\, a  &  a \\, b \\, c \\, d \\, e \\, f & f \\, e \\, d \\, c
\\end{array}
}
```
See also: [`Pad`](@ref), [`padarray`](@ref), [`Inner`](@ref) and
[`NoPad`](@ref)

# Example

This example compares the quality of the gradient estimation methods in terms of
the accuracy with which the orientation of the gradient is estimated.

```julia
using Images

values = LinRange(-1,1,128);
w = 1.6*pi;

# Define a function of a sinusoidal grating, f(x,y) = sin( (w*x)^2 + (w*y)^2 ),
# together with its exact partial derivatives.
I = [sin( (w*x)^2 + (w*y)^2 ) for y in values, x in values];
Ix = [2*w*x*cos( (w*x)^2 + (w*y)^2 ) for y in values, x in values];
Iy = [2*w*y*cos( (w*x)^2 + (w*y)^2 ) for y in values, x in values];

# Determine the exact orientation of the gradients.
direction_true = atan.(Iy./Ix);

for kernelfunc in (KernelFactors.prewitt, KernelFactors.sobel,
                   KernelFactors.ando3, KernelFactors.scharr,
                   KernelFactors.bickley)

    # Estimate the gradients and their orientations.
    Gy, Gx = imgradients(I,kernelfunc, "replicate");
    direction_estimated = atan.(Gy./Gx);

    # Determine the mean absolute deviation between the estimated and true
    # orientation. Ignore the values at the border since we expect them to be
    # erroneous.
    error = mean(abs.(direction_true[2:end-1,2:end-1] -
                     direction_estimated[2:end-1,2:end-1]));

    error = round(error, digits=5);
    println("Using \$kernelfunc results in a mean absolute deviation of \$error")
end

# output

Using ImageFiltering.KernelFactors.prewitt results in a mean absolute deviation of 0.01069
Using ImageFiltering.KernelFactors.sobel results in a mean absolute deviation of 0.00522
Using ImageFiltering.KernelFactors.ando3 results in a mean absolute deviation of 0.00365
Using ImageFiltering.KernelFactors.scharr results in a mean absolute deviation of 0.00126
Using ImageFiltering.KernelFactors.bickley results in a mean absolute deviation of 0.00038
```
# References
  1. B. Jahne, *Digital Image Processing* (5th ed.). Springer Publishing Company, Incorporated, 2005. [10.1007/3-540-27563-0](https://doi.org/10.1007/3-540-27563-0)
  2. M. Patra  and  M. Karttunen, "Stencils with isotropic discretization error for differential operators," *Numer. Methods Partial Differential Eq.*, vol. 22, pp. 936–953, 2006. [doi:10.1002/num.20129](https://doi.org/doi:10.1002/num.20129)
  3. J. M. Prewitt, "Object enhancement and extraction," *Picture processing and Psychopictorics*, vol. 10, no. 1, pp. 15–19, 1970.
  4. P.-E. Danielsson and O. Seger, "Generalized and separable sobel operators," in  *Machine Vision for Three-Dimensional Scenes*,  H. Freeman, Ed.  Academic Press, 1990,  pp. 347–379. [doi:10.1016/b978-0-12-266722-0.50016-6](https://doi.org/doi:10.1016/b978-0-12-266722-0.50016-6)
  5. S. Ando, "Consistent gradient operators," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 22, no.3, pp. 252–265, 2000. [doi:10.1109/34.841757](https://doi.org/doi:10.1109/34.841757)
  6. H. Scharr and  J. Weickert, "An anisotropic diffusion algorithm with optimized rotation invariance," *Mustererkennung 2000*, pp. 460–467, 2000. [doi:10.1007/978-3-642-59802-9_58](https://doi.org/doi:10.1007/978-3-642-59802-9_58)
  7. A. Belyaev, "Implicit image differentiation and filtering with applications to image sharpening," *SIAM Journal on Imaging Sciences*, vol. 6, no. 1, pp. 660–679, 2013. [doi:10.1137/12087092x](https://doi.org/doi:10.1137/12087092x)
  8. W. G. Bickley, "Finite difference formulae for the square lattice," *The Quarterly Journal of Mechanics and Applied Mathematics*, vol. 1, no. 1, pp. 35–42, 1948.  [doi:10.1093/qjmam/1.1.35](https://doi.org/doi:10.1093/qjmam/1.1.35)

***

"""
function imgradients(img::AbstractArray, kernelfun::Function, border="replicate")
    extended = map(isextended, axes(img))
    _imgradients(extended, img, kernelfun, extended, border)
end

isextended(ind) = length(ind) > 1

# Add the next dimension to G
function _imgradients(donewhenempty::NTuple{M}, img::AbstractArray{T,N}, kernelfun::Function, extended, border) where {T,M,N}
    d = N-M+1  # the dimension we're working on now
    kern = kernelfun(extended, d)
    return (imfilter(img, kern, border), _imgradients(Base.tail(donewhenempty), img, kernelfun, extended, border)...)
end
# When all N gradients have been calculated, return the result
_imgradients(::Tuple{}, img::AbstractArray, kernelfun::Function, extent, border) = ()


using ApproxFun

###
# Fractional Derivative
###

S = Legendre()

n = 50; # number of data points
p = range(-1,stop=1,length=n);   # a non-default grid

v = exp.(p);           # values at the non-default grid
m = 20   # number of basis functions
V = Array{Float64}(undef,n,m); # Create a Vandermonde matrix by evaluating the basis at the grid
for k = 1:size(V,2)
   V[:,k] = Fun(S,[zeros(k-1);1]).(p)
end
f = Fun(S,V\v); # least squares fit to data

f_rl = LeftDerivative(0.5)*f # RL left half-derivative
f_cap = LeftIntegral(0.5)*Fun(f', S) # Caputo left half-derivative

using Plots
plot(f_rl;ylims=(0,3),legend=:bottomright,label="RL")
plot!(f_cap; label="Caputo")
savefig("RLvCaputo.png")


using Random, CSV, Distributions, Plots; pyplot()
Random.seed!(0)

sampleData = CSV.read("../data/machine1.csv", header=false)[:,1]
n, N = length(sampleData), 10^6
alpha = 0.1

bootstrapSampleMeans = [mean(rand(sampleData, n)) for i in 1:N]
Lmean = quantile(bootstrapSampleMeans, alpha/2)
Umean = quantile(bootstrapSampleMeans, 1-alpha/2)

bootstrapSampleMedians = [median(rand(sampleData, n)) for i in 1:N]
Lmed = quantile(bootstrapSampleMedians, alpha/2)
Umed = quantile(bootstrapSampleMedians, 1-alpha/2)

println("Bootstrap confidence interval for the mean: ", (Lmean, Umean) )
println("Bootstrap confidence interval for the median: ", (Lmed, Umed) )

stephist(bootstrapSampleMeans, bins=1000, c=:blue,
    normed=true, label="Sample \nmeans")
plot!([Lmean, Lmean],[0,2], c=:black, ls=:dash, label="90% CI")
plot!([Umean, Umean],[0,2],c=:black, ls=:dash, label="",
    xlims=(52,54), ylims=(0,2), xlabel="Sample Means", ylabel="Density")


export Fourier

struct Fourier <: AbstractMethod

    kx   :: Vector{Float64}
    kv   :: Vector{Float64}

    function Fourier( meshx, meshv )

        nx  = meshx.len
        dx  = meshx.step
        Lx  = meshx.stop - meshx.start
        kx  = 2π/Lx .* [0:nx÷2-1;-nx÷2:-1]

        nv  = meshv.len
        dv  = meshv.step
        Lv  = meshv.stop - meshv.start
        kv  = 2π/Lv .* [0:nv÷2-1;-nv÷2:-1]

        new( kx, kv)

    end

end

function advection_v!(fᵗ  :: Array{ComplexF64,2},
                      adv :: Fourier,
		      e   :: Vector{ComplexF64},
		      dt  :: Float64 )
    fft!(fᵗ, 1)
    @. fᵗ *= exp(-1im * dt * adv.kv * $transpose(e))
    ifft!(fᵗ, 1)

end

function advection_x!( f   :: Array{ComplexF64,2},
                       adv :: Fourier,
		       e   :: Vector{ComplexF64},
		       v   :: Vector{Float64},
		       dt  :: Float64 )

    ev = exp.(-1im*dt * adv.kx * transpose(v))

    fft!(f,1)
    f .= f .* ev
    dv = v[2]-v[1]
    ρ = dv * vec(sum(f,dims=2))
    for i in 2:length(e)
        e[i] = -1im * ρ[i] ./ adv.kx[i]
    end
    e[1] = 0.0
    ifft!(f,1)
    ifft!(e)
    e .= real(e)
end


CARDPK = 1526110
DOORPK = 20175123

SUBJECT = 7
SALT = 20201227

function find_loop_size(pk, subject)
    i = 0
    n = 1
    while n != pk
        i += 1
        n *= subject
        n = mod(n, SALT)
    end
    return i
end

function enc(subject, loops)
    n = 1
    for _ in 1:loops
        n *= subject
        n = mod(n, SALT)
    end
    return n
end

card = find_loop_size(CARDPK, SUBJECT)
door = find_loop_size(DOORPK, SUBJECT)
p1 = enc(DOORPK, card)

println("-----------------------------------------------------------------------")
println("combo breaker -- part one :: $p1")
println("-----------------------------------------------------------------------")

@assert(p1 == 10924063)


"""
    scale(a::Polynomial,ρ::Number)

Scale the univariate polynomial `a` with the variable `s` given by ``a(s) = a₀ + a₁ s + a₂ s² + … + aₙ sⁿ`` by the scalar positive real number `ρ`. This amounts to replacing the original variable `s` with a new scaled variable `ρs`.

# Examples

```julia
julia> a = Polynomial(1:5,:s)
julia> ρ=10
julia> scale(a,ρ)
Polynomial(1 + 20*s + 300*s^2 + 4000*s^3 + 50000*s^4)
```
"""
function scale(a::Polynomial,ρ::Number)
    c = deepcopy(coeffs(a))     # coefficent vector of the original polynomial
    for k=1:degree(a)           # k is the power to which s is raised in s^k
        c[k+1] *= ρ^k           # coefficient vector of the new scaled polynomial
    end
    p = Polynomial(c,a.var)     # converting the coefficient vector to a polynomial
end

"""
    cconj(a::Polynomial)

Return the conjugate of a given univariate polynomial `a` given by ``a(s) = a₀ + a₁ s + a₂ s² + … + aₙ sⁿ`` with respect to the imaginary axis, that is, return the polynomial ``ã(s) = ̄ā(-s)=ā₀ - ̄ā₁s + ā₂ s² + … ± ̄āₙ sⁿ``.

It is used in the analysis and synthesis of continuous-time filters and controllers. This is reflected in prepending the letter `c` to the `conj` function name.

# Examples

```julia
julia> a = Polynomial(1:5,:s)
Polynomial(1 + 2*s + 3*s^2 + 4*s^3 + 5*s^4)

julia> p = cconj(a)
Polynomial(1 - 2*s + 3*s^2 - 4*s^3 + 5*s^4)

julia> a = Polynomial([1+1im, 2+2im, 3+3im, 4+4im, 5+5im],:s)
Polynomial((1 + 1im) + (2 + 2im)*s + (3 + 3im)*s^2 + (4 + 4im)*s^3 + (5 + 5im)*s^4)

julia> p = cconj(a)
Polynomial((1 - 1im) - (2 - 2im)*s + (3 - 3im)*s^2 - (4 - 4im)*s^3 + (5 - 5im)*s^4)
```
"""
function cconj(a::Polynomial)
    c = conj(a)
    cc = deepcopy(coeffs(c))
    cc[2:2:end] = -cc[2:2:end]
    return Polynomial(cc,a.var)
end

"""
    conjreciprocal(a::Polynomial)

Return the conjugate reciprocal polynomial for a given polynomial `a`.

Fof a polynomial ``a(s) = a₀ + a₁ s + a₂ s² + … + aₙ sⁿ``, return the polynomial ``r(s) = ̄āₙ + ̄āₙ₋₁ s + ̄āₙ₋₂ s² + … + ̄ā₀ sⁿ``.

# Examples

```julia
julia> a = Polynomial(1:5,:s)
Polynomial(1 + 2*s + 3*s^2 + 4*s^3 + 5*s^4)

julia> p = conjreciprocal(a)
Polynomial(5 + 4*s + 3*s^2 + 2*s^3 + s^4)

julia> c = Polynomial([1+1im, 2+2im, 3+3im, 4+4im, 5+5im],:s)
Polynomial((1 + 1im) + (2 + 2im)*s + (3 + 3im)*s^2 + (4 + 4im)*s^3 + (5 + 5im)*s^4)

julia> p = conjreciprocal(c)
Polynomial((5 - 5im) + (4 - 4im)*s + (3 - 3im)*s^2 + (2 - 2im)*s^3 + (1 - 1im)*s^4)
```
"""
function conjreciprocal(a::Polynomial)
    c = conj(a)
    cc = deepcopy(coeffs(c))
    pc = cc[end:-1:1]
    p = Polynomial(pc,a.var)
end

"""
    dconj(a::LaurentPolynomial)

Return the conjugate polynomial for a given polynomial ``a(z) = a₀ + a₁ z + a₂ z² + ... + aₙ zⁿ`` with respect to the unit circle in the complex plane, that is, return the Laurent polynomial ``ã(z) = āₙ z⁻ⁿ + āₙ₋₁ z⁻ⁿ⁺¹ + āₙ₋₂ z⁻ⁿ⁺² + … + ā₀``.

The function is only defined for `LaurentPolynomial` type even if it is used to represent a standard polynomial (with no negative powers).

It is used in the analysis and synthesis of discrete-time filters and controllers. This is reflected in prepending the letter `d` to the `conj` function name.

# Examples

```julia
julia> a = LaurentPolynomial(1:5,:z)
LaurentPolynomial(1 + 2*z + 3*z² + 4*z³ + 5*z⁴)

julia> p = dconj(a)
LaurentPolynomial(5*z⁻⁴ + 4*z⁻³ + 3*z⁻² + 2*z⁻¹ + 1)

julia> c = LaurentPolynomial([1+1im, 2+2im, 3+3im, 4+4im, 5+5im],:z)
LaurentPolynomial((1 + 1im) + (2 + 2im)*z + (3 + 3im)*z² + (4 + 4im)*z³ + (5 + 5im)*z⁴)

julia> p = dconj(c)
LaurentPolynomial((5 - 5im)*z⁻⁴ + (4 - 4im)*z⁻³ + (3 - 3im)*z⁻² + (2 - 2im)*z⁻¹ + (1 - 1im))
```
"""
function dconj(a::LaurentPolynomial)
    ac = deepcopy(coeffs(a))
    dc = conj.(ac[end:-1:1])             # flipping and conjugating the coeffs
    fr = -a.n[]:-a.m[]                   # flipping and negating the range
    d = LaurentPolynomial(dc,fr,a.var)
end

"""
    shift(p::LaurentPolynomial[,k::Integer=1])

Increase the powers of the variable in the Laurent polynomial by 1 or a given (possibly negative) number.

For a univariate Laurent polynomial `p(z) = pₘ zᵐ + p₋₁ z⁻¹ +  p₀ + p₁ z + … + pₙ zⁿ` and a given integer `k`, return a Laurent polynomial  `p(z) = pₘ zᵐ⁺ᵏ + p₋₁ z⁻¹⁺ᵏ +  p₀ zᵏ + p₁ zᵏ⁺¹ + … + pₙ zⁿ⁺ᵏ`. If `k` is not specified, it is assumed that `k=1`.

# Examples

```julia
julia> p = LaurentPolynomial([1,2,3,4],-2:1,:z)
LaurentPolynomial(z⁻² + 2*z⁻¹ + 3 + 4*z)

julia> shift(p)
LaurentPolynomial(z⁻¹ + 2 + 3*z + 4*z²)

julia> shift(p,-3)
LaurentPolynomial(z⁻⁵ + 2*z⁻⁴ + 3*z⁻³ + 4*z⁻²)
```
"""
function shift(p::LaurentPolynomial,k::Integer=1)
    r = range(p)
    s = LaurentPolynomial(coeffs(p),r.+k,p.var)
    return s
end


function tophat(x,p)   #first function evaluates the vector input x at parameter p
    return exp.(-x.^p)
end
function dtophat(x,p)  #second function is the derivative
	return -p* (x.^(p-1)) .* exp.(-x.^p)
end
function tophat() #third function, with no input arguments, gives the default range of parameters p
	return [1.1 5;]
end

function gaussian(x)
	return exp.(-x.^2)
end
function gaussian()
	return []
end
function dgaussian(x)
	return -2*x .* exp.(-x.^2)
end

function cubic(x)
	return x.^3
end
function cubic()
	return []
end
function dcubic(x)
	return 3 .* x.^2
end

function quintic(x)
	return x.^5
end
function quintic()
	return []
end
function dquintic(x)
	return 5 .* x.^4
end

function wavelet(x)
	return (2*x.^2-1) .* exp.(-x.^2)
end
function wavelet()
	return []
end
function dwavelet(x)
	return (2*x.*(3-2*x.^2)) .* exp.(-x.^2)
end

function sigmoid(x)
	return tanh.( x )
end
function sigmoid()
	return []
end
function dsigmoid(x)
	return (1-tanh.( x ).^2)
end

function morlet(x,p)
	return cos.(2*pi*x) .* exp.(-(2*(pi/p)^2)*x.^2) .- exp.(-p^2/2-(2*(pi/p)^2)*x.^2)
end
function morlet()
	return [1 9;]
end
function dmorlet(x,p)
	return -2*(pi.*sin(2*pi*x) .+ 2*((pi/p)^2)*x.*cos.(2*pi*x)) .* exp(-(2*(pi/p)^2)*x.^2) .+ (4*((pi/p)^2)*x) .* exp.(-p^2/2-(2*(pi/p)^2)*x.^2)
end


using Plots

abstract type DescentMethod end

struct GradientDescent <: DescentMethod
    α
end

# init! function does nothing.
init!(M::GradientDescent, f, ∇f, x) = M;

function step!(M::GradientDescent, f, ∇f, x)
    α, g = M.α, ∇f(x);
    return x - α*g;
end

# test

A = rand(2, 2);
b = rand(2);
c = rand(2);
x = rand(2);

f = (x) -> 1/2 * x'*A*x + b'*x .+ c
∇f = (x) -> 1/2 * (A + A') * x + b

p = contour(-10:0.1:10, -10:0.1:10, (x, y)->sum(f([x, y])))
scatter!(p, [x[1]], [x[2]])

m = GradientDescent(0.5);
init!(m, f, ∇f, x);
x_res = [];
y_res = [];

for i in 1:100
    x = step!(m, f, ∇f, x)
    push!(x_res, [x[1]])
    push!(y_res, [x[2]])
end

x_res
y_res

scatter!(p, x_res, y_res)

x_res
y_res


function l1_norm(a)
    return reduce(+, map(x-> abs(x),a))
end

function l2_norm(a)
    return sqrt(reduce(+, map(x -> x^2,a)))
end

function lp_norm(a,power)
    return (reduce(+, map(x -> abs(x)^power,a)))^(1/power)
end


using LinearAlgebra

"""
A kernel function takes two vectors and calculates
the distance between them.
"""
abstract type Kernel end

"""
    SquaredExponentialKernel(l, σf, σy)

Squared exponential kernel for noisy observations.
The parameter names follow (15.20) in MLAPP.

# Parameters
- `l`: length scale, which controls the horizontal scale.
      It can be in either of the forms
   1. `Float64`: each dimension will have the same length scale
   2. `Vector{Float64}`: length scale for each dimension.
- `σf`: vertical scale
- `σy`: observation noise
"""
mutable struct SquaredExponentialKernel <: Kernel
    l::Float64
    σf::Float64
    σy::Float64
end

function (kernel::SquaredExponentialKernel)(xp, xq, δpq)
    σf = kernel.σf
    σy = kernel.σy
    M = lengthscalematrix(kernel.l, length(xp))
    return σf^2 * exp(-0.5 * (xp - xq)' * M * (xp - xq)) + σy^2*δpq
end

lengthscalematrix(l::Float64, dimensionsize) = Diagonal(ones(dimensionsize)) / l^2

"""
    MixtureKernel(X, p, q)

A kernel mixture of Gaussian kernel and dot product.
The definition follows (6.63) in PRML.
"""
mutable struct MixtureKernel <: Kernel
    sekernel::SquaredExponentialKernel
    θ0::Float64
    θ1::Float64

    function MixtureKernel(l, σf, σy, θ0, θ1)
        sekernel = SquaredExponentialKernel(l, σf, σy)
        new(sekernel, θ0, θ1)
    end

end

function (kernel::MixtureKernel)(xp, xq, δpq)
    sekernel = kernel.sekernel
    θ0 = kernel.θ0
    θ1 = kernel.θ1
    return sekernel(xp, xq, δpq) + θ0 + θ1 * (xp' * xq)
end

function (kernel::Kernel)(inputtuple::Tuple)
    kernel(inputtuple...)
end

"""
    covarianceblock(kernel, X, Xs)

Construct covariance matrix between `X` and `Xs` based on
kernel function.

# Parameters

- `kernel`: kernel function.
- `X`: input matrix ``X``.
- `Xs`: another input matrix ``X_*`` (X-star).
- `addnoise`: if `true`, when building a covariance matrix, we add
      observation noise to the diagonal term.
"""
function covarianceblock(kernel, X, Xs; addnoise=false)
    N = size(X, 1)
    Ns = size(Xs, 1)
    K = zeros(N, Ns)
    for p in 1:N, q in 1:Ns
        xp = X[p, :]
        xq = Xs[q, :]
        if addnoise
            K[p, q] = kernel(xp, xq, p==q)
        else
            K[p, q] = kernel(xp, xq, false)
        end
    end
    K
end

function covariancematrix(kernel, X; addnoise=true)
    covarianceblock(kernel, X, X, addnoise=addnoise)
end

"""
Because inside squared exponential kernel we need Euclidean distance,
we can use the shortcut for it following (14.30). This can significantly
reduce the computational cost.
"""
function covariancematrix(kernel::SquaredExponentialKernel, X)
    l, σf, σy = kernel.l, kernel.σf, kernel.σy
    N = size(X, 1)
    euclideandist = X.^2 .+ (X.^2)' .- 2*X*X'
    return σf^2 * exp.(-0.5/l^2 .* euclideandist) + Diagonal(fill(σy^2, N))
end


using RegularizedRegression: ape, medape, describe_goodness_of_fit

@testset "Loss functions" begin

@testset "APE (Absolute Percentage Error)" begin
    @test ape([1.5, 2.0], [1.0, 1.0]) == [0.50, 1.0]
    @test ape([1.5, 2.0, 3.0], [1.0, 1.0, 1.0]) == [0.50, 1.0, 2.0]
end

@testset "MAPE (Mean Absolute Percentage Error)" begin
    @test mape([1.5, 2.0], [1.0, 1.0]) == mean([50.0, 100.0])
    @test mape([1.5, 2.0, 3.0], [1.0, 1.0, 1.0]) == mean([50.0, 100.0, 200.0])
    @test isapprox(mape([1.5, 2.0, 3.0], [1.0, 1.5, 2.0]), mean([50.0, 100.0/3, 50.0]))
end

@testset "MEDAPE (Median Absolute Percentage Error)" begin
    @test medape([1.5, 2.0], [1.0, 1.0]) == median([50.0, 100.0])
    @test medape([1.5, 2.0, 3.0], [1.0, 1.0, 1.0]) == median([50.0, 100.0, 200.0])
end

@testset "MASE (Mean Absolute Scaled Error)" begin
    @test mase([1.5, 2.0, 3.0], [1.0, 1.5, 2.0]) == (0.5+0.5+1.0)/(3.0/2.0 * (0.5 + 0.5))
end

#@testset "R2" begin
#    @test r2([1.5, 2.0, 3.0], [1.0, 1.5, 2.0]) == (1.0 - ()
#end

@testset "describe_goodness_of_fit" begin
    str = describe_goodness_of_fit([1.5, 2.0, 3.0], [1.0, 1.5, 2.0])
    @test typeof(str) <: AbstractString
    @test ismatch(r"MAPE = 44.44%", str)
    @test ismatch(r"MASE = 1.33", str)
    @test ismatch(r"R2 = ", str)

    str_w_adj = describe_goodness_of_fit([1.5, 2.0, 3.0], [1.0, 1.5, 2.0], 1)
    @test ismatch(r"R2adj = ", str_w_adj)
end

end


@doc raw"""
    semivariance(x; kind::Symbol=:lower)

Returns the semivariance of vector-like `x`, which is the variance
computed using only the observations below the mean.

Defaults to lower semivariance, but returns upper semivariance if `kind=:upper` is supplied.
"""
function semivariance(x; kind::Symbol=:lower)
    bigN = length(x)
    mu = mean(x)
    if kind==:upper
        tmp = filter(z -> z > mu,x)
    else
        tmp = filter(z -> z <= mu,x)
    end
    return sum((tmp .- mu).^2)/bigN
end

"""
    lowerpartialmom(x,t,n)

`n`-th Lower Partial Moment for vector-like `x` using target `t`.

Source: Harlow (1991)
"""
function lowerpartialmom(x,t,n)
    bigN = length(x)
    return sum(filter(z->z>=0,t .- x).^n)/bigN
end

"""
    upperpartialmom(x,t,n)

`n`-th Upper Partial Moment for vector-like `x` using target `t`.

Source: Harlow (1991)
"""
function upperpartialmom(x,t,n)
    bigN = length(x)
    return sum(filter(z->z>=0,x .- t).^n)/bigN
end

"""
    valueatrisk(D::ContinuousUnivariateDistribution,alpha)

Returns the VaR (value at risk) at the `alpha`-level for the distribution `D`.
"""
valueatrisk(D::Distributions.ContinuousUnivariateDistribution,alpha::Float64) = quantile(D,1-alpha)

"""
    valueatrisk(X::AbstractVector,alpha)

Returns the nonparametric estimate of VaR (value at risk) at the `alpha`-level for `X`.
"""
function valueatrisk(X::AbstractVector,alpha::Float64)
    ecdf = EmpiricalCDF()
    append!(ecdf,X)
    sort!(ecdf)
    icdf = finv(ecdf)
    return icdf(1-alpha)
end

"""
    expectedshortfall(D::ContinuousUnivariateDistribution,alpha)

Returns the Expected Shortfall at the `alpha`-level for the distribution `D`.
Assumes `D` is the distribution of payoffs `X`, not losses (which would be `-X`).
"""
function expectedshortfall(D::ContinuousUnivariateDistribution,alpha::Float64)
    return first(quadgk( x -> -(1/alpha)*valueatrisk(D,x),0.0,alpha))
end

"""
    expectedshortfall(X::AbstractVector,alpha)

Returns the nonparametric estimate of Expected Shortfall at the `alpha`-level for `X`.
Assumes `X` represents payoffs, not losses (which would be `-X`).
"""
function expectedshortfall(X::AbstractVector,alpha::Float64)
    ecdf = EmpiricalCDF()
    append!(ecdf,X)
    sort!(ecdf)
    icdf = finv(ecdf)
    return first(quadgk( x -> -(1/alpha)*icdf(1-x),0.0,alpha))
end

# Bertsimas, Lauprete, Samarov (2004 JEDC) good source for ES/VaR/LPM


using Plots, DataFrames, Flux, Plots, Random, ReverseDiff
using Turing # v0.20

# create half-moon data:
function make_moons(n_samples::Int = 100)

    n = n_samples

    n_samples_out = trunc(Int, round(n_samples / 2))
    n_samples_in = n_samples - n_samples_out

    outer_circ_x = cos.(LinRange(0, π, n_samples_out))
    outer_circ_y = sin.(LinRange(0, π, n_samples_out))
    inner_circ_x = 1 .- cos.(LinRange(0, π, n_samples_in))
    inner_circ_y = 1 .- sin.(LinRange(0, π, n_samples_in)) .- 0.5

    X = hcat(vcat(outer_circ_x, inner_circ_x), vcat(outer_circ_y, inner_circ_y))
    Y = hcat(zeros(Int, 1, 2 .* n_samples_out), ones(Int, 1, 2 .* n_samples_in))

    return X, Y, n

end

X, Y, n = make_moons()

# rearrange half-moon data:
n_2 = trunc(Int, n/2)
xscatter_0 = X[1:n_2]
xscatter_1 = X[1+n_2:n]
yscatter_0 = X[1+n:n+n_2]
yscatter_1 = X[1+n+n_2:end]

# plot half-moon data:
plot(
    xscatter_0, yscatter_0,
    seriestype = :scatter,
    legend = true,
    label = "Class 0",
    title = "half-moons problem"
)
plot!(
    xscatter_1, yscatter_1,
    seriestype = :scatter,
    legend = true,
    label = "Class 1"
)

# optional: half-moon data in a dataframe:
half_moon_df = DataFrame(
    X = vcat(xscatter_0, xscatter_1),
    Y = vcat(yscatter_0, yscatter_1),
    Class = vcat(zeros(Int, n_2), ones(Int, n_2))
)

# Turing settings:
Turing.setprogress!(true)
Turing.setadbackend(:reversediff)

# architecture:
network_shape = [
    (5,2, :tanh),
    (5,5, :tanh),
    (2,5, :σ)]

# Regularization, parameter variance, and total number of parameters.
alpha = 0.09
sig = sqrt(1.0 / alpha)
num_params = sum([i * o + i for (i, o, _) in network_shape])

# This modification of the unpack function generates a series of vectors given a network shape.
function unpack(θ::AbstractVector, network_shape::AbstractVector)
    index = 1
    weights = []
    biases = []
    for layer in network_shape
        rows, cols, _ = layer
        size = rows * cols
        last_index_w = size + index - 1
        last_index_b = last_index_w + rows
        push!(weights, reshape(θ[index:last_index_w], rows, cols))
        push!(biases, reshape(θ[last_index_w+1:last_index_b], rows))
        index = last_index_b + 1
    end
    return weights, biases
end

# Generate an abstract neural network given a shape, and return a prediction.
function nn_forward(x, θ::AbstractVector, network_shape::AbstractVector)
    weights, biases = unpack(θ, network_shape)
    layers = []
    for i in eachindex(network_shape)
        push!(layers, Dense(weights[i],
            biases[i],
            eval(network_shape[i][3])))
    end
    nn = Chain(layers...)
    return nn(x)
end

# rearrange half-moon data yet again:
xscatter = vcat(xscatter_0, xscatter_1)
yscatter = vcat(yscatter_0, yscatter_1)
xs = Array([[xscatter[i]; yscatter[i]] for i = 1:n])
ts = vcat(zeros(Int, n_2), ones(Int, n_2))

# General Turing specification for a BNN model.
@model bayes_nn_general(xs, ts, network_shape, num_params) = begin
    θ ~ MvNormal(zeros(num_params), sig .* ones(num_params))
    preds = nn_forward(xs, θ, network_shape)
    for i = 1:length(ts)
        ts[i] ~ Bernoulli(preds[i])
    end
end

# Perform inference.
num_samples = 2000
ch2 = sample(bayes_nn_general(hcat(xs...), ts, network_shape, num_params), NUTS(0.65), num_samples);

# This function makes predictions based on network shape.
function nn_predict(x, theta, num, network_shape)
    mean([nn_forward(x, theta[i,:], network_shape)[1] for i in 1:10:num])
end;

# Extract the θ parameters from the sampled chain.
params2 = MCMCChains.group(ch2, :θ).value

# classification plot
plot(
    xscatter_0, yscatter_0,
    seriestype = :scatter,
    label = "Class 0",
    title = "half-moons problem"
)
plot!(
    xscatter_1, yscatter_1,
    seriestype = :scatter,
    legend = :bottomright,
    label = "Class 1"
)
x_range = collect(range(-2.5,stop=3.5,length=200))
y_range = collect(range(-1.2,stop=1.7,length=200))
Z = [nn_predict([x, y], params2, length(ch2), network_shape)[1] for x=x_range, y=y_range]
contour!(x_range, y_range, Z)


"""
    conformal_cubed_sphere_mapping(x, y)

Conformal mapping of a cube onto a sphere. Maps `(x, y)` on the north-pole face of a cube
to (X, Y, Z) coordinates in physical space. The face is oriented normal to Z-axis with
X and Y increasing with x and y.

The input coordinates must lie within the range -1 <= x <= 1,  -1 <= y <= 1.

This numerical conformal mapping is described by Rančić et al. (1996).

This is a Julia translation of MATLAB code from MITgcm [1] that is based on
Fortran 77 code from Jim Purser & Misha Rančić.

[1] http://wwwcvs.mitgcm.org/viewvc/MITgcm/MITgcm_contrib/high_res_cube/matlab-grid-generator/map_xy2xyz.m?view=markup

Rančić et al., (1996): Quarterly Journal of the Royal Meteorological Society, A global shallow-water model
    using an expanded spherical cube - Gnomonic versus conformal coordinates
"""
function conformal_cubed_sphere_mapping(x, y)
    X = xᶜ = abs(x)
    Y = yᶜ = abs(y)

    kxy = yᶜ > xᶜ

    xᶜ = 1 - xᶜ
    yᶜ = 1 - yᶜ

    kxy && (xᶜ = 1 - Y)
    kxy && (yᶜ = 1 - X)

    Z = ((xᶜ + im * yᶜ) / 2)^4
    W = W_Rancic(Z)

    im³ = im^(1/3)
    ra = √3 - 1
    cb = -1 + im
    cc = ra * cb / 2

    W = im³ * (W * im)^(1/3)
    W = (W - ra) / (cb + cc * W)
    X, Y = reim(W)

    H = 2 / (1 + X^2 + Y^2)
    X = X * H
    Y = Y * H
    Z = H - 1

    if kxy
        X, Y = Y, X
    end

    y < 0 && (Y = -Y)
    x < 0 && (X = -X)

    # Fix truncation for x = 0 or y = 0.
    x == 0 && (X = 0)
    y == 0 && (Y = 0)

    return X, Y, Z
end

"""
   conformal_cubed_sphere_inverse_mapping(X, Y, Z)

Inverse mapping for conformal cube sphere for quadrant of north-pole face in which X and Y are both
positive. All other mappings to other cube face coordinates can be recovered from rotations of this map.
There a 3 other quadrants for the north-pole face and five other faces for a total of twenty-four quadrants.
Because of symmetry only the reverse for a single quadrant is needed. Because of branch cuts and the complex
transform the inverse mappings are multi-valued in general, using a single quadrant case allows a simple
set of rules to be applied.

The mapping is valid for the cube face quadrant defined by 0 < x < 1 and 0 < y < 1, where a full cube
face has extent -1 < x < 1 and -1 < y < 1. The quadrant for the mapping is from a cube face that has
"north-pole" at its center (x=0,y=0) i.e has X,Y,Z = (0,0,1) at its center. The valid ranges of X and Y
for this mapping and convention are a quadrant defined be geodesics that connect the points A, B, C and D,
on the shell of a sphere of radius R with X,Y coordinates as follows

A = (0,0)
B = (√2,0)
C = (√3/3,√3/3)
D = (0,√2)

"""
function conformal_cubed_sphere_inverse_mapping(X, Y, Z)
    H  = Z + 1
    Xˢ = X/H
    Yˢ = Y/H
    ω  = Xˢ + im*Yˢ

    ra = √3 - 1
    cb = -1 + im
    cc = ra * cb / 2
    ω⁰ = (ω*cb + ra)/(1-ω*cc)
    W⁰ = im*ω⁰^3*im
    Z  = Z_Rancic(W⁰)
    z  = (Z^0.25)*2
    x, y = reim(z)

    kxy = abs(y) > abs(x)
    xx = x
    yy = y
    !kxy && ( x = 1 - abs(yy) )
    !kxy && ( y = 1 - abs(xx) )

    xf = x
    yf = y
    ( X < Y ) && ( xf = y  )
    ( X < Y ) && ( yf = x  )
    x = xf
    y = yf

    return x, y
end

W_Rancic(Z) = sum(A_Rancic[k] * Z^(k-1) for k in 1:length(A_Rancic))
Z_Rancic(W) = sum(B_Rancic[k] * W^(k-1) for k in 1:length(B_Rancic))


using Base.Test
import Roots.newton, Roots.halley

@test_approx_eq_eps newton(sin, cos, 0.5) 0.0 100*eps(1.0)
@test_approx_eq newton(cos, x -> -sin(x), 1.0) pi/2
@test_approx_eq newton(x -> x^2 - 2x - 1, x -> 2x - 2, 3.0) 2.414213562373095
@test_approx_eq_eps newton(x -> exp(x) - cos(x),
                           x -> exp(x) + sin(x), 3.0) 0.0 eps(1.0)
@test_approx_eq halley(x -> x^2 - 2x - 1,
                       x -> 2x - 2,
                       x -> 2, 3.0) 2.414213562373095
@test_approx_eq_eps halley(x -> exp(x) - cos(x),
                           x -> exp(x) + sin(x),
                           x -> exp(x) + cos(x), 3.0) 0.0 eps(1.0)

## tests with auto derivatives
@test_approx_eq newton(sin, 3) float(pi)
@test_approx_eq halley(sin, 3) float(pi)

## More tests with autoderivaitves. Derivative of D operation:
isdefined(ForwardDiff, :derivative) && @test_approx_eq newton(D(sin), 1.5) pi/2

## test with Complex input

f = x ->  x^3 - 1; fp = x ->  3x^2
@test_approx_eq real(newton(f, fp, 1+im)) 1
@test_approx_eq real(newton(f, fp, 1+10im)) (-1/2)


n = 9 # rand(1:10)

A = matrixdepot("companion", Float32, n)
@test diag(A, -1) == ones(Float32, n-1)

v = [3., 4, 2, 10]
B = matrixdepot("companion", v)
@test B[:,end] == v

println("'companion' passed test...")


module MultiFieldFEAutodiffTests

using Test

using Gridap.Algebra
using Gridap.Geometry
using Gridap.CellData
using Gridap.ReferenceFEs
using Gridap.FESpaces
using Gridap.MultiField

domain = (0,1,0,1)
partition = (2,2)
model = CartesianDiscreteModel(domain,partition)

Ω = Triangulation(model)
dΩ = Measure(Ω,2)

V1 = FESpace(model,ReferenceFE(lagrangian,Float64,1))
V2 = FESpace(model,ReferenceFE(lagrangian,Float64,1))
Y = MultiFieldFESpace([V1,V2])

r((uh,ph),(v,q)) = ∫( v*uh + uh*q + ph*q )dΩ
j(xh,dx,dy) = jacobian(xh->r(xh,dy),xh)

e((uh,ph)) = ∫( uh*uh + uh*ph + ph*ph )dΩ
g(xh,dy) = gradient(xh->e(xh),xh)
#h(xh,dx,dy) = hessian(xh->e(xh),xh)

dx = get_trial_fe_basis(Y)
dy = get_fe_basis(Y)
xh = FEFunction(Y,rand(num_free_dofs(Y)))

#display(r(xh,dy)[Ω][end])
#display(j(xh,dx,dy)[Ω][end])
#display(e(xh)[Ω][end])
#display(g(xh,dy)[Ω][end])
##display(h(xh,dx,dy)[Ω][end])

@test j(xh,dx,dy)[Ω][end][1,1] != nothing
@test j(xh,dx,dy)[Ω][end][2,1] != nothing
@test j(xh,dx,dy)[Ω][end][1,2] != nothing
@test j(xh,dx,dy)[Ω][end][2,2] != nothing
@test g(xh,dy)[Ω][end][1] != nothing
@test g(xh,dy)[Ω][end][2] != nothing

V1 = FESpace(model,ReferenceFE(lagrangian,Float64,2))
V2 = FESpace(model,ReferenceFE(lagrangian,Float64,1))
Y = MultiFieldFESpace([V1,V2])

dx = get_trial_fe_basis(Y)
dy = get_fe_basis(Y)
xh = FEFunction(Y,rand(num_free_dofs(Y)))

@test j(xh,dx,dy)[Ω][end][1,1] != nothing
@test j(xh,dx,dy)[Ω][end][2,1] != nothing
@test j(xh,dx,dy)[Ω][end][1,2] != nothing
@test j(xh,dx,dy)[Ω][end][2,2] != nothing
@test g(xh,dy)[Ω][end][1] != nothing
@test g(xh,dy)[Ω][end][2] != nothing

eu(uh) = ∫( uh*uh )dΩ
ep(ph) = ∫( ph*ph )dΩ
eup((uh,ph)) = ∫( uh*uh + ph*ph )dΩ
geu(xh,dy) = gradient(xh->eu(xh),xh)
gep(xh,dy) = gradient(xh->ep(xh),xh)
geup(xh,dy) = gradient(xh->eup(xh),xh)

uh,ph=xh
geu_uh=geu(uh,dy)
gep_ph=gep(ph,dy)
geup_uh_ph=geup(xh,dy)

a=geu_uh[Ω]
b=gep_ph[Ω]
c=geup_uh_ph[Ω]

@test all(a .== map(x->x.array[1],c))
@test all(b .== map(x->x.array[2],c))

ru(uh,v) = ∫( v*uh*uh )dΩ
rp(ph,q) = ∫( q*ph*ph )dΩ
rup((uh,ph),(v,q)) = ∫( v*uh*uh + q*ph*ph )dΩ
ju(xh,dx,dy) = jacobian(xh->ru(xh,dy),xh)
jp(xh,dx,dy) = jacobian(xh->rp(xh,dy),xh)
jup(xh,dx,dy) = jacobian(xh->rup(xh,dy),xh)

uh=FEFunction(V1,get_free_dof_values(xh[1]))
ph=FEFunction(V2,get_free_dof_values(xh[2]))
du = get_trial_fe_basis(V1)
dv = get_fe_basis(V1)
dp = get_trial_fe_basis(V2)
dq = get_fe_basis(V2)

ju_uh=ju(uh,du,dv)
jp_ph=jp(ph,dp,dq)
jup_uh_ph=jup(xh,dx,dy)

a=ju_uh[Ω]
b=jp_ph[Ω]
c=jup_uh_ph[Ω]

@test all(a .== map(x->x.array[1,1],c))
@test all(b .== map(x->x.array[2,2],c))

end # module


"""
    Semicircle(r)

The Wigner semicircle distribution with radius parameter `r` has probability
density function

```math
f(x; r) = \\frac{2}{\\pi r^2} \\sqrt{r^2 - x^2}, \\quad x \\in [-r, r].
```

```julia
Semicircle(r)   # Wigner semicircle distribution with radius r

params(d)       # Get the radius parameter, i.e. (r,)
```

External links

* [Wigner semicircle distribution on Wikipedia](https://en.wikipedia.org/wiki/Wigner_semicircle_distribution)
"""
struct Semicircle{T<:Real} <: ContinuousUnivariateDistribution
    r::T
    Semicircle{T}(r::T) where {T <: Real} = new{T}(r)
end

function Semicircle(r::T) where {T <: Real}
    @check_args(Semicircle, r > 0)
    return Semicircle{T}(r)
end

Semicircle(r::T, ::NoArgCheck) where {T <: Real} = Semicircle{T}(r)
Semicircle(r::Integer) = Semicircle(float(r))

@distr_support Semicircle -d.r +d.r

params(d::Semicircle) = (d.r,)

mean(d::Semicircle) = zero(d.r)
var(d::Semicircle) = d.r^2 / 4
skewness(d::Semicircle) = zero(d.r)
median(d::Semicircle) = zero(d.r)
mode(d::Semicircle) = zero(d.r)
entropy(d::Semicircle) = log(π * d.r) - oftype(d.r, 0.5)

function pdf(d::Semicircle, x::Real)
    xx, r = promote(x, float(d.r))
    if insupport(d, xx)
        return 2 / (π * r^2) * sqrt(r^2 - xx^2)
    else
        return oftype(r, 0)
    end
end

function logpdf(d::Semicircle, x::Real)
    xx, r = promote(x, float(d.r))
    if insupport(d, xx)
        return log(oftype(r, 2) / π) - 2 * log(r) + log(r^2 - xx^2) / 2
    else
        return oftype(r, -Inf)
    end
end

function cdf(d::Semicircle, x::Real)
    xx, r = promote(x, float(d.r))
    if insupport(d, xx)
        u = xx / r
        return (u * sqrt(1 - u^2) + asin(u)) / π + one(xx) / 2
    elseif x < minimum(d)
        return zero(r)
    else
        return one(r)
    end
end

@quantile_newton Semicircle


using ModelingToolkit
using LinearAlgebra
using Test

A = [0 1 1 2 2 1 1 2 1 2
     0 1 -1 -3 -2 2 1 -5 0 -5
     0 1 2 2 1 1 2 1 1 2
     0 1 1 1 2 1 1 2 2 1
     0 2 1 2 2 2 2 1 1 1
     0 1 1 1 2 2 1 1 2 1
     0 2 1 2 2 1 2 1 1 2
     0 1 7 17 14 2 1 19 4 23
     0 1 -1 -3 -2 1 1 -4 0 -5
     0 1 1 2 2 1 1 2 2 2]
N = ModelingToolkit.nullspace(A)
@test size(N, 2) == 3
@test rank(N) == 3
@test iszero(A * N)

A = [0 1 2 0 1 0;
     0 0 0 0 0 1;
     0 0 0 0 0 1;
     1 0 1 2 0 1;
     0 0 0 2 1 0]
col_order = Int[]
N = ModelingToolkit.nullspace(A; col_order)
colspan = A[:, col_order[1:4]]   # rank is 4
@test iszero(ModelingToolkit.nullspace(colspan))
@test !iszero(ModelingToolkit.nullspace(A[:, col_order[1:5]]))
@test !iszero(ModelingToolkit.nullspace(A[:, [col_order[1:4]..., col_order[6]]]))


# ------------------------------------------------------------------
# Licensed under the MIT License. See LICENSE in the project root.
# ------------------------------------------------------------------

"""
    Scale(; low=0.25, high=0.75)

Applies the scale transform to all columns of the table.
The scale transform of the column `x` is defined by `(x .- xl) ./ (xh - xl)`,
where `xl = quantile(x, low)` and `xh = quantile(x, high)`.

# Examples

```julia
Scale()
Scale(low=0, high=1)
Scale(low=0.3, high=0.7)
```

## Notes

* The `low` and `high` values are restricted to the interval [0, 1].
"""
struct Scale{T<:Real} <: Colwise
  low::T
  high::T

  function Scale(low::T, high::T) where {T<:Real}
    @assert 0 ≤ low ≤ high ≤ 1 "invalid quantiles"
    new{T}(low, high)
  end
end

Scale(low::Real, high::Real) = Scale(promote(low, high)...)

Scale(; low=0.25, high=0.75) = Scale(low, high)

assertions(::Type{<:Scale}) = [assert_continuous]

isrevertible(::Type{<:Scale}) = true

function colcache(transform::Scale, x)
  levels = (transform.low, transform.high)
  xl, xh = quantile(x, levels)
  xl == xh && ((xl, xh) = (zero(xl), one(xh)))
  (xl=xl, xh=xh)
end

colapply(::Scale, x, c)  = @. (x - c.xl) / (c.xh - c.xl)

colrevert(::Scale, y, c) = @. (c.xh - c.xl) * y + c.xl

"""
    MinMax()

The transform that is equivalent to `Scale(low=0, high=1)`.

See also [`Scale`](@ref).
"""
MinMax() = Scale(low=0, high=1)

"""
    Interquartile()

The transform that is equivalent to `Scale(low=0.25, high=0.75)`.

See also [`Scale`](@ref).
"""
Interquartile() = Scale(low=0.25, high=0.75)


#---------------------------------------------------------------------#
#This function computes the Initial & Analytic Solutions.
#Written by F.X. Giraldo on 10/2003
#           Department of Applied Mathematics
#           Naval Postgraduate School
#           Monterey; CA 93943-5216
#---------------------------------------------------------------------#
function exact_solution(coord,Npoin,c,DFloat)

    #Initialize
    qe=zeros(DFloat,Npoin,1)
    fe=zeros(DFloat,Npoin,1)
    cc=c*π

    #Generate Grid Points
    for I=1:Npoin
        x=coord[1,I]
        y=coord[2,I]
        qe[I]=sin(cc*x)*sin(cc*y)
        fe[I]=-2*cc^2*sin(cc*x)*sin(cc*y)
    end #I

    return (qe,fe)

end


# Equation 33 of https://www.mdpi.com/2227-7390/6/2/16/htm

# Loaded Packages
using FdeSolver
using Plots
using SpecialFunctions

# Parameters
tSpan = [0, 5]         # Time Span
y0 = [1, 0.5, 0.3]     # Initial values
β = [0.5, 0.2, 0.6]    # Order of derivation

# Definition of the System
function F(t, n, β, y)

    F1 = 1 / sqrt(pi) * (((y[n, 2] - 0.5) * (y[n, 3] - 0.3))^(1 / 6) + t[n]^(1 / 2))
    F2 = gamma(2.2) * (y[n, 1] - 1)
    F3 = gamma(2.8) / gamma(2.2) * (y[n, 2] - 0.5)

    return [F1, F2, F3]

end

# Numerical Solution
t, Yapp = FDEsolver(F, tSpan, y0, β, nothing, nc = 5)

# Plot
plot(t, Yapp, linewidth = 5, title = "Solution of system 33",
     xaxis = "Time (t)", yaxis = "y(t)", label = "Approximation")

plot!(t, t -> (t .+ 1), lw = 3, ls = :dash, color= "red", label = "Exact solution")
plot!(t, t -> (t.^1.2 .+ 0.5), lw = 3, ls = :dash, color= "cyan", label = "Exact solution")
plot!(t, t -> (t.^1.8 .+ 0.3), lw = 3, ls = :dash, color= "black" ,label = "Exact solution")


keys = open("public_keys.txt") do f
    parse.(Int, readlines(f))
end

function transform(subject_number::Int64, loop_size::Int64)::Int64
    val = 1
    for i in 1:loop_size
        val = (val * subject_number) % 20201227
    end

    return val
end

function crack(public_keys::Array{Int64, 1}, sn::Int64, ls_range::UnitRange{Int64})::Tuple{Int64, Int64}
    val = 1
    for ls in ls_range
        val = (val * sn) % 20201227
        if val == public_keys[1]
            # println("sn $(sn), ls $(ls) match first")
            val2 = 1
            for ls2 in ls_range
                val2 = (val2 * sn) % 20201227
                if val2 == public_keys[2] && ls != ls2
                    # println("sn $(sn), ls $(ls2) match second")

                    return ls, ls2
                end
            end
            # println("no matching ls2 found")
        end
    end

    return nothing, nothing
end

ls1, ls2 = crack(keys, 7, 1:100_000_000)

println(transform(keys[1], ls2))
# println(transform(keys[2], ls1))


#############################################################################
# eigmin_max.jl
# Handles maximum and minimum eigenvalue of a symmetric positive definite matrix
# (and imposes the constraint that its argument be PSD)
# All expressions and atoms are subtypes of AbstractExpr.
# Please read expressions.jl first.
#############################################################################

import LinearAlgebra: eigmin, eigmax

### Eig max

struct EigMaxAtom <: AbstractExpr
    head::Symbol
    id_hash::UInt64
    children::Tuple{AbstractExpr}
    size::Tuple{Int, Int}

    function EigMaxAtom(x::AbstractExpr)
        children = (x,)
        m, n = size(x)
        if m == n
            return new(:eigmax, hash(children), children, (1, 1))
        else
            error("eigmax can only be applied to a square matrix.")
        end
    end
end

function sign(x::EigMaxAtom)
    return NoSign()
end

function monotonicity(x::EigMaxAtom)
    return (Nondecreasing(),)
end

function curvature(x::EigMaxAtom)
    return ConvexVexity()
end

function evaluate(x::EigMaxAtom)
    eigmax(evaluate(x.children[1]))
end

eigmax(x::AbstractExpr) = EigMaxAtom(x)

# Create the equivalent conic problem:
#   minimize t
#   subject to
#            tI - A is positive semidefinite
#            A      is positive semidefinite
function conic_form!(x::EigMaxAtom, unique_conic_forms)
    if !has_conic_form(unique_conic_forms, x)
        A = x.children[1]
        m, n = size(A)
        t = Variable()
        p = minimize(t, t*Matrix(1.0I, n, n) - A ⪰ 0)
        cache_conic_form!(unique_conic_forms, x, p)
    end
    return get_conic_form(unique_conic_forms, x)
end

### Eig min

struct EigMinAtom <: AbstractExpr
    head::Symbol
    id_hash::UInt64
    children::Tuple{AbstractExpr}
    size::Tuple{Int, Int}

    function EigMinAtom(x::AbstractExpr)
        children = (x,)
        m, n = size(x)
        if m == n
            return new(:eigmin, hash(children), children, (1,1))
        else
            error("eigmin can only be applied to a square matrix.")
        end
    end
end

function sign(x::EigMinAtom)
    return NoSign()
end

function monotonicity(x::EigMinAtom)
    return (Nondecreasing(),)
end

function curvature(x::EigMinAtom)
    return ConcaveVexity()
end

function evaluate(x::EigMinAtom)
    eigmin(evaluate(x.children[1]))
end

eigmin(x::AbstractExpr) = EigMinAtom(x)

# Create the equivalent conic problem:
#   maximize t
#   subject to
#            A - tI is positive semidefinite
#            A      is positive semidefinite
function conic_form!(x::EigMinAtom, unique_conic_forms)
    if !has_conic_form(unique_conic_forms, x)
        A = x.children[1]
        m, n = size(A)
        t = Variable()
        p = maximize(t, A - t*Matrix(1.0I, n, n) ⪰ 0)
        cache_conic_form!(unique_conic_forms, x, p)
    end
    return get_conic_form(unique_conic_forms, x)
end


@testset "Knapsack" begin
    mass    = [1, 5, 3, 7, 2, 10, 5]
    utility = [1, 3, 5, 2, 5,  8, 3]

    function fitness(n::AbstractVector)
        total_mass = sum(mass .* n)
        return (total_mass <= 20) ? sum(utility .* n) : 0
    end

    initpop = collect(rand(Bool,length(mass)))

    best, invbestfit, generations, tolerance, history = ga(
        x -> 1 / fitness(x),                    # Function to MINIMISE
        length(initpop),                        # Length of chromosome
        initPopulation = initpop,
        selection = roulette,                   # Options: sus
        mutation = inversion,                   # Options:
        crossover = singlepoint,                # Options:
        mutationRate = 0.2,
        crossoverRate = 0.5,
        ɛ = 0.1,                                # Elitism
        iterations = 20,
        populationSize = 50,
        interim = true);

    @test fitness(best) == 21.
    @test 1. /invbestfit == 21.
    @test sum(mass .* best) <= 20
end


# NOTE: this is mostly just copied from wikipedia
function gmres_cycle(A::SparseMatrixCSC, x0::Vector{Float64}, b::Vector{Float64}, m::Int64; err_thresh = 1e-20)

    # compute initial residual, norm(b) and initial error
    r = b - A * x0
    bnorm = norm(b)
    err = norm(r) / bnorm

    n = length(x0)
    Q = zeros(Float64, (n,m+1))
    H = zeros(Float64, (m+2,m+1))
    sn = zeros(Float64, m)
    cn = zeros(Float64, m)
    e1 = zeros(Float64, m+1)
    e1[1] = 1.0

    beta = norm(r) * e1
    Q[:,1] = r / norm(r)

    # begin iteration
    nk = 0
    for k=1:m
        nk += 1

        # do arnoldi iteration
        q = A * Q[:,k]
        for i=1:k
            H[i,k] = transpose(q) * Q[:,i]
            q = q - H[i,k] * Q[:,i]
        end
        qnorm = norm(q)
        H[k+1,k] = qnorm
        Q[:,k+1] = q / qnorm

        # apply givens rotation
        for i=1:k-1
            temp = cn[i] * H[i,k] + sn[i] * H[i+1,k]
            H[i+1,k] = -sn[i] * H[i,k] + cn[i] * H[i+1,k]
            H[i,k] = temp
        end

        # update sn and cn
        t = sqrt(H[k,k]^2 + H[k+1,k]^2)
        cn[k] = H[k,k] / t
        sn[k] = H[k+1,k] / t

        # eliminate H[k+1, k]
        H[k,k] = cn[k] * H[k,k] + sn[k] * H[k+1,k]
        H[k+1,k] = 0.0

        # update residual
        beta[k+1] = -sn[k]*beta[k]
        beta[k] = cn[k]*beta[k]
        err = abs(beta[k+1]) / bnorm

        if err < err_thresh
            break
        end
    end

    # compute correction to x
    y = H[1:nk,1:nk] \ beta[1:nk]
    x = x0 + Q[:,1:nk] * y
    return x, err
end

function gmres_solve(A::SparseMatrixCSC, x0::Vector{Float64}, b::Vector{Float64}, m::Int64; maxiters=10000, err_thresh = 1e-8)

    x = x0[:]
    err = 1.0
    iteration = 0

    while (err > err_thresh)
        x, err = gmres_cycle(A, x, b, m, err_thresh=err_thresh)
        iteration += 1

        if iteration > maxiters
            @warn "Iteration limit reached, defaulting to direct solver."
            return A \ b
        end
    end

    return x
end


using JuMP
using ECOS
using LinearAlgebra
using Random

Random.seed!(1234);

u0 = rand(10)
p = rand(10)
q = rand();

model = Model(optimizer_with_attributes(ECOS.Optimizer, "printlevel" => 0))
@variable(model, u[1:10])
@variable(model, t)
@objective(model, Min, t)
@constraint(model, [t, (u - u0)...] in SecondOrderCone())
@constraint(model, u' * p == q)
optimize!(model)

@show objective_value(model);
@show value.(u);

e1 = [1, zeros(10)...]
dual_model = Model(optimizer_with_attributes(ECOS.Optimizer, "printlevel" => 0))
@variable(dual_model, y1 <= 0)
@variable(dual_model, y2[1:11])
@objective(dual_model, Max, q * y1 + dot(vcat(0, u0), y2))
@constraint(dual_model, e1 - [0, p...] .* y1 - y2 .== 0)
@constraint(dual_model, y2 in SecondOrderCone())
optimize!(dual_model)

@show objective_value(dual_model);

model = Model(optimizer_with_attributes(ECOS.Optimizer, "printlevel" => 0))
@variable(model, u[1:10])
@variable(model, t)
@objective(model, Min, t)
@constraint(model, [t, 0.5, (u - u0)...] in RotatedSecondOrderCone())
@constraint(model, u' * p == q)
optimize!(model)

@show value.(u);

n = 15;
m = 10;
A = randn(m, n);
b = rand(m, 1);

model = Model(optimizer_with_attributes(ECOS.Optimizer, "printlevel" => 0))
@variable(model, t[1:n])
@variable(model, x[1:n])
@objective(model, Max, sum(t))
@constraint(model, sum(x) == 1)
@constraint(model, A * x .<= b )
# Cannot use the exponential cone directly in JuMP, hence we use MOI to specify the set.
@constraint(model, con[i = 1:n], [1, x[i], t[i]] in MOI.ExponentialCone())

optimize!(model);

@show objective_value(model);


"""
Remove minimum number of nodes so that the graph no longer exists k-core
"""
function decore(g, k=2)
	attack_nodes = Int[]
	deg = degree(g) # adaptive degrees
	degmax = maximum(deg)
	H = [Set{Int}() for i=1:degmax]

	for i in vertices(g)
		if deg[i] > 0
			degi = max(deg[i], k-1)
			push!(H[degi], i)
		end
	end

	d = isempty(H[k-1]) ? degmax : k-1
	cnt = 0
	done = false
	while cnt < nv(g) && done == false
		cnt += 1
		i = rand(H[d])
		delete!(H[d], i)
		deg[i] = 0
		d >= k && push!(attack_nodes, i)

		# update neighbors
		for j in neighbors(g,i)
			if deg[j] >= k
				delete!(H[deg[j]], j)
				deg[j] -= 1
				push!(H[deg[j]],j)
			end
		end

		while isempty(H[degmax])
			degmax -= 1
			done = degmax < k
		end
		d = isempty(H[k-1]) ? degmax : k-1
	end
	attack_nodes
end


using Sundials, Plots
using Test

pyplot(size=(700,700))

ProjDir = dirname(@__FILE__)
cd(ProjDir) do

  function sysfn(y, fy)
      fy[1] = y[1]^2 + y[2]^2 - 1.0
      fy[2] = y[2] - y[1]^2
  end

  res = Sundials.kinsol(sysfn, ones(2))
  res |> display
  println()

  @test round(res, 4) == round([0.786153, 0.618035], 4)

end


function voigt(x, σ, γ)
    faddeeva(z) = erfcx(-im * z)
    z = (x + im * γ) / σ / sqrt(2)
    return real(faddeeva(z)) / (σ * sqrt(2pi))
end

# J.J.Olivero and R.L. Longbothum in Empirical fits to the Voigt line width: A brief review, JQSRT 17, P233, 1977
# http://snst-hu.lzu.edu.cn/zhangyi/ndata/Voigt_profile.html
function voigt_fwhm(σ, γ)
    @assert σ > 0
    @assert γ > 0
    fg = σ * 2.355
    fl = γ * 2
    return fl * 0.5346 + sqrt(0.2166 * fl^2 + fg^2)
    #=
    Introducing dampening parameter:
    a = fl / fg
    fwhm = fl * (0.5346 + sqrt(0.2166 + (1/a)^2))
    =#
end

function voigt_σγ(fwhm, log_a)
    a = 10. ^log_a
    fl = fwhm / (0.5346 + sqrt(0.2166 + (1/a)^2))
    fg = fl / a
    return (fg / 2.355, fl / 2)
end
# log_a = log10((γ * 2) / (σ * 2.355));  QSFit.voigt_σγ(QSFit.voigt_fwhm(σ, γ), log_a), (σ, γ)

function voigt_γ(fwhm, σ)
    A = 0.5346
    B = 0.2166
    C = (σ * 2.355)^2
    # fwhm = A * fl + sqrt(B * fl^2 + C)
    fl = A * fwhm / (A^2 - B) - sqrt((A^2 * C - B * C + B * fwhm^2) / ((A^2 - B)^2))
    γ = fl / 2
    @assert (fwhm - voigt_fwhm(σ, γ)) / fwhm < 1e-5
    return γ
end
# QSFit.voigt_γ(QSFit.voigt_fwhm(σ, γ), σ), γ


using Quadrature
using Base.Test

xsamp = linspace(0.0, 10.0, 11)
ysamp = map(x->x^3, xsamp)
@test simps(ysamp, xsamp) == 2500.0

xsamp = linspace(0.0, 10.0, 1001)
ysamp = map(x->x^4, xsamp)
@test_approx_eq simps(ysamp, xsamp) 20000.0

# make sure we give errors for wrong input shape
# input ans samples are not the same length
@test_throws ArgumentError simps([0.5, 0.5, 0.3], [0.1, 0.2])
# simpsons rule needs odd input length
@test_throws ArgumentError simps([0.5, 0.5, 0.3, 0.2], [0.1, 0.2, 0.3, 0.4])


# ---
# title: 504. Base 7
# id: problem504
# author: zhwang
# date: 2022-03-10
# difficulty: Easy
# categories:
# link: <https://leetcode.com/problems/base-7/description/>
# hidden: true
# ---
#
# Given an integer, return its base 7 string representation.
#
# **Example 1:**
#
#
#
#     Input: 100
#     Output: "202"
#
#
# **Example 2:**
#
#
#
#     Input: -7
#     Output: "-10"
#
#
# **Note:** The input will be in range of [-1e7, 1e7].
#
#
## @lc code=start
using LeetCode

function convert_to_base7(num::Int)::String
    num == 0 && return "0"
    sign = num < 0 ? -1 : 1
    num *= sign
    res = Int[]
    while num > 0
        push!(res, num % 7)
        num ÷= 7
    end
    reverse!(res)
    return sign != -1 ? join(res) : "-" * join(res)
end

## @lc code=end


## Julia 1.1.0

# Integer: (min, max)
println("\n\n---------------- Integer: (min, max) ----------------")
for T in [Int8,Int16,Int32,Int64,Int128,UInt8,UInt16,UInt32,UInt64,UInt128]
    println("$(lpad(T,7)): [$(typemin(T)),$(typemax(T))]")
end

# Integer: overflow
println("\n\n---------------- Integer: overflow ----------------")
max = typemax(Int64)
if (max + 1 == typemin(Int64))
    println("Int64 overflow detected ", max, " + 1 = ", max + 1)
end
println("BigInt(typemax(Int64) + 1) is $(BigInt(typemax(Int64)))")

# Float: zero
println("\n\n---------------- Float: zero ----------------")
println("0.0 == 0.0 is ", 0.0 == 0.0)
println("bitstring(0.0) is ", bitstring(0.0))
println("bitstring(-0.0) is ", bitstring(-0.0))
println("1/Inf is ", 1/Inf)

# Float: (min, max)
println("\n\n---------------- Float: (min, max) ----------------")
for T in [Float32, Float64]
    println("$(lpad(T,7)): [$(typemin(T)), $(typemax(T))], eps: $(eps(T))")
end

# Float: eps, nextfloat
println("\n\n---------------- Float: eps, nextfloat ----------------")
for V in [1e-10, 1e-5, 1.0, 2.0, 10.0, 100.0, 1000.0]
    println("eps($(V)) is $(eps(V)), nextfloat is $(nextfloat(V))")
end

# Operator: exponential
x = 3
println("\n\n---------------- Operator: exponential ----------------")
println("x = $(x), 2(x-1)^2 - 3(x-1) + 1 = $(2(x-1)^2 - 3(x-1) + 1)")
println("x = $(x), 2^2x = $(2^2x)")


# # An equation of second order in two dimmensions
#
# Let \\(\Omega = (0, 1)^2\\) the unit square and denote \\( \Gamma = \partial \Omega\\) its boundary.
# Consider the following problem. Given \\(f \in C(\Omega)\\), find a function \\(u\\) satisfying
# -\Delta u + u = f in \Omega
# u = 0 on \Gamma

using SymFEL
using SymPy
using LinearAlgebra
using SparseArrays
using WriteVTK
using JLD2

## discretization parameters
# we use the mesh square.msh (in gmsh format)
# obtained from square.geo using gmsh
# this mesh is formed by quad elements
# we use gmsh module for read the mesh
# there are some problems using Threads and gmsh (in linux)
# the file is square.jld2 is prepared separetely
@load "square.jld2"

# elementary matrices - P2 x P2
elem_Mxy = SymFEL.get_square_lagrange_em((2, 2), (0, 0), (0, 0))
elem_Kxy = SymFEL.get_square_lagrange_em((2, 2), (1, 0), (1, 0)) +
    SymFEL.get_square_lagrange_em((2, 2), (0, 1), (0, 1))

dx = norm(nodes_coordinate[:, elements_bound[1,1]] - nodes_coordinate[:, elements_bound[2,1]])

elem_Kxy_dx = convert(Matrix{Float64}, elem_Kxy.subs(h, dx))
elem_Mxy_dx = convert(Matrix{Float64}, elem_Mxy.subs(h, dx));

# global matrices
K = SymFEL.assemble_squaremesh_FE_matrix(elem_Kxy_dx, elements_int, order1=2, order2=2)
M = SymFEL.assemble_squaremesh_FE_matrix(elem_Mxy_dx, elements_int, order1=2, order2=2)

f = (2*pi^2 + 1) * sin.(pi * nodes_coordinate[1,:]) .* sin.(pi * nodes_coordinate[2,:])

F = M * f

A = K + M

# boundary condition
tgv = 1e30
A[nodes_boundary, nodes_boundary] += tgv * sparse(Matrix{Float64}(I, nodes_boundary_N, nodes_boundary_N))

u = A \ F
u_exact = sin.(pi*nodes_coordinate[1,:]) .* sin.(pi*nodes_coordinate[2,:])
err = u - u_exact

println("L2 error : ", sqrt(err' * M * err))
println("H1 error : ", sqrt(err' * K * err))

# export to vtk
cells = [MeshCell(VTKCellTypes.VTK_QUADRATIC_QUAD, elements_int[1:8, i]) for i = 1:elements_int_N]

points_x = nodes_coordinate[1, :]
points_y = nodes_coordinate[2, :]
vtkfile = vtk_grid("ex3-output", points_x, points_y, cells)

vtkfile["my_point_data", VTKPointData()] = u
outfiles = vtk_save(vtkfile)


## Figures for the Quadratic Equation slides
using Plots
using LaTeXStrings
pyplot() # nicer output
theme(:dark)

##
quadratic1(a,b,c) = x-> a*x^2 + b*x + c
plot(-10:0.01:5, quadratic1(0.5, 3, -2.0),
  linewidth=2,
  legend=false, framestyle = :origin,
  xlabel=L"$x$", ylabel=L"$f(x)$",
  title=L"f(x) = $0.5x^2 + 3x -2$",
  size=(350,350))e
savefig("quadratic1.pdf")

##
ezcontour(x, y, f) = begin
    X = repeat(x', length(y), 1)
    Y = repeat(y, 1, length(x))
    # Evaluate each f(x, y)
    Z = map((x,y) -> f([x,y]), X, Y)
    plot(x, y, Z, st=:surface)
end
quadratic2(A,b) = x -> x'*A*x + x'*b
ezcontour(-3:0.1:3,-3:0.1:3,
  quadratic2([1 0;0 1],[0,0]))
plot!(colorbar=false,size=(500,450))
##
anim = @animate for i=1:180
  plot!(camera=(i,30))
end
gif(anim, "quadratic2a.gif")
##
ezcontour(-4:0.1:4,-4:0.1:4,
  quadratic2([1 0.5;0.5 1],[-1,-1.5]))
plot!(colorbar=false)
##
anim = @animate for i=1:180
  plot!(camera=(i,30))
end
gif(anim, "quadratic2b.gif")
##
Q = [1 0.5; 0.5 1]
s = [-1,-1.5]
x = -Q\(s/2)
@show x
##
using LinearAlgebra
Q = [1 0.5; 0.5 1]
s = [-1,-1.5]
lam,V = eigen(Q)
p1 = ezcontour(-4:0.1:4,-4:0.1:4,
  quadratic2(Q,s))
title!(p1, L"$f(\mathbf{x})$")
p2 = ezcontour(-4:0.1:4,-4:0.1:4,
  quadratic2(Diagonal(lam),V'*s))
title!(p2, L"$g(\mathbf{y})$")
plot(p1,p2,layout=(2,1), colorbar=false)

##
plot(p1,p2,layout=(2,1), colorbar=false,size=(300,650))
savefig("quadratic2b-diag.pdf")

##
using LinearAlgebra
Q = [1 2; 2 1]
s = [-1,-1.5]
lam,V = eigen(Q)
p1 = ezcontour(-4:0.1:4,-4:0.1:4,
  quadratic2(Q,s))
title!(p1, L"$f(\mathbf{x})$")
p2 = ezcontour(-4:0.1:4,-4:0.1:4,
  quadratic2(Diagonal(lam),V'*s))
title!(p2, L"$g(\mathbf{y})$")
plot(p1,p2,layout=(2,1), colorbar=false)

##


#=
from "Error-free transformations in real and complex floating point arithmetics"
by Stef Graillat and Valérie Ménissier-Morain
2007 International Symposium on Nonlinear Theory and its Applications
NOLTA'07, Vancouver, Canada, September 16-19, 2007
=#
@inline function two_sum(a::Complex{T}, b::Complex{T}) where {T<:Real}
    hi1, lo1 = two_sum(a.re, b.re)
    hi2, lo2 = two_sum(a.im, b.im)
    hi = Complex{T}(hi1, hi2)
    lo = Complex{T}(lo1, lo2)
    return hi, lo
end

@inline function one_sum(a::Complex{T}, b::Complex{T}) where {T<:Real}
    hi1, lo1 = two_sum(a.re, b.re)
    hi2, lo2 = two_sum(a.im, b.im)
    hi = Complex{T}(hi1, hi2)
    return hi
end

two_hilo_sum(a::Complex{T}, b::Complex{T}) where {T<:Real} = two_sum(a, b)
one_hilo_sum(a::Complex{T}, b::Complex{T}) where {T<:Real} = one_sum(a, b)

@inline function two_diff(a::Complex{T}, b::Complex{T}) where {T<:Real}
    hi1, lo1 = two_diff(a.re, b.re)
    hi2, lo2 = two_diff(a.im, b.im)
    hi = Complex{T}(hi1, hi2)
    lo = Complex{T}(lo1, lo2)
    return hi, lo
end

@inline function one_diff(a::Complex{T}, b::Complex{T}) where {T<:Real}
    hi1, lo1 = two_diff(a.re, b.re)
    hi2, lo2 = two_diff(a.im, b.im)
    hi = Complex{T}(hi1, hi2)
    return hi
end

two_hilo_diff(a::Complex{T}, b::Complex{T}) where {T<:Real} = two_diff(a, b)
one_hilo_diff(a::Complex{T}, b::Complex{T}) where {T<:Real} = one_diff(a, b)

#=
from "Error-free transformations in real and complex floating point arithmetics"
by Stef Graillat and Valérie Ménissier-Morain
2007 International Symposium on Nonlinear Theory and its Applications
NOLTA'07, Vancouver, Canada, September 16-19, 2007
N.B. modified to order q,r,s
=#
@inline function four_prod(a::Complex{T}, b::Complex{T}) where {T<:Real}
    hi1, lo1 = two_prod(a.re, b.re)
    hi2, lo2 = two_prod(a.im, b.im)
    hi3, lo3 = two_prod(a.re, b.im)
    hi4, lo4 = two_prod(a.im, b.re)
    hi5, lo5 = two_diff(hi1, hi2)
    hi6, lo6 = two_sum(hi3, hi4)
    p = Complex{T}(hi5, hi6)
    q = Complex{T}(lo5, lo6)
    r = Complex{T}(lo1, lo3)
    s = Complex{T}(-lo2, lo4)
    r, s = two_sum(r, s)
    q, r = two_sum(q, r)
    return p, q, r, s
end

@inline function three_prod(a::Complex{T}, b::Complex{T}) where {T<:Real}
    hi1, lo1 = two_prod(a.re, b.re)
    hi2, lo2 = two_prod(a.im, b.im)
    hi3, lo3 = two_prod(a.re, b.im)
    hi4, lo4 = two_prod(a.im, b.re)
    hi5, lo5 = two_diff(hi1, hi2)
    hi6, lo6 = two_sum(hi3, hi4)
    p = Complex{T}(hi5, hi6)
    q = Complex{T}(lo5, lo6)
    r = Complex{T}(lo1, lo3)
    s = Complex{T}(-lo2, lo4)
    q, r = two_sum(q, r, s)
    return p, q, r
end

@inline function two_prod(a::Complex{T}, b::Complex{T}) where {T<:Real}
    hi1, lo1 = two_prod(a.re, b.re)
    hi2, lo2 = two_prod(a.im, b.im)
    hi3, lo3 = two_prod(a.re, b.im)
    hi4, lo4 = two_prod(a.im, b.re)
    hi5, lo5 = two_diff(hi1, hi2)
    hi6, lo6 = two_sum(hi3, hi4)
    p = Complex{T}(hi5, hi6)
    re = one_sum(lo5, lo1, lo2)
    im = one_sum(lo6, lo3, lo4)
    q = Complex{T}(re, im)
    return p, q
end

@inline function one_prod(a::Complex{T}, b::Complex{T}) where {T<:Real}
    hi1, lo1 = two_prod(a.re, b.re)
    hi2, lo2 = two_prod(a.im, b.im)
    hi3, lo3 = two_prod(a.re, b.im)
    hi4, lo4 = two_prod(a.im, b.re)
    hi5, lo5 = two_diff(hi1, hi2)
    hi6 = hi3 + hi4
    p = Complex{T}(hi5, hi6)
    return p
end


"""
    lu_decompose(mat)
Decomposes a `n x n` non singular matrix into a lower triangular matrix (L) and an upper triangular matrix (U)
"""
function lu_decompose(mat)
    n = first(size(mat))
    L = zeros(n, n)
    U = zeros(n, n)

    for i in 1:n
        for j in i:n
            s = 0
            for k in 1:i
                s += L[i, k] * U[k, j]
            end
            U[i, j] = mat[i, j] - s
        end

        for k in i:n
            if i == k
                L[i, i] = 1
            else
                s = 0
                for j in 1:i
                    s += L[k, j] * U[j, i]
                end
                L[k, i] = (mat[k, i] - s) / U[i, i]
            end
        end
    end

    return L, U
end


"""
    Euler(; h = 0.0) :: ExplicitRungeKuttaSolver
    ExplicitEuler(args...; kwargs...) :: ExplicitRungeKuttaSolver

returns an [`ExplicitRungeKuttaSolver`](@ref) for the 1st-order Euler method.
```
"""
function Euler(; h = 0.0)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
        0 0;
        1 1;
    ])
    return ERK(tableau, h)
end
@doc (@doc Euler) ExplicitEuler(args...; kwargs...) = Euler(args...; kwargs...)

"""
    Heun2(; h = 0.0) :: ExplicitRungeKuttaSolver

returns an [`ExplicitRungeKuttaSolver`](@ref) for the 2nd-order Heun method.
"""
function Heun2(; h = 0.0)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
        0   0   0 ;
        1   1   0 ;
        2  1/2 1/2;
    ])
    return ERK(tableau, h)
end

"""
    Ralston2(; h = 0.0) :: ExplicitRungeKuttaSolver

returns an [`ExplicitRungeKuttaSolver`](@ref) for the 2nd-order Ralston method.
"""
function Ralston2(; h = 0.0)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
         0   0   0 ;
        2/3 2/3  0 ;
         2  1/4 3/4;
    ])
    return ERK(tableau, h)
end

"""
    Midpoint(; h = 0.0) :: ExplicitRungeKuttaSolver
    ExplicitMidpoint(args...; kwargs...) :: ExplicitRungeKuttaSolver

returns an [`ExplicitRungeKuttaSolver`](@ref) for the 2nd-order midpoint method.
"""
function Midpoint(; h = 0.0)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
         0   0   0;
        1/2 1/2  0;
         2   0   1;
    ])
    return ERK(tableau, h)
end
@doc (@doc Midpoint) ExplicitMidpoint(args...; kwargs...) = Midpoint(args...; kwargs...)

"""
    Heun3(; h = 0.0) :: ExplicitRungeKuttaSolver

returns an [`ExplicitRungeKuttaSolver`](@ref) for the 3rd-order Heun method.
"""
function Heun3(; h = 0.0)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
         0   0   0   0 ;
        1/3 1/3  0   0 ;
        2/3  0  2/3  0 ;
         3  1/4  0  3/4;
    ])
    return ERK(tableau, h)
end

"""
    Kutta3(; h = 0.0) :: ExplicitRungeKuttaSolver

returns an [`ExplicitRungeKuttaSolver`](@ref) for the 3rd-order Kutta method.
"""
function Kutta3(; h = 0.0)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
         0   0   0   0 ;
        1/2 1/2  0   0 ;
         1  -1   2   0 ;
         3  1/6 2/3 1/6;
    ])
    return ERK(tableau, h)
end

"""
    Ralston3(; h = 0.0) :: ExplicitRungeKuttaSolver

returns an [`ExplicitRungeKuttaSolver`](@ref) for the 3rd-order Ralston method.
"""
function Ralston3(; h = 0.0)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
         0   0   0   0 ;
        1/2 1/2  0   0 ;
        3/4  0  3/4  0 ;
         3  2/9 1/3 4/9;
    ])
    return ERK(tableau, h)
end

"""
    SSPRK3(; h = 0.0) :: ExplicitRungeKuttaSolver

returns an [`ExplicitRungeKuttaSolver`](@ref) for the 3rd-order Strong-Stability-Preserving Runge-Kutta method.
"""
function SSPRK3(; h = 0.0)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
         0   0   0   0 ;
         1   1   0   0 ;
        1/2 1/4 1/4  0 ;
         3  1/6 1/6 2/3;
    ])
    return ERK(tableau, h)
end

"""
    RK4(; h = 0.0) :: ExplicitRungeKuttaSolver

returns an [`ExplicitRungeKuttaSolver`](@ref) for the 4th-order Runge-Kutta method.
"""
function RK4(; h = 0.0)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
         0   0   0   0   0 ;
        1/2 1/2  0   0   0 ;
        1/2  0  1/2  0   0 ;
         1   0   0   1   0 ;
         4  1/6 1/3 1/3 1/6;
    ])
    return ERK(tableau, h)
end

"""
    Rule38(; h = 0.0) :: ExplicitRungeKuttaSolver

returns an [`ExplicitRungeKuttaSolver`](@ref) for the 4th-order 3/8-rule method.
"""
function Rule38(; h = 0.0)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
         0    0    0    0    0 ;
        1/3  1/3   0    0    0 ;
        2/3 -1/3   1    0    0 ;
         1    1   -1    1    0 ;
         4   1/8  3/8  3/8  1/8;
    ])
    return ERK(tableau, h)
end

"""
    HeunEuler(; h = 0.0, δ = 0.0, ϵ = 1e-5, K = 100) :: ExplicitRungeKuttaSolver

returns an [`ExplicitRungeKuttaSolver`](@ref) for the 2nd-order Heun-Euler method with 1st-order error estimate.
"""
function HeunEuler(; h = 0.0, δ = 0.0, ϵ = 1e-5, K = 100)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
        0   0   0 ;
        1   1   0 ;
        2  1/2 1/2;
        1   1   0 ;
    ])
    adaptive = AdaptiveParameters(δ=δ, ϵ=ϵ, K=K)
    return ERK(tableau, h, adaptive)
end

"""
    Fehlberg45(; h = 0.0, δ = 0.0, ϵ = 1e-5, K = 100) :: ExplicitRungeKuttaSolver
    F45(args...; kwargs...) :: ExplicitRungeKuttaSolver

returns an [`ExplicitRungeKuttaSolver`](@ref) for the 4th-order Fehlberg method with 5th-order error estimate.
"""
function Fehlberg45(; h = 0.0, δ = 0.0, ϵ = 1e-5, K = 100)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
          0       0          0          0           0         0    0  ;
         1/4     1/4         0          0           0         0    0  ;
         3/8     3/32       9/32        0           0         0    0  ;
        12/13 1932/2197 -7200/2197  7296/2197       0         0    0  ;
          1    439/216      -8      3680/513    -845/4104     0    0  ;
         1/2    -8/27        2     -3544/2565   1859/4104  -11/40  0  ;
          4     25/216       0      1408/2565   2197/4104   -1/5   0  ;
          5     16/135       0      6656/12825 28561/56430  -9/50 2/55;
    ])
    adaptive = AdaptiveParameters(δ=δ, ϵ=ϵ, K=K)
    return ERK(tableau, h, adaptive)
end
@doc (@doc Fehlberg45) F45(args...; kwargs...) = Fehlberg45(args...; kwargs...)

"""
    DormandPrince54(; h = 0.0, δ = 0.0, ϵ = 1e-5, K = 100) :: ExplicitRungeKuttaSolver
    DP54(args...; kwargs...) :: ExplicitRungeKuttaSolver

returns an [`ExplicitRungeKuttaSolver`](@ref) for the 5th-order Dormand-Prince method with 4th-order error estimate.
"""
function DormandPrince54(; h = 0.0, δ = 0.0, ϵ = 1e-5, K = 100)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
         0        0            0          0          0          0          0      0  ;
        1/5      1/5           0          0          0          0          0      0  ;
        3/10     3/40         9/40        0          0          0          0      0  ;
        4/5     44/45       -56/15      32/9         0          0          0      0  ;
        8/9  19372/6561  -25360/2187 64448/6561  -212/729       0          0      0  ;
         1    9017/3168    -355/33   46732/5247    49/176  -5103/18656     0      0  ;
         1      35/384         0       500/1113   125/192  -2187/6784    11/84    0  ;
         5      35/384         0       500/1113   125/192  -2187/6784    11/84    0  ;
         4    5179/57600       0      7571/16695  393/640 -92097/339200 187/2100 1/40;
    ])
    adaptive = AdaptiveParameters(δ=δ, ϵ=ϵ, K=K)
    return ERK(tableau, h, adaptive)
end
@doc (@doc DormandPrince54) DP54(args...; kwargs...) = DormandPrince54(args...; kwargs...)

"""
    Verner65(; h = 0.0, δ = 0.0, ϵ = 1e-5, K = 100) :: ExplicitRungeKuttaSolver
    V65(args...; kwargs...) :: ExplicitRungeKuttaSolver

returns an [`ExplicitRungeKuttaSolver`](@ref) for the 6th-order Verner method with 5th-order error estimate.
"""
function Verner65(; h = 0.0, δ = 0.0, ϵ = 1e-5, K = 100)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
         0        0          0         0          0          0       0       0        0   ;
        1/6      1/6         0         0          0          0       0       0        0   ;
        4/15     4/75      16/75       0          0          0       0       0        0   ;
        2/3      5/6       -8/3       5/2         0          0       0       0        0   ;
        5/6   -165/64      55/6    -425/64      85/96        0       0       0        0   ;
         1      12/5        -8     4015/612    -11/36      88/255    0       0        0   ;
        1/15 -8263/15000  124/75   -643/680    -81/250   2484/10625  0       0        0   ;
         1    3501/1720  -300/43 297275/52632 -319/2322 24068/84065  0   3850/26703   0   ;
         6       3/40        0      875/2244    23/72     264/1955   0    125/11592 43/616;
         5      13/160       0     2375/5984     5/16      12/85    3/44     0        0   ;
    ])
    adaptive = AdaptiveParameters(δ=δ, ϵ=ϵ, K=K)
    return ERK(tableau, h, adaptive)
end
@doc (@doc Verner65) V65(args...; kwargs...) = Verner65(args...; kwargs...)

"""
    BackwardEuler(; h = 0.0, ϵ = 1e-3, K = 10) :: ImplicitRungeKuttaSolver
    ImplicitEuler(args...; kwargs...) :: ImplicitRungeKuttaSolver

returns an [`ImplicitRungeKuttaSolver`](@ref) for the 1st-order backward Euler method.
"""
function BackwardEuler(; h = 0.0, ϵ = 1e-3, K = 10)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
        1 1;
        1 1;
    ])
    newton = NewtonParameters(ϵ=ϵ, K=K)
    return IRK(tableau, h, newton)
end
@doc (@doc BackwardEuler) ImplicitEuler(args...; kwargs...) = BackwardEuler(args...; kwargs...)

"""
    ImplicitMidpoint(; h = 0.0, ϵ = 1e-3, K = 10) :: ImplicitRungeKuttaSolver

returns an [`ImplicitRungeKuttaSolver`](@ref) for the 2nd-order implicit midpoint method.
"""
function ImplicitMidpoint(; h = 0.0, ϵ = 1e-3, K = 10)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
        1/2 1/2;
         2   1 ;
    ])
    newton = NewtonParameters(ϵ=ϵ, K=K)
    return IRK(tableau, h, newton)
end

"""
    CrankNicolson(; h = 0.0, ϵ = 1e-3, K = 10) :: ImplicitRungeKuttaSolver

returns an [`ImplicitRungeKuttaSolver`](@ref) for the 2nd-order Crank-Nicolson method.
"""
function CrankNicolson(; h = 0.0, ϵ = 1e-3, K = 10)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
        0   0   0 ;
        1  1/2 1/2;
        2  1/2 1/2;
    ])
    newton = NewtonParameters(ϵ=ϵ, K=K)
    return IRK(tableau, h, newton)
end

"""
    SDIRK3(; h = 0.0, ϵ = 1e-3, K = 10) :: ImplicitRungeKuttaSolver

returns an [`ImplicitRungeKuttaSolver`](@ref) for the 3rd-order SDIRK method.
"""
function SDIRK3(; h = 0.0, ϵ = 1e-3, K = 10)
    γ = 1/2 + √3/6
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
         γ    γ   0 ;
        1-γ 1-2γ  γ ;
         3   1/2 1/2;
    ])
    newton = NewtonParameters(ϵ=ϵ, K=K)
    return IRK(tableau, h, newton)
end

"""
    GaussLegendre4(; h = 0.0, ϵ = 1e-3, K = 10) :: ImplicitRungeKuttaSolver
    GL4(args...; kwargs...) :: ImplicitRungeKuttaSolver

returns an [`ImplicitRungeKuttaSolver`](@ref) for the 4th-order Gauss-Legendre method.
"""
function GaussLegendre4(; h = 0.0, ϵ = 1e-3, K = 10)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
        1/2-√3/6   1/4    1/4-√3/6;
        1/2+√3/6 1/4+√3/6   1/4   ;
           4       1/2      1/2   ;
    ])
    newton = NewtonParameters(ϵ=ϵ, K=K)
    return IRK(tableau, h, newton)
end
@doc (@doc GaussLegendre4) GL4(args...; kwargs...) = GaussLegendre4(args...; kwargs...)

"""
    GaussLegendre6(; h = 0.0, ϵ = 1e-3, K = 10) :: ImplicitRungeKuttaSolver
    GL6(args...; kwargs...) :: ImplicitRungeKuttaSolver

returns an [`ImplicitRungeKuttaSolver`](@ref) for the 6th-order Gauss–Legendre method.
"""
function GaussLegendre6(; h = 0.0, ϵ = 1e-3, K = 10)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
        1/2-√15/10     5/36    2/9-√15/15 5/36-√15/30;
           1/2     5/36+√15/24    2/9     5/36-√15/24;
        1/2+√15/10 5/36+√15/30 2/9+√15/15     5/36   ;
            6          5/18       4/9         5/18   ;
    ])
    newton = NewtonParameters(ϵ=ϵ, K=K)
    return IRK(tableau, h, newton)
end
@doc (@doc GaussLegendre6) GL6(args...; kwargs...) = GaussLegendre6(args...; kwargs...)

"""
    LobattoIIIA4(; h = 0.0, ϵ = 1e-3, K = 10) :: ImplicitRungeKuttaSolver

returns an [`ImplicitRungeKuttaSolver`](@ref) for the 4th-order Lobatto IIIA method.
"""
function LobattoIIIA4(; h = 0.0, ϵ = 1e-3, K = 10)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
         0   0    0    0  ;
        1/2 5/24 1/3 -1/24;
         1  1/6  2/3  1/6 ;
         4  1/6  2/3  1/6 ;
    ])
    newton = NewtonParameters(ϵ=ϵ, K=K)
    return IRK(tableau, h, newton)
end

"""
    LobattoIIIB2(; h = 0.0, ϵ = 1e-3, K = 10) :: ImplicitRungeKuttaSolver

returns an [`ImplicitRungeKuttaSolver`](@ref) for the 2nd-order Lobatto IIIB method.
"""
function LobattoIIIB2(; h = 0.0, ϵ = 1e-3, K = 10)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
        1/2 1/2  0 ;
        1/2 1/2  0 ;
         2  1/2 1/2;
    ])
    newton = NewtonParameters(ϵ=ϵ, K=K)
    return IRK(tableau, h, newton)
end

"""
    LobattoIIIB4(; h = 0.0, ϵ = 1e-3, K = 10) :: ImplicitRungeKuttaSolver

returns an [`ImplicitRungeKuttaSolver`](@ref) for the 4th-order Lobatto IIIB method.
"""
function LobattoIIIB4(; h = 0.0, ϵ = 1e-3, K = 10)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
         0  1/6 -1/6  0 ;
        1/2 1/6  1/3  0 ;
         1  1/6  5/6  0 ;
         4  1/6  2/3 1/6;
    ])
    newton = NewtonParameters(ϵ=ϵ, K=K)
    return IRK(tableau, h, newton)
end

"""
    LobattoIIIC2(; h = 0.0, ϵ = 1e-3, K = 10) :: ImplicitRungeKuttaSolver

returns an [`ImplicitRungeKuttaSolver`](@ref) for the 2nd-order Lobatto IIIC method.
"""
function LobattoIIIC2(; h = 0.0, ϵ = 1e-3, K = 10)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
        0  1/2 -1/2;
        1  1/2  1/2;
        2  1/2  1/2;
    ])
    newton = NewtonParameters(ϵ=ϵ, K=K)
    return IRK(tableau, h, newton)
end

"""
    LobattoIIIC4(; h = 0.0, ϵ = 1e-3, K = 10) :: ImplicitRungeKuttaSolver

returns an [`ImplicitRungeKuttaSolver`](@ref) for the 4th-order Lobatto IIIC method.
"""
function LobattoIIIC4(; h = 0.0, ϵ = 1e-3, K = 10)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
         0   1/6 -1/3   1/6 ;
        1/2  1/6  5/12 -1/12;
         1   1/6  2/3   1/6 ;
         4   1/6  2/3   1/6 ;
    ])
    newton = NewtonParameters(ϵ=ϵ, K=K)
    return IRK(tableau, h, newton)
end

"""
    RadauIA3(; h = 0.0, ϵ = 1e-3, K = 10) :: ImplicitRungeKuttaSolver

returns an [`ImplicitRungeKuttaSolver`](@ref) for the 3rd-order Radau IA method.
"""
function RadauIA3(; h = 0.0, ϵ = 1e-3, K = 10)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
         0  1/4 -1/4 ;
        2/3 1/4  5/12;
         3  1/4  3/4 ;
    ])
    newton = NewtonParameters(ϵ=ϵ, K=K)
    return IRK(tableau, h, newton)
end

"""
    RadauIA5(; h = 0.0, ϵ = 1e-3, K = 10) :: ImplicitRungeKuttaSolver

returns an [`ImplicitRungeKuttaSolver`](@ref) for the 5th-order Radau IA method.
"""
function RadauIA5(; h = 0.0, ϵ = 1e-3, K = 10)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
            0     1/9   -1/18-√6/18     -1/18+√6/18  ;
        3/5-√6/10 1/9  11/45+7*√6/360 11/45-43*√6/360;
        3/5+√6/10 1/9 11/45+43*√6/360  11/45-7*√6/360;
            5     1/9    4/9+√6/36       4/9-√6/36   ;
    ])
    newton = NewtonParameters(ϵ=ϵ, K=K)
    return IRK(tableau, h, newton)
end

"""
    RadauIIA3(; h = 0.0, ϵ = 1e-3, K = 10) :: ImplicitRungeKuttaSolver

returns an [`ImplicitRungeKuttaSolver`](@ref) for the 3rd-order Radau IIA method.
"""
function RadauIIA3(; h = 0.0, ϵ = 1e-3, K = 10)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
        1/3  5/12 -1/12;
         1   3/4   1/4 ;
         3   3/4   1/4 ;
    ])
    newton = NewtonParameters(ϵ=ϵ, K=K)
    return IRK(tableau, h, newton)
end

"""
    RadauIIA5(; h = 0.0, ϵ = 1e-3, K = 10) :: ImplicitRungeKuttaSolver

returns an [`ImplicitRungeKuttaSolver`](@ref) for the 5th-order Radau IIA method.
"""
function RadauIIA5(; h = 0.0, ϵ = 1e-3, K = 10)
    h_T = typeof(h)
    tableau = ButcherTableau(h_T[
        2/5-√6/10   11/45-7*√6/360   37/225-169*√6/1800 -2/225+√6/75;
        2/5+√6/10 37/225+169*√6/1800   11/45+7*√6/360   -2/225-√6/75;
           1           4/9-√6/36          4/9+√6/36          1/9    ;
           5           4/9-√6/36          4/9+√6/36          1/9    ;
    ])
    newton = NewtonParameters(ϵ=ϵ, K=K)
    return IRK(tableau, h, newton)
end

"""
    ExponentialRK4(; h = 0.0) :: ExplicitExponentialRungeKuttaSolver

returns an [`ExplicitExponentialRungeKuttaSolver`](@ref) for the 4th-order EERK method.
"""
function ExponentialRK4(; h = 0.0)
    return EERK(h)
end
@doc (@doc ExponentialRK4) ERK4(args...; kwargs...) = ExponentialRK4(args...; kwargs...)


using PyCall
using Plots

np = pyimport("numpy")

function main()

    # x = 5
    # x = [π 3.0 0 12312;
    #      5 12. 1 3412.]
    # display(np.sin(x))

    x = [1, 2, 3, 4, 5]
    y = [5, 3, 1, 3, 5]

    p = np.polyfit(x, y, 2)
    # display(p)

    x1 = 1:0.1:5
    p1 = np.polyval(p, x1)

    scatter(x, y, label="data")
    plot!(x1, p1, label="fit")

end

main()


using Test, PolynomialBases
import SymPy

tol = 5.e-15

for p in 0:6
    basis_sympy = LobattoLegendre(p, SymPy.Sym)
    basis_float = LobattoLegendre(p, Float64)
    @test maximum(abs.( float.(basis_sympy.nodes) - basis_float.nodes )) < tol
    @test maximum(abs.( float.(basis_sympy.weights) - basis_float.weights )) < tol
    @test maximum(abs.( float.(basis_sympy.baryweights) - basis_float.baryweights )) < tol
    @test maximum(abs.( float.(basis_sympy.D) - basis_float.D, )) < 2tol
end
@test_throws ArgumentError LobattoLegendre(7, SymPy.Sym)

for p in 0:4
    basis_sympy = GaussLegendre(p, SymPy.Sym)
    basis_float = GaussLegendre(p, Float64)
    @test maximum(abs.( float.(basis_sympy.nodes) - basis_float.nodes )) < tol
    @test maximum(abs.( float.(basis_sympy.weights) - basis_float.weights )) < tol
    @test maximum(abs.( float.(basis_sympy.baryweights) - basis_float.baryweights )) < tol
    @test maximum(abs.( float.(basis_sympy.D) - basis_float.D, )) < 2tol
end
@test_throws ArgumentError GaussLegendre(5, SymPy.Sym)

interpolation_matrix([-1, 1], LobattoLegendre(4, SymPy.Sym))
interpolation_matrix([-1, 1], GaussLegendre(4, SymPy.Sym))


using NLsolve, LinearAlgebra
"""
`Base.length(tab::ODERKTableau)`

Defines the length of a Runge-Kutta method to be the number of stages.
"""
Base.length(tab::ODERKTableau) = tab.stages

"""
`stability_region(z,tab::ODERKTableau)`

Calculates the stability function from the tableau at `z`. Stable if <1.

```math
r(z) = \\frac{\\det(I-zA+zeb^T)}{\\det(I-zA)}
```
"""
stability_region(z,tab::ODERKTableau) = det(Matrix{Float64}(I,tab.stages,tab.stages)- z*tab.A + z*ones(tab.stages)*tab.α')/det(Matrix{Float64}(I,tab.stages,tab.stages)-z*tab.A)

"""
`stability_region(tab::ODERKTableau; initial_guess=-3.0)`

Calculates the length of the stability region in the real axis.
"""
function stability_region(tab::ODERKTableau; initial_guess=-3.0)
  residual! = function (resid, x)
    resid[1] = abs(stability_region(x[1], tab)) - 1
  end
  sol = nlsolve(residual!, [initial_guess])
  sol.zero[1]
end


# # clip-06-10.jl

# Load Julia packages (libraries) needed  for the snippets in chapter 0

using StatisticalRethinking, Optim
#gr(size=(600,600));

# ### snippet 3.2

p_grid = range(0, step=0.001, stop=1)
prior = ones(length(p_grid))
likelihood = [pdf(Binomial(9, p), 6) for p in p_grid]
posterior = likelihood .* prior
posterior = posterior / sum(posterior)

# ### snippet 3.3
# Draw 10000 samples from this posterior distribution

N = 10000
samples = sample(p_grid, Weights(posterior), N);

# In StatisticalRethinkingJulia samples will always be stored
# in an MCMCChains.Chains object.

chn = MCMCChains.Chains(reshape(samples, N, 1, 1), ["toss"]);

# Describe the chain

MCMCChains.describe(chn)

# ### snippet 3.6

v = 0.0
for i in 1:length(p_grid)
  global v
  if p_grid[i] < 0.5
    v += posterior[i]
  end
end
v

# ### snippet 3.7

mapreduce(p -> p < 0.5 ? 1 : 0, +, samples) / N   |> display

# ### snippet 3.8

mapreduce(p -> (p > 0.5 && p < 0.75) ? 1 : 0, +, samples) / N   |> display

# ### snippet 3.9

quantile(samples, 0.8)

# ### snippet 3.10

quantile(samples, [0.1, 0.9])

# End of `03/clip-06-10.jl`


@testset "Construction" begin

    # basic constructor
    xs = rand(5)
    ys = rand(5)
    p = Lagrange(xs, ys)
    @test all(p.(xs) .≈  ys)

    # test weights
    @test all(p.ws .≈  SP.lagrange_barycentric_weights(xs))
    zs = rand(5)
    q = Lagrange(xs, p.ws, zs)
    @test all(q.(xs) .≈  zs)

    # fit is alternate
    q = fit(Lagrange, xs, ys)
    us = rand(5)
    @test all(p.(us) .≈ q.(us))

    # test one, zero, variable
    x = variable()
    q = fit(Lagrange, [1,2,3],[2,3,1])
    @test iszero(zero(q))
    @test convert(Polynomial, one(q)) == one(x)
    @test convert(Polynomial, variable(q)) == x

end

@testset "Conversion" begin

    xs = [1,2,3]
    ys = xs.^2
    p = fit(Lagrange, xs, ys)
    q = convert(Polynomial, p)
    @test coeffs(q)  == [0,0,1]

    pp = fit(Lagrange, xs, q)
    @test coeffs(pp) == coeffs(p)

end

@testset "Evaluation" begin

    xs = rand(5)
    ys = sin.(xs)
    p = Lagrange(xs, ys)
    @test all(p.(xs) == ys)   # shortcuts  for xs
    us = rand(5)
    @test all(abs.(p.(us) - sin.(us)) .<= 1e-1)

    xs, ws = SP.lagrange_barycentric_nodes_weights(Chebyshev, 64)
    ys = sin.(xs)
    p = Lagrange(xs, ws, ys)
    us = 0.1 .+ 0.8*rand(5) # avoid edges...
    @test all(abs.(sin.(us) - p.(us)) .<=  1e-3)

end

@testset "Arithmetic" begin

    p = fit(Lagrange, [1,2,3], x -> x^2)
    q = fit(Lagrange, [2,3,4,5], x -> x^3)

    us = rand(5)

    @test maximum(abs, p.(us) + q.(us) - (p + q).(us)) <= 1e-5
    @test maximum(abs, p.(us) .* q.(us) - (p * q).(us)) <= 1e-2  # this  can  be way  off for even moderate sized polynomials.

end

@testset "Elementwise operations" begin

    p = fit(Lagrange, [1,2,3], x -> x^2)
    p + 1
    -p
    2p

end

@testset "roots" begin

    q = fit(Lagrange, [2,3,4,5], x -> (x-1)*(x-1/2)*(x+1/2))
    @test all(roots(q) .≈ [-1/2, 1/2, 1])

end


# indicator of an affine set

using LinearAlgebra
using SparseArrays
using SuiteSparse

export IndAffine

### ABSTRACT TYPE

abstract type IndAffine <: ProximableFunction end

is_affine(f::IndAffine) = true
is_generalized_quadratic(f::IndAffine) = true

fun_name(f::IndAffine) = "Indicator of an affine subspace"

### CONSTRUCTORS

"""
**Indicator of an affine subspace**

    IndAffine(A, b; iterative=false)

If `A` is a matrix (dense or sparse) and `b` is a vector, returns the indicator function of the set
```math
S = \\{x : Ax = b\\}.
```
If `A` is a vector and `b` is a scalar, returns the indicator function of the set
```math
S = \\{x : \\langle A, x \\rangle = b\\}.
```
By default, a direct method (QR factorization of matrix `A'`) is used to evaluate `prox!`.
If `iterative=true`, then `prox!` is evaluated approximately using an iterative method instead.
"""
function IndAffine(A::M, b::V; iterative=false) where {M, V}
    if iterative == false
        IndAffineDirect(A, b)
    else
        IndAffineIterative(A, b)
    end
end

### INCLUDE CONCRETE TYPES

using LinearAlgebra
using SparseArrays
using SuiteSparse

include("indAffineDirect.jl")
include("indAffineIterative.jl")


# example copied from th0br0 with modifications
using CmdStan

y = collect(1:100);
x = [3:3:300 5:5:500];

model = """
data {
  int<lower = 0> N;
  int<lower = 0> k;
  matrix[N, k] x;
  real<lower=0> y[N];
}
parameters {
  vector[k] beta;
  real<lower=0> sigma;
}
model {
  y ~ normal(x * beta, sigma);
}
""";

# beta parameters are accessed using beta.[1-9+] syntax
data = Dict("N"=>100, "k" => 2, "x" => x, "y" => y);
stanmodel = Stanmodel(monitors = ["beta.1", "beta.2", "sigma"], model=model,
          output_format=:mcmcchains);

rc, sim, cnames = stan(stanmodel, data; diagnostics = false);

cnames |> display
# 3-element Array{String,1}: "beta.1", "beta.2" ,"sigma"
println()

describe(sim)


"""
    newton(x, xd, yd)
adalah fungsi yang digunakan untuk mencari nilai interpolasi pada titik/vektor `x`, jika
diketahui suatu himpunan pasangan terurut `(xd,yd)`.

# Example
```jl
julia> xd = [1,2,3,5];

julia> yd = [1.06 1.12 1.34 1.78];

julia> y,D = newton(4,xd,yd);

julia> y
1.6

julia> D
4×4 Array{Float64,2}:
 1.06  0.0   0.0    0.0
 1.12  0.06  0.0    0.0
 1.34  0.22  0.08   0.0
 1.78  0.22  0.0   -0.02
```
return solusi hampiran interpolasi `y` dan matriks beda-terbagi `D`.
"""
function newton(x, xd, yd)
  m=length(x);
  y=zeros(m)
  if m > 1
    for i=1:m
      # proses satu titik demi satu titik
      y[i],D =newton(x[i], xd, yd);
    end
    return y,D
  end
  #% periksa jumlah titik dan tentukan derajat polinom
  ntitik = length(xd);
  #% hitung tabel beda-terbagi (divided-difference)
  D = zeros(ntitik,ntitik)
  D[:,1]=yd;          #% kolom pertama
  for j=2:ntitik      #% kolom ke-2 dan seterusnya
    for k=j:ntitik
        D[k,j] = (D[k,j-1]-D[k-1,j-1])/(xd[k]-xd[k-j+1]);
    end
  end
  #% hitung interpolasi Newton
  y = D[1,1]; s = 1;
  for i=2:ntitik
    s = s * (x-xd[i-1]);
    y = y + D[i,i]*s;
  end
  return y, D
end


include("../aimajulia.jl");

using Test;

using Main.aimajulia;

#The following mdp tests are from the aima-python doctests

tm = Dict([Pair("A", Dict([Pair("a1", (0.3, "B")), Pair("a2", (0.7, "C"))])),
            Pair("B", Dict([Pair("a1", (0.5, "B")), Pair("a2", (0.5, "A"))])),
            Pair("C", Dict([Pair("a1", (0.9, "A")), Pair("a2", (0.1, "B"))]))]);

mdp = MarkovDecisionProcess("A", Set(["a1", "a2"]), Set(["C"]), tm, states=Set(["A","B","C"]));

@test (transition_model(mdp, "A", "a1") == (0.3, "B"));

@test (transition_model(mdp, "B", "a2") == (0.5, "A"));

@test (transition_model(mdp, "C", "a1") == (0.9, "A"));

@test (repr(value_iteration(aimajulia.sequential_decision_environment, epsilon=0.01)) ==
        "Dict((2, 3)=>0.486437,(2, 1)=>0.398102,(3, 1)=>0.509285,(1, 4)=>0.129589,(3, 3)=>0.795361,(1, 3)=>0.344613,(3, 2)=>0.649581,(2, 4)=>-1.0,(1, 1)=>0.295435,(1, 2)=>0.253487,(3, 4)=>1.0)");

pi = optimal_policy(aimajulia.sequential_decision_environment, value_iteration(aimajulia.sequential_decision_environment, epsilon=0.01));

@test (repr(pi) == "Dict{Any,Any}((2, 3)=>(1, 0),(2, 1)=>(1, 0),(3, 1)=>(0, 1),(1, 4)=>(0, -1),(3, 3)=>(0, 1),(1, 3)=>(1, 0),(3, 2)=>(0, 1),(2, 4)=>nothing,(1, 1)=>(1, 0),(1, 2)=>(0, 1),(3, 4)=>nothing)");

@test (repr(to_arrows(aimajulia.sequential_decision_environment, pi)) == "Union{Nothing, String}[\"v\" \">\" \"v\" \"<\"; \"v\" nothing \"v\" \".\"; \">\" \">\" \">\" \".\"]");

@test (policy_iteration(aimajulia.sequential_decision_environment) ==
        Dict([Pair((2,3),(1,0)),
            Pair((2,1),(1,0)),
            Pair((3,1),(0,1)),
            Pair((1,4),(0,-1)),
            Pair((3,3),(0,1)),
            Pair((1,3),(1,0)),
            Pair((3,2),(0,1)),
            Pair((2,4),nothing),
            Pair((1,1),(1,0)),
            Pair((1,2),(0,1)),
            Pair((3,4),nothing)]));


"Square the sum of the first `n` positive integers"
function square_of_sum(n::Int)

end

"Sum the squares of the first `n` positive integers"
function sum_of_squares(n::Int)

end

"Subtract the sum of squares from square of the sum of the first `n` positive ints"
function difference(n::Int)

end


const TRANSITION = [1.0 1.0; 0.0 1.0]
const POS_NOISE = 1.5
const NOISE = [POS_NOISE 0.0; 0.0 sqrt(2)*POS_NOISE]
const PREDICT_NOISE = [0.1 0.0; 0.0 0.3]

struct State
    p::Float64
    v::Float64
    Σ::Array{Float64, 2}
end
State(p, v) = State(p, v, NOISE)

function predict(s::State, noise=PREDICT_NOISE)
    p, v = TRANSITION * [s.p ; s.v]
    Σ = TRANSITION * s.Σ * TRANSITION' + noise
    State(p, v, Σ)
end

function update(predicted::State, observed::State)
    K = predicted.Σ / (predicted.Σ + observed.Σ)
    # x = [predicted.p ; predicted.v]
    # z = [observed.p; observed.v]
    # p, v = x + K * (z - x)
    p = observed.p
    v = observed.v
    Σ = predicted.Σ - K * predicted.Σ
    State(p, v, Σ)
end

function smoothed_velocity_estimate(old::State, new::State, λ)
    diff = new.p - old.p
    scale = λ
    #scale = 1.0 - 0.5 * (1 - λ) * (1 + 1.0 / sqrt(2π * old.Σ[2, 2]^2))
    return scale * diff + (1 - scale) * old.v
end

function pdf(s::State, p::Float64)
    σ = s.Σ[1, 1]
    exp(-((s.p - p)^2) / (2σ^2)) / (sqrt(2π * σ^2))
end