# ------------------------------------------------------------------------------------------
# # Delay Coordinates & Neighborhoods
#
#
# Topics:
# * Delay coordinates & why they ROCK!
# * A `Reconstruction` is a subtype of `AbstractDataset`
# * Estimating Parameters for `Reconstruction`s
# * Multiple-time, multiple timeseries `Reconstruction`s
# * Finding neighborhoods of points in a `Dataset`
# * Excluding temporal neighbors
#
# ---
#
# # Delay Coordinates Reconstruction
# Let's say you have a "real-world system" which you measure in an experimental set-up. You
# are assuming that the system is composed of several dynamic variables, but you can only
# measure one of them (or some function of the variable).
#
# You have a severe lack of recorded information for the system. What do you do?
# 1. Give up on science, it is a complete waste of time.
# 2. Use [Taken's theorem](https://en.wikipedia.org/wiki/Takens%27s_theorem), which is
# indistinguishable from magic.
#
# **DynamicalSystems.jl** suggests the third approach.
#
# From a timeseries $s$ one can *reconstruct* a state-space $\mathbf{z}$ simply by shifting
# $s$ in time, like
#
#   $$\mathbf{z}(n) = (s(n), s(n+\tau), s(n+2\tau), \dots, s(n+(D-1)\tau))$$
#
# This is done with the `Reconstruction(s, D, τ)` function
# ------------------------------------------------------------------------------------------

using DynamicalSystems

s = rand(100000)
D = 3 # reconstruction dimension
τ = 4 # reconstruction delay
R = Reconstruction(s, D, τ)

# ------------------------------------------------------------------------------------------
# ---
#
# Here are some nice examples of `Reconstruction`s of a 3D continuous chaotic system, using
# each of the variables of the system, different delay times and dimension of `2`:
# ------------------------------------------------------------------------------------------

using DynamicalSystems, PyPlot

ds = Systems.gissinger(ones(3)) # 3D continuous chaotic system, also shown in orbit diagrams tutorial
dt = 0.05
data = trajectory(ds, 1000.0, dt = dt)

xyz = columns(data)

figure(figsize = (12,10))
k = 1
for i in 1:3
    for τ in [5, 30, 100]
        R = Reconstruction(xyz[i], 2, τ)
        ax = subplot(3,3,k)
        plot(R[:, 1], R[:, 2], color = "C$(k-1)", lw = 0.8)
        title("var = $i, τ = $τ")
        k+=1
    end
end

tight_layout()
suptitle("2D Reconstructions")
subplots_adjust(top=0.9);

# ------------------------------------------------------------------------------------------
# How does this compare to the "real" two-dimensional representation of the system?
# ------------------------------------------------------------------------------------------

figure(figsize=(6,4))
plot(data[:, 2], data[:, 3], lw=1.0);

# ------------------------------------------------------------------------------------------
# ---
#
# # `Reconstruction <: AbstractDataset`
# A `Reconstruction` instance can be passed around and used exactly like a `Dataset`!
#
# Let's look at a `Reconstruction` of data from a gissinger system's trajectory above
# ------------------------------------------------------------------------------------------

R = Reconstruction(data[:, 1], 2, 30)
R[31:end, 1] == R[1:end-30, 2]

a = 0.0
for point ∈ R
    a += mean(point)
end
a/length(R)

# ------------------------------------------------------------------------------------------
# **Taken's theorem says that some quantities remain invariant under a reconstruction**
#
# We'll show this using a `Reconstruction` with the 3rd dimension of `data`.
#
# (D = 2 is not best for this system. D = 3 is better!)
# ------------------------------------------------------------------------------------------

R = Reconstruction(data[:, 1], 3, 30)

# ------------------------------------------------------------------------------------------
# Let's compare the information dimension of the `Reconstruction`
# ------------------------------------------------------------------------------------------

I1 = information_dim(R)

# ------------------------------------------------------------------------------------------
# and the information dimension of the attractor directly
# ------------------------------------------------------------------------------------------

I2 = information_dim(data)

println("|Reconstructed - original| dimension: $(abs(I1 - I2))")

# ------------------------------------------------------------------------------------------
# # Estimating Reconstruction Parameters
#
# It is important to understand that even though Taken's theorem is 99% magic, it is **not**
# 100%. One still has to choose "appropriately good" values for both the delay time as well
# as the reconstruction dimension! Thankfully, **DynamicalSystems.jl** has some support for
# that as well!
# ------------------------------------------------------------------------------------------

# ------------------------------------------------------------------------------------------
# * `estimate_delay` estimates delay time `τ` using the autocorrelation of the signal
# * `estimate_dimension` returns an estimator of the embedding dimension `D` using Cao's
# method
# ------------------------------------------------------------------------------------------

# ------------------------------------------------------------------------------------------
# # Multiple-time, multiple-timeseries Reconstructions
#
# The `Reconstruction` we have seen so far is just a "Vanilla version"...
#
# One can also perform:
#
# 1. Reconstructions with multiple delay times, which tries to capture the effect of
# multiple timescales existing in a system.
# 2. Reconstructions with multiple timeseries.
# 3. Reconstructions with multiple timeseries *and* multiple delay times!
#
# See the documentation string of `Reconstruction` for more!
# ------------------------------------------------------------------------------------------

# ------------------------------------------------------------------------------------------
# # Neighborhoods
#
# A "neighborhood" is a collection of points that is near a given point. `Dataset`s
# interface the module
# [`NearestNeighbors`](https://github.com/KristofferC/NearestNeighbors.jl) in order to find
# this neighborhood.
#
# We use the function `neighborhood`. The call signature is:
# ```julia
# neighborhood(point, tree, ntype)
# ```
#
# ------------------------------------------------------------------------------------------

# ------------------------------------------------------------------------------------------
# `point` is simply the query point. `tree` is the structure required by
# [`NearestNeighbors`](https://github.com/KristofferC/NearestNeighbors.jl), and is obtained
# simply by:
# ------------------------------------------------------------------------------------------

dataset = Dataset(rand(1000,3))
tree = KDTree(dataset)

# ------------------------------------------------------------------------------------------
# The third argument to `neighborhood` is the *type* of the neighborhood.
#
# * There are two types of neighborhoods!
#
# The first one is defined as the `k` nearest points to a given point. It is represented in
# code by:
# ------------------------------------------------------------------------------------------

mybuddies = FixedMassNeighborhood(3)

# ------------------------------------------------------------------------------------------
# *For experts: for a `FixedMassNeighborhood` a KNN search is done*
# ------------------------------------------------------------------------------------------

point = ones(3)
n = neighborhood(point, tree, mybuddies)

# ------------------------------------------------------------------------------------------
# Notice that the `neighborhood` function does not return the points themselves, but rather
# the indices of the points in the original data:
# ------------------------------------------------------------------------------------------

println("Fixed mass neighborhood of $(point) is:")

for i in n
    println(dataset[i])
end

# ------------------------------------------------------------------------------------------
# ---
#
# The second type of neighborhood contains all the points that are within some given
# distance `ε` from the query.
#
# In code, we represent this as:
# ------------------------------------------------------------------------------------------

where_u_at = FixedSizeNeighborhood(0.001)

# ------------------------------------------------------------------------------------------
#  *For experts: for `FixedSizeNeighborhood` an inrange search is done*
# ------------------------------------------------------------------------------------------

n2 = neighborhood(point, tree, where_u_at)

plz_come_closer = FixedSizeNeighborhood(0.2)
n2 = neighborhood(point, tree, plz_come_closer)

println("Fixed size neighborhood of $(point) is:")

for i in n2
    println(dataset[i])
end

# ------------------------------------------------------------------------------------------
# Okay, so points that have distance < ε are accepted as a neighborhood.
#
# How do we define the "distance" though? When defining a `tree`, you can optionally give a
# distance function. By default Euclidean distance is used, but others also work. For
# example, we can use the `Distances` package to get distance functions,
# ------------------------------------------------------------------------------------------

using Distances

# ------------------------------------------------------------------------------------------
# and define the distance as the `Distances`'s `Chebyshev` distance:
# ------------------------------------------------------------------------------------------

funky_tree = KDTree(dataset, Chebyshev())

n3 = neighborhood(point, funky_tree, plz_come_closer)

# ------------------------------------------------------------------------------------------
# # Excluding temporal neighbors
#
# Before moving on, let's see one last thing.
#
# In this example, the point I want the neighborhood of is now part of my dataset:
# ------------------------------------------------------------------------------------------

point = dataset[end]

# ------------------------------------------------------------------------------------------
# Let's calculate again the two neighborhoods
# ------------------------------------------------------------------------------------------

tree = KDTree(dataset)

# ------------------------------------------------------------------------------------------
# We'll find suuuuuuuper close neighbors with a **very** small $\epsilon$:
# ------------------------------------------------------------------------------------------

ε = 0.000001
where_u_at = FixedSizeNeighborhood(ε)
n2 = neighborhood(point, tree, where_u_at)

# ------------------------------------------------------------------------------------------
# and now we can find the nearest neighbor:
# ------------------------------------------------------------------------------------------

my_best_friend = FixedMassNeighborhood(1)
n3 = neighborhood(point, tree, my_best_friend)

println(n2)
println(n3)

length(dataset) == n2[1] == n3[1]

# ------------------------------------------------------------------------------------------
# **What is happening here is that the `neighborhood` also counted the `point` itself, since
# it is also part of the dataset.**
#
# * Almost always this behavior needs to be avoided. For this reason, there is a second
# method for `neighborhood`:
#
# ```julia
# neighborhood(point, tree, ntype, idx::Int, w::Int = 1)
# ```
#
# In this case, `idx` is the index of the point in the original data. `w` stands for the
# Theiler window (positive integer).
#
# Only points that have index
# `abs(i - idx) ≥ w` are returned as a neighborhood, to exclude close temporal neighbors.
#
# * The default `w=1` is the case of excluding the `point` itself.
#
# ---
#
# Let's revisit the last example (using the default value of `w = 1`):
# ------------------------------------------------------------------------------------------

point = dataset[end]
idx = length(dataset)

n2 = neighborhood(point, tree, where_u_at, idx)
n3 = neighborhood(point, tree, my_best_friend, idx)

println(n2)
println(n3)

# ------------------------------------------------------------------------------------------
# As you can see, there isn't *any* neighbor of `point` with distance `< 0.000001` in this
# dataset, but there is always a nearest neighbor:
# ------------------------------------------------------------------------------------------

println(dataset[n3[1]], " is the nearest neighbor of ", point)

# ------------------------------------------------------------------------------------------
# # Docstrings
# ------------------------------------------------------------------------------------------

?Reconstruction

?neighborhood

?AbstractNeighborhood

.nbexports/introductory-tutorials/broader-topics-and-ecosystem/introduction-to-dynamicalsystems.jl/5. Delay Coordinates & Neighborhoods.jl


using LinearAlgebra
function softmax(a)
    if ndims(a) == 1
        return exp.(a) ./ sum(exp.(a))
    else
        return exp.(a) ./ sum(exp.(a), dims = 2)
    end
end
function fit(x, t; alpha = 0.01, tau_max = 1000)
    function CEE(w, x, t) #this is Cross entropy Error
        p = softmax(x * w)
        grad = -(x' * (t - p))
        return grad / length(x[:, 1])
    end
    if size(x)[1] != size(t)[1]#Processing when the matrix is organized by dependent variable
        x = x'
    end
    x = hcat(ones(size(x)[1], 1), x)
    w = ones(size(x)[2], size(t)[2])
    for tau in 1 : tau_max
        grad = CEE(w, x, t)
        w -= alpha * grad
    end
    return w
end

function forecast(x, w)
    x = hcat(ones(size(x)[1], 1), x)
    return softmax(x * w)
end

#Pass so that the row is each data sample and the column is each feature.
function predict(x, w)
    x = hcat(ones(size(x)[1], 1), x)
    s = softmax(x * w)
    p = [findfirst(s[i, :] .== maximum(s[i, :])) for i in 1:size(s)[1]]
end


using OrdinaryDiffEq, ModelingToolkit, DiffEqOperators, DomainSets
# Method of Manufactured Solutions: exact solution
u_exact = (x,t) -> exp.(-t) * cos.(x)

@parameters t x y
@variables u(..)
Dxx = Differential(x)^2
Dyy = Differential(y)^2
Dt = Differential(t)
t_min= 0.
t_max = 2.0
x_min = 0.
x_max = 2.
y_min = 0.
y_max = 2.

#xmax = 50.0
#ymax = 30.0
#ncells = 50.0
#dx = x_max / ncells

lecturer(x, y) = x > x_max / 2 - dx && x < x_max / 2 + dx && y > y_max / 10 - dx && y < y_max / 10 + dx
emission(x, y, emisrate) = lecturer(x, y) ? emisrate : 0
@register emission(x, y, emisrate)
emisrate = 10;

# 3D PDE
eq  = Dt(u(t,x,y)) ~ Dxx(u(t,x,y)) + Dyy(u(t,x,y)) + emission(x, y, emisrate)
#emission(x, y, emisrate) = emisrate

analytic_sol_func(t,x,y) = exp(x+y)*cos(x+y+4t)
# Initial and boundary conditionn

bcs = #[u(t_min,x,y) ~ analytic_sol_func(t_min,x,y),
      [u(t_min,x,y) ~ 0,
       u(t,x_min,y) ~ 0,
       u(t,x_max,y) ~ 0,
       u(t,x,y_min) ~ 0,
       u(t,x,y_max) ~ 0]

# Space and time domains
domains = [t ∈ IntervalDomain(t_min,t_max),
           x ∈ IntervalDomain(x_min,x_max),
           y ∈ IntervalDomain(y_min,y_max)]
pdesys = PDESystem([eq],bcs,domains,[t,x,y],[u(t,x,y)])

# Method of lines discretization
dx = 0.1; dy = 0.1
discretization = MOLFiniteDifference([x=>dx,y=>dy],t)
prob = ModelingToolkit.discretize(pdesys,discretization)
sol = solve(prob,Tsit5())

# Ploting
using Plots
xs,ys = [infimum(d.domain):dx:supremum(d.domain) for d in domains]
u_sol = reshape(sol.u[300], length(xs)-2,length(ys)-2)

plot(xs[2:length(xs)-1], ys[2:length(ys)-1], u_sol, linetype=:contourf,title = "solution")

#Animation
anim = @animate for i ∈ 1:length(sol.t)
    u_sol = reshape(sol.u[i], length(xs)-2,length(ys)-2)
    plot(xs[2:length(xs)-1], ys[2:length(ys)-1], u_sol, linetype=:contourf,title = "solution")

end
gif(anim, "anim_fps15.gif", fps = 15)


module CorrelationDimension

import Plots

using Statistics: mean, covm, varm

using Distributions: fit
using LinearAlgebra: norm
using StatsBase: Histogram

export correlation_dimension, plot_correlation

"""
    correlation_dimension(x; dist=(x1,x2)->norm(x1.-x2)) -> (corrdim, logr, logNr, intercept)

Compute an estimate of the correlation dimension for a set of points in space, with
coordinates for the ith point given as `x[:,i]`.

This is given by

δ̂ = ∂[log N(r)]/∂[log r]

where N(r) is the number of point pairs within a distance r of each other.  This is
estimated by binning the pair-distances into bins (chosen automatically)
and fitting a straight line through the log-distribution.

The distance function `dist` by default is the Cartesian 2-norm, but any function may be
supplied.

The function returns a named tuple of the estimate of the correlation dimension, `corrdim`,
the lower edges of the distance bins, `logr`, the weights in the distance bins `logNr`, and
the `intercept` of the best fitting line (of which the slope is `corrdim`).
"""
function correlation_dimension(x; dist=(a,b)->norm(a .- b))
    D, n = size(x)
    # List of log(pairwise distances)
    point1 = Vector{float(eltype(x))}(undef, D)
    point2 = similar(point1)
    logr = float(eltype(x))[]
    for i in 1:n, j in (i+1):n
        point1 .= x[:,i]
        point2 .= x[:,j]
        push!(logr, log(dist(point1, point2)))
    end
    # Construct histogram
    h = fit(Histogram, logr)
    # Linear regression
    logr = h.edges[1][1:end-1]
    logNr = log.(cumsum(h.weights./(n*(n-1)/2)))
    c, δ̂ = linear_regression(logr, logNr)
    (corrdim=δ̂, logr=logr, logNr=logNr, intercept=c)
end

"""
    correlation_dimension(x1, x2, ..., xN; kwargs...) -> δ̂

Specify coordinates with dimension N as N separate vectors.
"""
correlation_dimension(x, y, args...; kwargs...) = correlation_dimension(permutedims(hcat(x, y, args...)))

"""
    plot_correlation((logr, logNr, slope, intercept))

Create a plot of the output from `correlation_dimension`.  Input is a tuple
of the log of the distance bins (`logr`), the weights of the histogram (`logNr`),
plus the `slope` and `intercept` of the best fitting line.  The `slope` therefore
is the estimate of the ccorrelation dimension.
"""
function plot_correlation((corrdim, logr, logNr, intercept))
    Plots.scatter(logr, logNr, label="Data", legend=:topleft)
    Plots.plot!(x->corrdim*x+intercept, label="y = $(round(corrdim, sigdigits=3))x + $(round(intercept, sigdigits=3))")
end

"""
    linear_regression(x, y)

Perform simple linear regression using Ordinary Least Squares. Returns `a` and `b` such
that `a + b*x` is the closest straight line to the given points `(x, y)`, i.e., such that
the squared error between `y` and `a + b*x` is minimized.
"""
function linear_regression(x::AbstractVector, y::AbstractVector)
    size(x) == size(y) || throw(DimensionMismatch("x and y must be the same size"))
    mx, my = mean(x), mean(y)
    b = covm(x, mx, y, my)/varm(x, mx)
    a = my - b*mx
    return a, b
end

end # module


# Michelle Medina
# August 11, 2020
# Testing Convergence of Derivative

# USING L2 NORM
H = []
E = []

for N in 50:200
    grid = Params(N,4)

	xVal = collect(range(0.0, stop=1.0, length = 129))
	y = calcNodal(x -> sin(4*pi*x), xVal)
	cosx = calcNodal(x -> 4*pi*cos(4*pi*x), xVal)
	modal = Nodal_2_H(y)

	dy = zeros(length(xVal))
	for index in CartesianIndices(xVal)
		dy[index] = derivEvaluate(modal, xVal[index])
	end

    # u = functionOnGrid(x->sin(x),grid)
    # dUdx = functionOnGrid(x->cos(x),grid) #known derivative of sin(x)
    # derivative = Deriv1D(u,grid)

    #L 2 norm: sum of all the differences between the numerical derivative and
    #the analytical one squared and then take the square root
    error = sqrt(sum((dy[2:end-1] - cosx[2:end-1]).^2))

    h = dX(grid)
    #append h to the array H and error to the array E
    append!(H,h)
    append!(E,error)

end

#plotting error vs h
loglog(H,E,"r-")

#first order convergence
loglog(H,H,"g--",label="1")

#second order convergence
loglog(H,H.^2,"b--",label="2")
xlabel("h")
ylabel("L 2 Norm")
legend(loc="best",frameon=false)
show()


function nlp3(oct::Bool = false)
    m = JuMP.Model()
    @variable(m, 0 <= x[1:10])
    for i = 1:8
        JuMP.set_lower_bound(x[[1, 2, 4, 6, 7, 8, 9, 10][i]],
                             [1,1,1,85,90,3,1.2,145][i])
    end
    for i = 1:10
        JuMP.set_upper_bound(x[i], [2000, 16000, 120, 5000, 2000,
                                    93, 95, 12, 4, 162][i])
        JuMP.set_start_value(x[i], [1724.90452208, 16000,
                                    98.0900813608, 3049.1211364,
                                    1995.02326433, 90.718089075,
                                    94.2274481766, 10.432474977,
                                    2.59051951438, 149.682344530][i])
    end
    @constraint(m, e1, x[1] - 1.22*x[4] + x[5] == 0)
    @constraint(m, e2, x[9] + 0.222*x[10] == 35.82)
    @constraint(m, e3, 3*x[7] - x[10] == 133)
    if !oct
        @NLconstraint(m, e4, x[7] - 1.098*x[8] + 0.038*(x[8]^2) - 0.325*(x[6] - 89) == 86.35)
        @NLconstraint(m, e5, x[4]*x[9]*x[6] + 1000*x[3]*x[6] - 98000*x[3] == 0)
        @NLconstraint(m, e6, x[2] + x[5] - x[1]*x[8] == 0)
        @NLconstraint(m, e7, 1.12*x[1] + 0.13167*x[8]*x[1] - 0.00667*(x[8]^2)*x[1] - x[4] >= 0)
        @NLobjective(m, Min, 5.04*x[1] + 0.035*x[2] + 10*x[3] + 3.36*x[5] - 0.063*x[4]*x[7])
        return m
    else
        @variable(m, obj)
        @objective(m, Min, obj)
        gm = GlobalModel(model = m, name = "nlp3")
        add_nonlinear_constraint(gm, :(x -> x[7] - 1.098*x[8] + 0.038*(x[8]^2) - 0.325*(x[6] - 89) - 86.35),
                                vars = [x[6], x[7], x[8]], name = "e4", equality=true)
        add_nonlinear_constraint(gm, :(x -> x[4]*x[9]*x[6] + 1000*x[3]*x[6] - 98000*x[3]),
                                vars = [x[3], x[4], x[6], x[9]], name = "e5", equality = true)
        add_nonlinear_constraint(gm, :(x -> x[2] + x[5] - x[1]*x[8]),
                                vars = [x[1], x[2], x[5], x[8]], name = "e6", equality = true)
        add_nonlinear_constraint(gm, :(x -> 1.12*x[1] + 0.13167*x[8]*x[1] - 0.00667*(x[8]^2)*x[1] - x[4]),
                                vars = [x[1], x[4], x[8]], name = "e7")
        add_nonlinear_constraint(gm, :(x -> 5.04*x[1] + 0.035*x[2] + 10*x[3] + 3.36*x[5] - 0.063*x[4]*x[7]),
                                vars = [x[1], x[2], x[3], x[4], x[5], x[7]],
                                dependent_var = obj, name = "obj")
        return gm
    end
end


### A Pluto.jl notebook ###
# v0.15.1

using Markdown
using InteractiveUtils

# ╔═╡ 1e037376-8252-41fb-ae95-93bc9011b7fa
# we need to install a special package in order to use the gamma function
using SpecialFunctions

# ╔═╡ 76d6c2d4-e370-4e83-9d04-ca6fc546bdd9
# we write a function that returns the exact volume of a hypersphere in d dimensions
function exactVol(d)
    vol = pi^(d/2) / gamma(1 + d/2)
    return vol
end

# ╔═╡ b58cf6d8-291c-49f4-a61a-51f1a7454651
n = 10000;

# ╔═╡ 2f4a8b5e-f7c9-4abc-b04d-3286207a97b3
dim = 20;

# ╔═╡ 4da795ef-b8d3-4c94-bb89-7eeaca9602fb
md"Exact volume is:"

# ╔═╡ 8d8e10d0-2597-4198-92b1-9ab2ebe4b3c8
exV = exactVol(dim)

# ╔═╡ 6a8c6c37-2a48-47c7-8d40-ca276c8a09d2
# we write a function that returns a random vector from [0,1]^d with exponential density
function randExp(d, lam)

	# we generate a uniformly random vector
	y = rand(d)

	# we transform (elementwise) to random numbers in [exp(-lambda),1]
	y = exp(-lam) .+ (1-exp(-lam))*y

	# we now get numbers with exponential density
	x = -log.(y)/lam
	return x
end

# ╔═╡ 8de1cad7-ac2c-42ec-9581-1d7ff06d576d
# we write an improoved function that uses Monte Carlo with exponential importance sampling
# to estimate the volume of the unit hypersphere in d dimensions.
# We choose our points x with independent Cartesian components and prob density exp(-lam.xi) / [ lam*((1-exp(-lam)) ].
# We achieve this by taking xi = (1/lam) log (1/y) where y is chosen uniformly in (exp(-lam),1)
function sphereVolMCImpExp(d, n, lam)
	count = 0.0
	for i = 1:n

		# we generate x according to our distribution
		x = randExp(d, lam)

		# we calculate the squared norm of x and check if x is inside the hypersphere
		norm2 = x' * x
		if norm2 < 1

			# we compute the density at x
			rho = 1
			for i = 1:d

				# note that y(i) = exp(-lam * x(i))
				rho *= exp(-lam*x[i]) * lam / (1-exp(-lam))
			end
			count += 1/rho
		end
	end
	vol =  2^d * count / n
	return vol
end

# ╔═╡ c8381d29-3b9f-4399-9a60-c0b6b620e99c
sphereVolMCImpExp(20, n, 2)

# ╔═╡ 0e1580e1-cdae-498e-ab14-261f8a96a802
md" We do the calculation $m$ times and check mean/variance:"

# ╔═╡ 770e2019-420d-415b-b498-96493ac676bf
m = 1000;

# ╔═╡ dc6e6729-7ce5-45a8-8ecc-b320d283340a
data = [sphereVolMCImpExp(20, n, 2) for i in 1:m];

# ╔═╡ fb6c8ff3-d200-4e48-b9ad-aa34d188e675
mean = sum(data)/m

# ╔═╡ 48b51297-b3fb-4bce-8d41-7af6120b553c
error = mean - exV

# ╔═╡ bb18d766-cdad-45b7-aebb-c59763896bd4
variance = sum(broadcast(x -> (x-mean)^2, data))/m

# ╔═╡ 00000000-0000-0000-0000-000000000001
PLUTO_PROJECT_TOML_CONTENTS = """
[deps]
SpecialFunctions = "276daf66-3868-5448-9aa4-cd146d93841b"

[compat]
SpecialFunctions = "~1.6.1"
"""

# ╔═╡ 00000000-0000-0000-0000-000000000002
PLUTO_MANIFEST_TOML_CONTENTS = """
# This file is machine-generated - editing it directly is not advised

[[ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"

[[Artifacts]]
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"

[[Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[ChainRulesCore]]
deps = ["Compat", "LinearAlgebra", "SparseArrays"]
git-tree-sha1 = "bdc0937269321858ab2a4f288486cb258b9a0af7"
uuid = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
version = "1.3.0"

[[Compat]]
deps = ["Base64", "Dates", "DelimitedFiles", "Distributed", "InteractiveUtils", "LibGit2", "Libdl", "LinearAlgebra", "Markdown", "Mmap", "Pkg", "Printf", "REPL", "Random", "SHA", "Serialization", "SharedArrays", "Sockets", "SparseArrays", "Statistics", "Test", "UUIDs", "Unicode"]
git-tree-sha1 = "727e463cfebd0c7b999bbf3e9e7e16f254b94193"
uuid = "34da2185-b29b-5c13-b0c7-acf172513d20"
version = "3.34.0"

[[CompilerSupportLibraries_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"

[[Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[DelimitedFiles]]
deps = ["Mmap"]
uuid = "8bb1440f-4735-579b-a4ab-409b98df4dab"

[[Distributed]]
deps = ["Random", "Serialization", "Sockets"]
uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b"

[[DocStringExtensions]]
deps = ["LibGit2"]
git-tree-sha1 = "a32185f5428d3986f47c2ab78b1f216d5e6cc96f"
uuid = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae"
version = "0.8.5"

[[Downloads]]
deps = ["ArgTools", "LibCURL", "NetworkOptions"]
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"

[[InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[IrrationalConstants]]
git-tree-sha1 = "f76424439413893a832026ca355fe273e93bce94"
uuid = "92d709cd-6900-40b7-9082-c6be49f344b6"
version = "0.1.0"

[[JLLWrappers]]
deps = ["Preferences"]
git-tree-sha1 = "642a199af8b68253517b80bd3bfd17eb4e84df6e"
uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210"
version = "1.3.0"

[[LibCURL]]
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"

[[LibCURL_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"

[[LibGit2]]
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"

[[LibSSH2_jll]]
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"

[[Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[LinearAlgebra]]
deps = ["Libdl"]
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[[LogExpFunctions]]
deps = ["DocStringExtensions", "IrrationalConstants", "LinearAlgebra"]
git-tree-sha1 = "3d682c07e6dd250ed082f883dc88aee7996bf2cc"
uuid = "2ab3a3ac-af41-5b50-aa03-7779005ae688"
version = "0.3.0"

[[Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[MbedTLS_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"

[[Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"

[[MozillaCACerts_jll]]
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"

[[NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"

[[OpenSpecFun_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "13652491f6856acfd2db29360e1bbcd4565d04f1"
uuid = "efe28fd5-8261-553b-a9e1-b2916fc3738e"
version = "0.5.5+0"

[[Pkg]]
deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"

[[Preferences]]
deps = ["TOML"]
git-tree-sha1 = "00cfd92944ca9c760982747e9a1d0d5d86ab1e5a"
uuid = "21216c6a-2e73-6563-6e65-726566657250"
version = "1.2.2"

[[Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[[Random]]
deps = ["Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"

[[Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[SharedArrays]]
deps = ["Distributed", "Mmap", "Random", "Serialization"]
uuid = "1a1011a3-84de-559e-8e89-a11a2f7dc383"

[[Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"

[[SparseArrays]]
deps = ["LinearAlgebra", "Random"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"

[[SpecialFunctions]]
deps = ["ChainRulesCore", "LogExpFunctions", "OpenSpecFun_jll"]
git-tree-sha1 = "a322a9493e49c5f3a10b50df3aedaf1cdb3244b7"
uuid = "276daf66-3868-5448-9aa4-cd146d93841b"
version = "1.6.1"

[[Statistics]]
deps = ["LinearAlgebra", "SparseArrays"]
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[[TOML]]
deps = ["Dates"]
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"

[[Tar]]
deps = ["ArgTools", "SHA"]
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"

[[Test]]
deps = ["InteractiveUtils", "Logging", "Random", "Serialization"]
uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[[UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[[Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"

[[Zlib_jll]]
deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"

[[nghttp2_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"

[[p7zip_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"
"""

# ╔═╡ Cell order:
# ╠═1e037376-8252-41fb-ae95-93bc9011b7fa
# ╠═76d6c2d4-e370-4e83-9d04-ca6fc546bdd9
# ╠═b58cf6d8-291c-49f4-a61a-51f1a7454651
# ╠═2f4a8b5e-f7c9-4abc-b04d-3286207a97b3
# ╟─4da795ef-b8d3-4c94-bb89-7eeaca9602fb
# ╠═8d8e10d0-2597-4198-92b1-9ab2ebe4b3c8
# ╠═6a8c6c37-2a48-47c7-8d40-ca276c8a09d2
# ╠═8de1cad7-ac2c-42ec-9581-1d7ff06d576d
# ╠═c8381d29-3b9f-4399-9a60-c0b6b620e99c
# ╟─0e1580e1-cdae-498e-ab14-261f8a96a802
# ╠═770e2019-420d-415b-b498-96493ac676bf
# ╠═dc6e6729-7ce5-45a8-8ecc-b320d283340a
# ╠═fb6c8ff3-d200-4e48-b9ad-aa34d188e675
# ╠═48b51297-b3fb-4bce-8d41-7af6120b553c
# ╠═bb18d766-cdad-45b7-aebb-c59763896bd4
# ╟─00000000-0000-0000-0000-000000000001
# ╟─00000000-0000-0000-0000-000000000002


#=
This file is part of Pseudospectra.jl.

Julia translation
copyright 2017 Ralph Smith

Portions derived from EigTool
Copyright (c) 2002-2014, The Chancellor, Masters and Scholars
of the University of Oxford, and the EigTool Developers. All rights reserved.

SPDX-License-Identifier: BSD-3-Clause
License-Filename: LICENSES/BSD-3-Clause_Eigtool
=#

"""
    numerical_range(A, nstep=20) -> Vector{Complex}

Compute points along the numerical range of a matrix.

Note: this solves an eigensystem for each point, so may be expensive.
"""
function numerical_range(A::AbstractMatrix, thmax=20)
    rayleighquotient(B,x) = dot(x, B * x) / dot(x,x)
    # m,n = size(A)
    T = eltype(A)
    CT = (T <: Real) ? Complex{T} : T
    z = zeros(CT,2*thmax+2)
    for i = 0:thmax
        # upstream allows for interactive cancellation

        # get Hermitian part of rotated A
        th = (i/thmax)*π
        Ath = exp(th*1im)*A
        H = (1/2)*(Ath + Ath')
        F = eigen(H)
        d,X = F.values,F.vectors

        # RQ's of A correspond to eigenvalues of H w/ extreme real parts
        k = sortperm(real(d))
        z[i+1] = rayleighquotient(A,X[:,k[1]])
        z[1+i+thmax] = rayleighquotient(A,X[:,k[end]])
    end
    z[end] = z[1] # close curve for plotting
    z
end

"""
    numrange!(ps_data,nstep=20)

compute the numerical range (a.k.a. field of values) of a dense matrix
then store it in the `ps_data`.

Note: this solves an eigensystem for each point, so may be expensive.
"""
function numrange!(ps_data::PSAStruct,thmax=20)
    ps_dict = ps_data.ps_dict
    # don't recompute if no change
    if !haskey(ps_dict,:fov) || (length(ps_dict[:fov]) != 2*(thmax+1))
        # Why does upstream enforce this constraint?
        if !haskey(ps_dict,:schur_mtx)
            throw(ArgumentError("only implemented for Schur-factored matrices"))
        end
        z = numerical_range(ps_dict[:schur_mtx],thmax)
        ps_dict[:fov] = z
    else
        z = ps_dict[:fov]
    end

    zoom = ps_data.zoom_list[ps_data.zoom_pos]
    ax = zoom.ax
    if mapreduce(w -> ((real(w) < ax[1]) | (real(w) > ax[2]) |
                       (imag(w) < ax[3]) | (imag(w) > ax[4])),
                 &, z, init=true)
        warn("The boundary of the numerical range is not visible on the "
             * "current axes; expand axis limits to see it. "
             * "A bounding box is $(extrema(real(z))) $(extrema(imag(z))).")
    end
end

"""
    numerical_abscissa(A)

Compute the numerical abscissa of a matrix `A`, `ω(A)`.

Uses `eigvals()`. `ω(A)` provides bounds and limiting behavior for
`norm(expm(t*A))`.
"""
function numerical_abscissa(A::AbstractMatrix)
    (1/2) * maximum(eigvals(A+A'))
end


# p28.jl - eigenmodes of Laplacian on the disk (compare p22.jl)

# r coordinate, ranging from -1 to 1 (N must be odd):
N = 25; N2 = Int((N-1)/2);
(D,r) = cheb(N); D2 = D^2;
D1 = D2[2:N2+1,2:N2+1]; D2 = D2[2:N2+1,N:-1:N2+2];
E1 =  D[2:N2+1,2:N2+1]; E2 =  D[2:N2+1,N:-1:N2+2];

# t = theta coordinate, ranging from 0 to 2*pi (M must be even):
M = 20; dt = 2*pi/M; t = dt*(1:M); M2 = Int(M/2);
D2t = toeplitz([-pi^2/(3*dt^2)-1/6; @. .5*(-1)^(2:M)/sin(dt*(1:M-1)/2)^2]);

# Laplacian in polar coordinates:
R = diagm(1./r[2:N2+1]);
Z = zeros(M2,M2); I = eye(M2);
L = kron(D1+R*E1,eye(M)) + kron(D2+R*E2,[Z I;I Z]) + kron(R^2,D2t);

# Compute four eigenmodes:
index = [1 3 6 10];
(Lam,V) = eig(-L); ii = sortperm(abs.(Lam))[index];
Lam = Lam[ii]; V = V[:,ii];
Lam = sqrt.(real(Lam/Lam[1]));

# Plot eigenmodes with nodal lines underneath:
(rr,tt) = (r[1:N2+1],[0;t]);
(xx,yy) = @. (cos(tt)*rr',sin(tt)*rr');
z = exp.(1im*pi*(-100:100)/100);
for i = 1:4
    figure(i); clf();
    u = reshape(real(V[:,i]),M,N2);
    u = [zeros(M+1) u[[M;1:M],:]];
    u = u/norm(u[:],Inf);
    #plot3D(real(z),imag(z),zeros(size(z)));
    xlim(-1.05,1.05); ylim(-1.05,1.05); zlim(-1.05,1.05); axis("off");
    surf(xx,yy,u)
    gca()[:view_init](20,90); axis("square");
    contour3D(xx,yy,u-1,levels=[-1]);
    plot3D(real(z),imag(z),-abs.(z));
    title("Mode $(index[i]):  λ = $(signif(Lam[i],11))",fontsize=9);
end


#=
A builder is looking to build a row of N houses that can be of K different colors. He has a goal of minimizing cost while ensuring that no two neighboring houses are of the same color.

Given an N by K matrix where the nth row and kth column represents the cost to build the nth house with kth color, return the minimum cost which achieves this goal.
=#

function minimum_building_cost(cost_matrix)
    # Create a matrix with the cumulative minimum cost of building the nth house (row) with the kth color (column)
    cumulative_minimum_cost = zeros(Float64, size(cost_matrix))
    # For the first house, it is the first row of the cost_matrix
    cumulative_minimum_cost[1,:] = cost_matrix[1,:]
    number_of_colors = size(cost_matrix, 2)
    number_of_houses = size(cost_matrix, 1)
    # For the second to last houst, for each color, the minimum cumulative cost is the price of building the house in that color + the minimum cumulative cost of building the last house in a different color
    for house in 2:number_of_houses
        for color in 1:number_of_colors
            allowed_prev_colors = setdiff(1:number_of_colors, color)
            cumulative_minimum_cost[house, color] = cost_matrix[house, color] + minimum(cumulative_minimum_cost[house-1,allowed_prev_colors])
        end
    end
    # We want the minimum cost of building all the house
    minimum_cost = minimum(cumulative_minimum_cost[end,:])
    return minimum_cost
end


module StockCorr

if VERSION >= v"0.7.0-DEV.3449"
    using LinearAlgebra
end

# The original, unoptimised code that simulates two correlated assets
function perf_stockcorr()
    ## Correlated asset information
    CurrentPrice = [78. 102.] # Initial Prices of the two stocks
    Corr = [1. 0.4; 0.4 1.]   # Correlation Matrix
    T = 500                   # Number of days to simulate = 2years = 500days
    n = 10000                 # Number of simulations
    dt = 1/250                # Time step (1year = 250days)
    Div = [0.01 0.01]         # Dividend
    Vol = [0.2 0.3]           # Volatility

    ## Market Information
    r = 0.03 # Risk-free rate

    ## Define storages
    SimulPriceA = zeros(T,n) # Simulated Price of Asset A
    SimulPriceA[1,:] .= CurrentPrice[1]
    SimulPriceB = zeros(T,n) # Simulated Price of Asset B
    SimulPriceB[1,:] .= CurrentPrice[2]

    ## Generating the paths of stock prices by Geometric Brownian Motion
    @static if VERSION <= v"0.7.0-DEV.5211"
        UpperTriangle = chol(Corr) # UpperTriangle Matrix by Cholesky decomposition
    else
        UpperTriangle = cholesky(Corr).U
    end

    for i = 1:n
       Wiener = randn(T-1,2)
       CorrWiener = Wiener*UpperTriangle
       for j = 2:T
          SimulPriceA[j,i] = SimulPriceA[j-1,i]*exp((r-Div[1]-Vol[1]^2/2)*dt+Vol[1]*sqrt(dt)*CorrWiener[j-1,1])
          SimulPriceB[j,i] = SimulPriceB[j-1,i]*exp((r-Div[2]-Vol[2]^2/2)*dt+Vol[2]*sqrt(dt)*CorrWiener[j-1,2])
       end
    end

    return (SimulPriceA, SimulPriceB)
end

end # module


function factorial(n::Int64)
    if n == 0 || n == 1
        return 1
    else
        return n * factorial(n-1)
    end
end
n = parse(Int64, readline())
println(factorial(n))

Hackerrank/30 Days of Code/Julia/day 09.jl


# This file is a part of BAT.jl, licensed under the MIT License (MIT).

# simple 2d convolution with padding
function convolution(input, filter; padding=:same)
    input_r, input_c = size(input)
    filter_r, filter_c = size(filter)

    if padding == :same
        pad_r = (filter_r - 1) ÷ 2
        pad_c = (filter_c - 1) ÷ 2

        input_padded = zeros(input_r+(2*pad_r), input_c+(2*pad_c))
        for i in 1:input_r, j in 1:input_c
            input_padded[i+pad_r, j+pad_c] = input[i, j]
        end
        input = input_padded
        input_r, input_c = size(input)
    end

    result = zeros(input_r-filter_r+1, input_c-filter_c+1)
    result_r, result_c = size(result)

    for i in 1:result_r
        for j in 1:result_c
            for k in 1:filter_r
                for l in 1:filter_c
                    result[i,j] += input[i+k-1,j+l-1]*filter[k,l]
                end
            end
        end
    end

    return result
end

# gaussian kernel with same σ in both dimensions
function gaussian_kernel(σ::Real; l::Int = 4*ceil(Int,σ)+1)
    isodd(l) || throw(ArgumentError("length must be odd"))
    w = l>>1
    g = σ == 0 ? [exp(0/(2*oftype(σ, 1)^2))] : [exp(-x^2/(2*σ^2)) for x=-w:w]
    k = g/sum(g)
    return (k * k')
end

# gaussian kernel with different σs in both dimensions
function gaussian_kernel(
    σs::Tuple{Real, Real};
    l::Tuple{Int, Int} = (4*ceil(Int,σs[1])+1, 4*ceil(Int,σs[2])+1)
)
    all(isodd.(l)) || throw(ArgumentError("length must be odd"))
    w1 = l[1]>>1
    g1 = σs[1] == 0 ? [exp(0/(2*oftype(σs[1], 1)^2))] : [exp(-x^2/(2*σs[1]^2)) for x=-w1:w1]
    k1 = g1/sum(g1)

    w2 = l[2]>>1
    g2 = σs[2] == 0 ? [exp(0/(2*oftype(σs[2], 1)^2))] : [exp(-x^2/(2*σs[2]^2)) for x=-w2:w2]
    k2 = g2/sum(g2)

    return (k1 * k2')
end


#Constant mean function

"""
    MeanConst <: Mean

Constant mean function
```math
m(x) = β
```
with constant ``β``.
"""
mutable struct MeanConst <: Mean
    "Constant"
    β::Float64
    "Priors for mean parameters"
    priors::Array

    """
        MeanConst(β::Float64)

    Create `MeanConst` with constant `β`.
    """
    MeanConst(β::Float64) = new(β, [])
end

mean(mConst::MeanConst, x::AbstractVector) = mConst.β
mean(mConst::MeanConst, X::AbstractMatrix) = fill(mConst.β, size(X,2))

get_params(mConst::MeanConst) = Float64[mConst.β]
get_param_names(::MeanConst) = [:β]
num_params(mConst::MeanConst) = 1
function set_params!(mConst::MeanConst, hyp::AbstractVector)
    length(hyp) == 1 || throw(ArgumentError("Constant mean function only has 1 parameter"))
    mConst.β = hyp[1]
end
function grad_mean(mConst::MeanConst, x::AbstractVector)
    dM_theta = ones(1)
    return dM_theta
end


push!( LOAD_PATH, "../" )
import InformationInequalities  # Import the module to  have access to functions

"""
List down all conditional Entropy expressions for a given number `n` of random variables. Conditional Entropies are of the form `H(X,Y|Z)`
```julia-repl
julia> ConditionalEntropyList(2,"🍉")
["H(🍉1)" "H(🍉1|🍉2)" "H(🍉2)" "H(🍉2|🍉1)"]

julia> ConditionalEntropyList(2)
["H(X1)" "H(X1|X2)" "H(X2)" "H(X2|X1)"]

julia> ConditionalEntropyList(3,"dice💠")
["H(dice💠1)"
"H(dice💠1|dice💠2)"
"H(dice💠1|dice💠2,dice💠3)"
"H(dice💠1|dice💠3)"
"H(dice💠2)"
"H(dice💠2|dice💠1)"
"H(dice💠2|dice💠1,dice💠3)"
"H(dice💠2|dice💠3)"
"H(dice💠3)"
"H(dice💠3|dice💠1)"
"H(dice💠3|dice💠1,dice💠2)"
"H(dice💠3|dice💠2)"]

julia> ConditionalEntropy(3,"Z")
["H(Z1)"
"H(Z1|Z2)"
"H(Z1|Z2,Z3)"
"H(Z1|Z3)"
"H(Z2)"
"H(Z2|Z1)"
"H(Z2|Z1,Z3)"
"H(Z2|Z3)"
"H(Z3)"
"H(Z3|Z1)"
"H(Z3|Z1,Z2)"
"H(Z3|Z2)"]
```
"""
function ConditionalEntropyList(m::Int64=2,RV::AbstractString="X")
	y=  [("$(RV)" .* string(i) *  "|" * find_subset(m,i,[],RV)[k]) for i=1:m for k=1:Int64(exp2(m-1)) ]
	y0=replace.(y,"|" => "| ")
	y1=replace.(y0,"| $(RV)" => "|$(RV)")
	y2=replace.(y1,"| " => "")
	yy = "H(" .* y2 .* ")"
	return yy[:]
end


import StatsBase: sample

# Model selection

# Taken from https://royalsocietypublishing.org/doi/pdf/10.1098/rspa.2017.0009
"""
	$(SIGNATURES)

Computes the Akaike Information Criterion (AIC) given the free parameters `k` for the data `X` and its
estimate `Y` of the model. `likelihood` can be any function of `X` and `Y`.
"""
function AIC(k::Int64, X::AbstractArray, Y::AbstractArray; likelihood = (X,Y) -> sum(abs2, X-Y))
    @assert size(X) == size(Y) "Dimensions of trajectories should be equal !"
    return 2*k - 2*log(likelihood(X, Y))
end

# Taken from https://royalsocietypublishing.org/doi/pdf/10.1098/rspa.2017.0009
"""
	$(SIGNATURES)

Computes the Akaike Information Criterion compensated for finite samples (AICC) given the free parameters `k` for the data `X` and its
estimate `Y` of the model. `likelihood` can be any function of `X` and `Y`.
"""
function AICC(k::Int64, X::AbstractMatrix, Y::AbstractMatrix; likelihood = (X,Y) -> sum(abs2, X-Y))
    @assert size(X) == size(Y) "Dimensions of trajectories should be equal !"
    return AIC(k, X, Y, likelihood = likelihood)+ 2*(k+1)*(k+2)/(size(X)[2]-k-2)
end

function AICC(k::Int64, X::AbstractVector, Y::AbstractVector; likelihood = (X,Y) -> sum(abs2, X-Y))
    @assert size(X) == size(Y) "Dimensions of trajectories should be equal !"
    return AIC(k, X, Y, likelihood = likelihood)+ 2*(k+1)*(k+2)/(length(X)-k-2)
end

# Double check on that
# Taken from https://www.immagic.com/eLibrary/ARCHIVES/GENERAL/WIKIPEDI/W120607B.pdf
"""
	$(SIGNATURES)

Computes Bayes Information Criterion (BIC) given the free parameters `k` for the data `X` and its
estimate `Y` of the model. `likelihood` can be any function of `X` and `Y`.
"""
function BIC(k::Int64, X::AbstractMatrix, Y::AbstractMatrix; likelihood = (X,Y) -> sum(abs2, X-Y))
    @assert size(X) == size(Y) "Dimensions of trajectories should be equal !"
    return - 2*log(likelihood(X, Y)) + k*log(size(X)[2])
end

function BIC(k::Int64, X::AbstractVector, Y::AbstractVector; likelihood = (X,Y) -> sum(abs2, X-Y))
    @assert size(X) == size(Y) "Dimensions of trajectories should be equal !"
    return - 2*log(likelihood(X, Y)) + k*log(length(X))
end

# Optimal Shrinkage for data in presence of white noise
# See D. L. Donoho and M. Gavish, "The Optimal Hard Threshold for Singular
# Values is 4/sqrt(3)", http://arxiv.org/abs/1305.5870
# Code taken from https://github.com/erichson/optht

function optimal_svht(m::Int64, n::Int64; known_noise::Bool = false)
    @assert m/n > 0
    @assert m/n <= 1

    β = m/n
    ω = (8*β) / (β+1+sqrt(β^2+14β+1))
    c = sqrt(2*(β+1)+ω)

    if known_noise
        return c
    else
        median = median_marcenko_pastur(β)
        return c / sqrt(median)
    end
end

function marcenko_pastur_density(t, lower, upper, beta)
    sqrt((upper-t).*(t-lower))./(2π*beta*t)
end

function incremental_marcenko_pastur(x, beta, gamma)
    @assert beta <= 1
    upper = (1+sqrt(beta))^2
    lower = (1-sqrt(beta))^2

    @inline marcenko_pastur(x) = begin
        if (upper-x)*(x-lower) > 0
            return marcenko_pastur_density(x, lower, upper, beta)
        else
            return zero(eltype(x))
        end
    end

    if gamma ≈ zero(eltype(gamma))
        i, ϵ = quadgk(x->(x^gamma)*marcenko_pastur(x), x, upper)
        return i
    else
        i, ϵ = quadgk(x->marcenko_pastur(x), x, upper)
        return i
    end
end

function median_marcenko_pastur(beta)
    @assert 0 < beta <= 1
    upper = (1+sqrt(beta))^2
    lower = (1-sqrt(beta))^2
    change = true
    x = ones(eltype(upper), 5)
    y = similar(x)
    while change && (upper - lower > 1e-5)
        x = range(lower, upper, length = 5)
        for (i,xi) in enumerate(x)
            y[i] = one(eltype(x)) - incremental_marcenko_pastur(xi, beta, 0)
        end
        any(y .< 0.5) ? lower = maximum(x[y .< 0.5]) : change = false
        any(y .> 0.5) ? upper = minimum(x[y .> 0.5]) : change = false
    end
    return (lower+upper)/2
end

"""
    $(SIGNATURES)

Compute a feature reduced version of the data array `X` via thresholding the
singular values by computing the [optimal threshold for singular values](http://arxiv.org/abs/1305.5870).
"""
function optimal_shrinkage(X::AbstractArray{T, 2}) where T <: Number
    m,n = minimum(size(X)), maximum(size(X))
    U, S, V = svd(X)
    τ = optimal_svht(m,n)
    inds = S .>= τ*median(S)
    return U[:, inds]*Diagonal(S[inds])*V[:, inds]'
end

"""
    $(SIGNATURES)

Compute a feature reduced version of the data array `X` inplace via thresholding the
singular values by computing the [optimal threshold for singular values](http://arxiv.org/abs/1305.5870).
"""
function optimal_shrinkage!(X::AbstractArray{T, 2}) where T <: Number
    m,n = minimum(size(X)), maximum(size(X))
    U, S, V = svd(X)
    τ = optimal_svht(m,n)
    inds = S .>= τ*median(S)
    X .= U[:, inds]*Diagonal(S[inds])*V[:, inds]'
    return
end


# Comparison of high-level programming languages
# This is Julia (version 1.3.1)

# Function
function power(x, a)
  x^a
end
three_squared = power(3, 2)

# Anonymous function
power2 = (x, a) ->  x^a
@assert(power2(3, 2) == power(3, 2))

# Assignment form
power3(x, a) = x^a
@assert(power3(3, 2) == power(3, 2))

# Variable assignment
x = power(2, 0.5)

# Standard output
println(x)

# Formatted output
using Printf
@printf("%5.2f\n", x)

# Definite loop
for i in 1:5
  println(i)
end

# Indefinite loop
i = 1
while i <= 5
  println(i)
  global i += 1
end

# List
elements = ["H", "He", "Li"]  # Array
println(elements)

# Dictionary
elements = Dict(
    "H" => 1,
    "He" => 2,
    "Li" => 3
)
println(elements)

# Iterating over collections
for (symbol, n) in elements
  @printf("%s: %d\n", symbol, n)
end

# Array literals

# Vector (1d)
x = [1, 2, 3]
println(x)

# Matrix (2d)
A = [1 2; 3 4; 5 6]
println(A)

# 3d array (3x2x3)
Z = ones(3, 2, 3)
println(Z)

# Array dimensions
println(size(x))
println(size(A))
println(size(Z))

# Indexing arrays
println(x[2])
println(A[2,2])
println(A[2])

# Slicing arrays
println(x[2:end])
println(A[:,2])

# Concatenation of vector and 2-d array
C = [x A]
println(C)

# Array broadcasting
println(1 .- A)
println(A .+ x)

# Linear algebra
using LinearAlgebra
A = [0.8 0; 0 1]
K = [0.942; 1.074]
C = [0.2 1]
X = (I - A + K*C)^-1
println(X)

# Symbolic math
y = :(a*x^2 + b*x + c)  # Symbolic expressions are built in
using Reduce
dydx = Algebra.df(y,:x)
x_sol = Algebra.solve(dydx,:x)
y_sol = Algebra.sub(x_sol,y)
println(dydx)
println(x_sol)
println(y_sol)

# Non-vectorized method
function lorenz(y1, y2, y3, sigma::Float64,
                beta::Float64, rho::Float64)
  return (sigma * (y2 - y1), y1 * (rho - y3) - y2,
          y1 * y2 - beta * y3)
end
lorenz(y::Tuple{Float64,Float64,Float64},
       sigma::Float64, beta::Float64,
       rho::Float64) = (
         lorenz(y[1], y[2], y[3], sigma, beta, rho)
       )

# Vectorized method 1
# function lorenz(y1::Array, y2::Array, y3::Array,
#                 sigma::Float64, beta::Float64,
#                 rho::Float64)
#   dy1 = sigma.*(y2.-y1)
#   dy2 = y1.*(rho.-y3).-y2
#   dy3 = y1.*y2.-beta.*y3
#   return [dy1 dy2 dy3]'
# end
# lorenz(y::Array, sigma::Float64, beta::Float64,
#        rho::Float64) = (
#          lorenz(y[1,:], y[2,:], y[3,:], sigma, beta, rho)
#        )
# Vectorized method 2
# function lorenz(y::Array, sigma::Float64, beta::Float64,
#   rho::Float64)
# dy = similar(y)
# dy[1,:] = sigma.*(y[2,:].-y[1,:])
# dy[2,:] = y[1,:].*(rho.-y[3,:]).-y[2,:]
# dy[3,:] = y[1,:].*y[2,:].-beta.*y[3,:]
# return dy
# end
# Vectorized method 3
function lorenz(
  y::Tuple{Array{Float64,1},Array{Float64,1},Array{Float64,1}},
  sigma::Float64, beta::Float64, rho::Float64
  )
  return (sigma.*(y[2].-y[1]), y[1].*(rho.-y[3]).-y[2],
          y[1].*y[2].-beta.*y[3])
end
# This is not faster than method 3 above:
function lorenz(y::Array{Float64,2}, sigma::Float64, beta::Float64,
                rho::Float64)
  return (sigma.*(y[2,:].-y[1,:]), y[1,:].*(rho.-y[3,:]).-y[2,:],
          y[1,:].*y[2,:].-beta.*y[3,:])
end

# Parameter values
beta = 8 / 3
sigma = 10.0
rho = 28.0
y0 = (-8.0, 8.0, 27.0)
dy = lorenz(y0, sigma, beta, rho)
println(dy)
@assert dy == (160.0, -16.0, -136.0)

# Test - vectorized method 2
# y = repeat([-8.0, 8.0, 27.0], 1, 10)
# dy = lorenz(y, sigma, beta, rho)
# @assert size(dy) == size(y)
# @assert dy == repeat([160.0,  -16.0, -136.0], 1, 10)

# Test - vectorized method 3
y = (repeat([-8.0],10), repeat([8.0],10), repeat([27.0],10))
dy = lorenz(y, sigma, beta, rho)
@assert length(dy) == length(y)

y = randn(3,10000)
y = (y[1,:], y[2,:], y[3,:])
@time dy = lorenz(y, sigma, beta, rho)

# Speed Tests
# Single function call - 0.000006 seconds
# Vectorized function call method 1 - 0.000404 seconds
# Vectorized function call method 2 - 0.001036 seconds
# Vectorized function call method 3 - 0.000187 seconds
# Note: Why is this 7 times slower than Python?
# (Using: @time dy = lorenz(y, sigma, beta, rho);)


export line, sigmoid, hardsigmoid, relu, elu, selu, tanh, hardtanh, softsign, softplus

function line(x)
    return x
end

line(x, back::Bool) = 1

function sigmoid(x)
    t = exp(-abs(x))
    return (x>=0) ? inv(1 + t) : t / (t + 1)
end

sigmoid(x, back::Bool) = sigmoid(x)*(1-sigmoid(x))

hardsigmoid(x) = max(0, min(1, (x + 2.5) / 6))

function hardsigmoid(x, back::Bool)
    if -2.5 <= x <= 2.5
        return 0.2
    else
        return 0
    end
end

relu(x) = max(0, x)

function relu(x, back::Bool)
    if x <= 0
        return 0
    else
        return 1
    end
end

function elu(x; alpha = 1)
    return (x > 0) ? x : (exp(x) - 1)alpha
end

function elu(x, back::Bool; alpha = 1)
    if x < 0
        return alpha*exp(x)
    else
        return 1
    end
end

function selu(x)
    λ = oftype(float(x), selu_λ)
    α = oftype(float(x), selu_α)
    return ((x > 0) ? x : (exp(x)-1)α)*λ
end

function selu(x, back::Bool)
    λ = oftype(float(x), selu_λ)
    α = oftype(float(x), selu_α)
    if x < 0
        return λ*alpha*exp(x)
    else
        return λ
    end
end

const selu_λ = 1.0507009873554804934193349852946
const selu_α = 1.6732632423543772848170429916717

function tanh(x)
    return exp(x) - exp(-x) / exp(x) + exp(-x)
end

tanh(x, back::Bool) = 4 / (exp(x) + exp(-x)) ^ 2

function hardtanh(x)
    if x > 1
        return 1
    elseif x < -1
        return -1
    else
        return x
    end
end

function hardtanh(x, back::Bool)
    if x < -1 || 1 <= x
        return 0
    else
        return 1
    end
end

function softsign(x)
    return x / (1 + abs(x))
end

softsign(x, back::Bool) = 1 / ((1 + abs(x))^2)

function softplus(x)
    return log(1 + exp(x))
end

softplus(x, back::Bool) = 1 / (1 + exp(-x))


"""
Determine the number of meters per pixel

### Parameters
* `lat`: latitude in radians
* `z`: zoom level

Source: http://wiki.openstreetmap.org/wiki/Slippy_map_tilenames#Resolution_and_Scale
"""
function resolution(lat::Real, z::Integer)
    meter_per_pixel = 156543.03 # For zoom = 0 at equator
    meter_per_pixel * cos(lat) / (2^z)
end

"Convert from `lat,lon` to tile numbers (float64)"
function lonlat2tile(lon::Real, lat::Real, z::Integer)
    latradian = deg2rad(lat); n = 2^z
    x = (lon + 180)/360*n
    y = (1 - log(tan(latradian) + 1/cos(latradian))/pi)/2*n
    x, y
end

"Convert from `lat,lon` to tile numbers, rounding them down to integers"
function ilonlat2tile(lon::Real, lat::Real, z::Integer)
    x, y = lonlat2tile(lon, lat, z)
    floor(Int, x), floor(Int, y)
end

"Convert geographical coordinates to tile coordinates"
function lonlat2tile(
        lon::Real, lat::Real, z::Integer, xmin::Real, ymin::Real,
        tilesizex::Integer, tilesizey::Integer
    )
    x, y = lonlat2tile(lon, lat, z)
    floor(Int,(x-xmin)*tilesizex)+1, floor(Int,(y-ymin)*tilesizey)+1
end

lonlat2tile(basemap::BaseMap, lon::Real, lat::Real) = lonlat2tile(
    lon, lat, basemap.zoom, basemap.xmin, basemap.ymin,
    basemap.tilesize[2], basemap.tilesize[1]
)

"""
Convert a box from geographical to tile coordinates (integers), at a given zoom.
"""
function tilebox(
        minlon::Real,
        minlat::Real,
        maxlon::Real,
        maxlat::Real,
        z::Integer
    )
    xmin, ymin = ilonlat2tile(minlon, minlat, z)
    xmax, ymax = ilonlat2tile(maxlon, maxlat, z)
    xmin, ymin, xmax, ymax
end

"Determine the correct tiles for `(xmin, ymin, xmax, ymax)` at zoom `z`"
function correctbox(
        xmin::Integer,
        ymin::Integer,
        xmax::Integer,
        ymax::Integer,
        z::Integer
    )
    new_xmin = max(0, min(xmin, xmax))
    new_ymin = max(0, min(ymin, ymax))
    new_xmax = min(2^z - 1, max(xmin, xmax))
    new_ymax = min(2^z - 1, max(ymin, ymax))
    new_xmin, new_ymin, new_xmax, new_ymax
end

function boxsize(provider::AbstractProvider, xmin, ymin, xmax, ymax)
    abs(xmax - xmin) + 1, abs(ymax - ymin) + 1
end


Base.zero(x::arb_series) = arb_series(x.poly.parent(0), x.length)

"""
    bessel_j(ν::arb, z::arb_series[, n = length(z)])

> Compute the Taylor series of the Bessel function.

It's computed using a recursive formula for the Taylor coefficients
of the Bessel functions and then composing it with `z`.
"""
function bessel_j(ν::arb, z::arb_series, n = length(z))
    res = arb_series(parent(z.poly)(), n)

    if n > 0
        x = z[0]
        a0 = bessel_j(ν, x)
        res[0] = a0

        if !isfinite(a0)
            return arb_series(parent(z.poly)(fill(NaN, n)), n)
        end

        if n > 1
            a1 = 1 // 2 * (bessel_j(ν - 1, x) - bessel_j(ν + 1, x))
            res[1] = a1

            if !isfinite(a0)
                return arb_series(parent(z.poly)(fill(NaN, n)), n)
            end

            if n > 2
                a2 = 1 // 8 * (bessel_j(ν - 2, x) + bessel_j(ν + 2, x) - 2a0)
                res[2] = a2

                if !isfinite(a0)
                    return arb_series(parent(z.poly)(fill(NaN, n)), n)
                end

                if n > 3
                    a3 = 1 // 48 * (bessel_j(ν - 3, x) - bessel_j(ν + 3, x) - 6a1)
                    res[3] = a3

                    if !isfinite(a0)
                        return arb_series(parent(z.poly)(fill(NaN, n)), n)
                    end

                    if n > 4
                        x2 = x^2
                        ν2 = ν^2
                        for i = 4:n-1
                            k = i - 4
                            ai =
                                -(
                                    res[k] +
                                    2x * res[k+1] +
                                    (k^2 + 4k + 4 - ν2 + x2) * res[k+2] +
                                    (2k^2 + 11k + 15) * x * res[k+3]
                                ) / (x2 * (k^2 + 7k + 12))
                            res[i] = ai
                        end
                    end
                end
            end
        end
    end
    # Compose the Taylor series for the Bessel function with that of z
    z_tmp = arb_series(deepcopy(z.poly))
    z_tmp[0] = base_ring(parent(z.poly))(0)

    return Nemo.compose(res, z_tmp, n)
end

"""
    bessel_y(ν::arb, z::arb_series[, n = length(z)])

Compute the Taylor series of the Bessel function.

It's computed using a recursive formula for the Taylor coefficients of
the Bessel functions and then composing it with `z`. Note that since
both `bessel_j` and `bessel_y` satisfy the same differential equation
the recursive formula is the same.
"""
function bessel_y(ν::arb, z::arb_series, n = length(z))
    res = arb_series(parent(z.poly)(), n)

    if n > 0
        x = z[0]
        a0 = bessel_y(ν, x)
        res[0] = a0

        if !isfinite(a0)
            return arb_series(parent(z.poly)(fill(NaN, n)), n)
        end

        if n > 1
            a1 = 1 // 2 * (bessel_y(ν - 1, x) - bessel_y(ν + 1, x))
            res[1] = a1

            if !isfinite(a0)
                return arb_series(parent(z.poly)(fill(NaN, n)), n)
            end

            if n > 2
                a2 = 1 // 8 * (bessel_y(ν - 2, x) + bessel_y(ν + 2, x) - 2a0)
                res[2] = a2

                if !isfinite(a0)
                    return arb_series(parent(z.poly)(fill(NaN, n)), n)
                end

                if n > 3
                    a3 = 1 // 48 * (bessel_y(ν - 3, x) - bessel_y(ν + 3, x) - 6a1)
                    res[3] = a3

                    if !isfinite(a0)
                        return arb_series(parent(z.poly)(fill(NaN, n)), n)
                    end

                    if n > 4
                        x2 = x^2
                        ν2 = ν^2
                        for i = 4:n-1
                            k = i - 4
                            ai =
                                -(
                                    res[k] +
                                    2x * res[k+1] +
                                    (k^2 + 4k + 4 - ν2 + x2) * res[k+2] +
                                    (2k^2 + 11k + 15) * x * res[k+3]
                                ) / (x2 * (k^2 + 7k + 12))
                            res[i] = ai
                        end
                    end
                end
            end
        end
    end
    # Compose the Taylor series for the Bessel function with that of z
    z_tmp = arb_series(deepcopy(z.poly))
    z_tmp[0] = base_ring(parent(z.poly))(0)

    return Nemo.compose(res, z_tmp, n)
end

"""
    legendre_p(ν::arb_series, μ::arb, z::arb[, n = length(ν)])

> Compute the Taylor series of the Legendre function with respect to
  the parameter ν.
"""
function legendre_p(ν::arb_series, μ::arb, z::arb, n = length(ν))
    CC = ComplexField(z.parent.prec)
    PP = AcbPolyRing(CC, :x)
    νν = PP(ν.poly)
    μμ = PP(μ)
    zz = PP(z)

    a = -νν
    b = νν + 1
    c = 1 - μμ

    res = PP()

    ccall(
        (:acb_hypgeom_2f1_series_direct, :libarb),
        Cvoid,
        (
            Ref{acb_poly},
            Ref{acb_poly},
            Ref{acb_poly},
            Ref{acb_poly},
            Ref{acb_poly},
            Cint,
            Clong,
            Clong,
        ),
        res,
        a,
        b,
        c,
        0.5 * (1 - zz),
        1,
        2,
        CC.prec,
    )

    realres = arb_series(ν.poly.parent([real(coeff(res, i)) for i = 0:degree(res)]), n)
    ((1 + z) / (1 - z))^(μ / 2) * realres
end

"""
    legendre_p(ν::arb, μ::arb, z::arb_series[, n = length(z)])

> Compute the Taylor series of the Legendre function.

It's computed using a recursive formula for the Taylor coefficients
of the Legendre functions and then composing it with `z`.
"""
function legendre_p(ν::arb, μ::arb, z::arb_series, n = length(z))
    # TODO: Precompute values used several times
    res = arb_series(parent(z.poly)(), n)

    if n > 0
        x = z[0]
        a0 = legendre_p_safe(ν, μ, x)
        res[0] = a0

        if !isfinite(a0)
            return arb_series(parent(z.poly)(fill(NaN, n)), n)
        end

        if n > 1
            x2 = x^2
            onemx2 = 1 - x2

            a1 = ((ν + 1) * x * a0 - (ν - μ + 1) * legendre_p_safe(ν + 1, μ, x)) / (onemx2)
            res[1] = a1

            if !isfinite(a1)
                return arb_series(parent(z.poly)(fill(NaN, n)), n)
            end

            if n > 2
                ν2 = ν^2
                μ2 = μ^2

                a2 = (2x * a1 - (ν2 + ν - μ2 / (onemx2)) * a0) / (2(onemx2))
                res[2] = a2

                if n > 3
                    a3 =
                        (
                            4(onemx2)^2 * x * a2 +
                            2(2μ2 - (ν2 + ν) * (onemx2)) * x * a0 +
                            ((ν2 + ν + 2) * x2 + μ2 - ν2 - ν + 2) * (onemx2) * a1
                        ) / (6 * (onemx2)^3)
                    res[3] = a3

                    if n > 4
                        onemx22 = onemx2^2
                        xonemx2 = x * onemx2
                        ν2pν = ν2 + ν
                        μ2mν2pν = μ2 - ν2pν
                        fourx = 4x
                        for i = 4:n-1
                            k = i - 4
                            ai =
                                (
                                    (k + 1 + ν) * (ν - k) * res[k] -
                                    (0.5 * (2k^2 + (5k + 3) - ν2pν)) * fourx * res[k+1] +
                                    (
                                        ((-6k^2 - 24k - 24) + ν2pν) * x2 +
                                        (2k^2 + 8k + 8) +
                                        μ2mν2pν
                                    ) * res[k+2] +
                                    (2(k + 3) * (2k + 5)) * xonemx2 * res[k+3]
                                ) / (onemx22 * ((k + 4) * (k + 3)))
                            res[i] = ai
                        end
                    end
                end
            end
        end
    end
    # Compose the Taylor series for the Legendre function with that of
    # z
    z_tmp = arb_series(deepcopy(z.poly))
    z_tmp[0] = base_ring(parent(z.poly))(0)

    return Nemo.compose(res, z_tmp, n)
end

function legendre_p_safe(ν::arb, μ::arb, z::arb_series, n = length(z))
    legendre_p(ν, μ, z, n)
end


#How to build your first Model

using StatsBase, Gadfly
using Distributions
using Random
using MCHammer

clearconsole()

#Key Variables
n_trials = 10000
Revenue = rand(TriangularDist(2500000,4000000,3000000), n_trials)
Expenses = rand(TriangularDist(1400000,3000000,2000000), n_trials)

#Uncorrelated Model (1)
Profit = Revenue - Expenses

#Apply correlation to random samples
Rev_Exp_Cor = -0.8
cor_matrix = [1 Rev_Exp_Cor; Rev_Exp_Cor 1]

#Join Trial into an array and apply correlation
Trials = hcat(Revenue, Expenses)
Trials = corvar(Trials, n_trials, cor_matrix)

#Correlated Model(2) - Create Correlated Results Array
Profit_C = Trials[!,1] -Trials[!,2]
Trials = hcat(Trials[!,1], Trials[!,2], Profit_C)
cormat(Trials,1)

#Plot Density
plot(x=[Profit Profit_C], Geom.density, color=["Uncorrelated","Correlated"], Guide.Title("Compare Results Methods"))

#Plot S-Curves
# plot(layer(ecdf(Profit),minimum(Profit), maximum(Profit), Theme(default_color="orange")),layer(ecdf(Profit_C), minimum(Profit_C), maximum(Profit_C)), Guide.Title("Compare Portfolio Methods"))
# s_table = hcat(Profit_C,Revenue, Expenses)
# sensitivity_chrt(s_table,1)

println("Probability of Making 1m or less (uncorrelated) :",GetCertainty(Profit, 1000000, 0))
println("Input Correlation: ", cor(Revenue,Expenses),"\n")
println("Probability of Making 1m or less (correlated) :",GetCertainty(Profit_C, 1000000, 0))
println("Input Correlation: ", cor(Trials[:, 1],Trials[:,2]))
println("\n")

println("Model Outputs: Revenue, Expenses, Profit, Profit_C")
Profit_C


using LinearAlgebra

@enum Orientation::Int8 Colinear = 0 Clockwise = -1 CounterClockwise = 1

function mid_point(p1, p2)
    return (p1 .+ p2) ./ 2
end

function orientation(p1, p2, p3) # orientation
    val = (p2[2] - p1[2]) * (p3[1] - p2[1]) - (p2[1] - p1[1]) * (p3[2] - p2[2])
    return val == 0.0 ? Colinear : (val < 0 ? CounterClockwise : Clockwise)
end

# make edges and dual edges always be in CounterClockwise orientation
function rectify_orientation!(de, dv, v)
    p1 = [0, 0]
    p2 = v[2] .- v[1]
    p3 = dv[2] .- dv[1]
    or = orientation(p1, p2, p3)
    if or == Clockwise
        reverse!(de)
    elseif or == Colinear
        error("Found three colinear vertices trying to sneak as a triangle! 😠")
    end
end

function rectify_orientation!(triangles, vertices)
    for tri in triangles
        or = orientation(vertices[tri[1:3]]...)
        if or == Clockwise
            reverse!(tri)
        elseif or == Colinear
            error("Found three colinear vertices trying to sneak as a triangle! 😠")
        end
    end
end

function circumcenter(p1, p2, p3) # circumcenter
    mat = SMatrix{2}((p1[1] - p2[1]), (p1[1] - p3[1]), (p1[2] - p2[2]), (p1[2] - p3[2]))
    b = -SVector((p2' * p2 - p1' * p1) / 2, (p3' * p3 - p1' * p1) / 2)
    x = mat \ b
end

function longitude(p1, p2) # measure of a line
    return norm(p1 .- p2)
end

function area(p) # measure of a polygon
    A = 0.0
    for i = 2:length(p)
        A += p[i - 1][1] * p[i][2] - p[i - 1][2] * p[i][1]
    end
    A += p[end][1] * p[1][2] - p[end][2] * p[1][1]
    return abs(A) / 2
end


function EuropeanCallMC_DeltaBased(K, T, S, sig, r, div, N, M)

## setting parameters

dt = T/N
nudt = (r - div - 0.5 * sig^2) * dt
sigsdt = sig * sqrt(dt)
erddt = exp((r - div) * dt)

beta1 = -1

## variables to store the sums of the option values

sum_CT = 0
sum_CT2 = 0

function CND(X)
#   y::float64
#   Exponential::float64
#   SumA::float64
#   SumB::float64

  y = abs(X)

  if y > 37
    cnd = 0
  else
    Exponential = exp(-y^2/2)

    if y < 7.07106781186547
      SumA = 0.0352624965998911 * y + 0.700383064443688
      SumA = SumA * y + 6.37396220353165
      SumA = SumA * y + 33.912866078383
      SumA = SumA * y + 112.079291497871
      SumA = SumA * y + 221.213596169931
      SumA = SumA * y + 220.206867912376
      SumB = 0.0883883476483184 * y + 1.75566716318264
      SumB = SumB * y + 16.064177579207
      SumB = SumB * y + 86.7807322029461
      SumB = SumB * y + 296.564248779674
      SumB = SumB * y + 637.333633378831
      SumB = SumB * y + 793.826512519948
      SumB = SumB * y + 440.413735824752
      cnd = Exponential * SumA / SumB
    else
      SumA = y + 0.65
      SumA = y + 4 / SumA
      SumA = y + 3 / SumA
      SumA = y + 2 / SumA
      SumA = y + 1 / SumA
      cnd = Exponential / (SumA * 2.506628274631)

      end
  end

  if X > 0
    cnd = 1 - cnd
  end

cnd

end

function GDelta(CallPutFlag, S, X, T, r, b, v)

  #require("CND.jl")
  d1::Float64

  d1 = (log(S / X) + (b + v^2 / 2) * T) / (v * sqrt(T))

  if CallPutFlag == "c"
    delta = exp((b - r) * T) * CND(d1)

  elseif CallPutFlag == "p"
    delta = -exp((b - r) * T) * CND(-d1)
  end

    delta

end

## outer loop is the number of simulations bigger is better

for j = 1:M

  St = S
    cv = 0

    ## inner loop is simulating each price path of length N
  for i = 1:N

    t = (i-1) * dt
    delta = GDelta("c", St, K, t, r, div, sig)

    error = randn()
    Stn = St * exp(nudt + sigsdt * error) ## note, it only add the changes, does not store the path
    cv = cv + delta * (Stn-St*erddt)
    St = Stn

  end

    ## wraps up and stores all the values of each path

  CT = max(0, St - K) + beta1*cv
  sum_CT = sum_CT + CT
  sum_CT2 = sum_CT2 + CT^2

end

  ## discounting the average call value
call_value = sum_CT / M * exp(-r*T)

SD = (sqrt(sum_CT2 - 1/M * (sum_CT^2)) * exp(-2*r*T)) / (M-1)
SE = SD / sqrt(M)

println("The Call Value is $call_value")
println("The SD is $SD")
println("The SE is $SE")

MC = [call_value, SD, SE]

end


@with_kw struct BlackmanHarrisPulse <: AbstractWaveform
	half_bandwidth::Float64
	fc::Float64 = 0.0
end
function (pulse::BlackmanHarrisPulse)(t)
	a0 = 0.353222222
	a1 = -0.488
	a2 = 0.145
	a3 = -0.010222222
	T = 1 / pulse.half_bandwidth
	 if 0 ≤ t ≤ T
	 return (a0 + a1 * cos((2π*t)/T) + a2 * cos((2π*2*t)/T) +  a3 * cos((2π*3*t)/T)) * cos(2π*pulse.fc*(t))
	 else
		 return 0
	 end
end
@with_kw struct DBlackmanHarrisPulse <: AbstractWaveform
	half_bandwidth::Float64
	fc::Float64 = 0.0
end
function (pulse::DBlackmanHarrisPulse)(t)
	a0 = 0.353222222
	a1 = -0.488
	a2 = 0.145
	a3 = -0.010222222
	T = 1 / pulse.half_bandwidth
	 if 0 ≤ t ≤ T
	 return -( a1 * sin((2π*t)/T) + a2 * sin((2π*2*t)/T) +  a3 * sin((2π*3*t)/T)) * cos(2π*pulse.fc*(t))
	 else
		 return 0
	 end
end

const BlackmanHarrisWaveForm = Union{BlackmanHarrisPulse,DBlackmanHarrisPulse}

@recipe function f(s::BlackmanHarrisWaveForm;nsamples = 20,spectrum = true)
	linewidth := 2
	label := "BlackmanHarris Pulse"
	T = 1/s.half_bandwidth
	if spectrum
		xlabel := "frequency"
		xrange := [-s.fc- 10*s.half_bandwidth,s.fc+ 10*s.half_bandwidth]
	else
		xlabel := "time"
		xrange := [0,2T]
	end
	δt = 1/(s.fc+ s.half_bandwidth)/nsamples
	t_samples = range(0,20T,step = δt)
 if spectrum return fourier(t_samples,s.(t_samples)) else return (t_samples,s.(t_samples)) end
end


using Graphs
using SparseArrays

function dijkstra_digraph(mat, nrows, ncols, startindex, endindex)
    dist = spzeros(Int, length(mat), length(mat))
    g = SimpleDiGraph(length(mat))
    for i in 1:length(mat)
        if i % nrows != 0
            add_edge!(g, i, i + 1)
            dist[i, i + 1] = mat[i + 1]
            add_edge!(g, i + 1, i)
            dist[i + 1, i] = mat[i]
        end
        if i <= length(mat) - nrows
            add_edge!(g, i, i + nrows)
            dist[i, i + nrows] = mat[i + nrows]
            add_edge!(g, i + nrows, i)
            dist[i + nrows, i] = mat[i]
        end
    end
    return dijkstra_shortest_paths(g, startindex, dist).dists[endindex]
end

function day15()
    part = [0, 0]
    lines = filter(!isempty, strip.(readlines("AoCdata/AoC_2021_day15.txt")))
    nrows, ncols = length(lines), length(first(lines))
    mat = zeros(Int, nrows, ncols)
    for i in 1:nrows
        mat[i, :] = parse.(Int, collect(lines[i]))
    end
    part[1] = dijkstra_digraph(mat, nrows, ncols, 1, length(mat))

    mat2 = zeros(Int, ncols * 5, nrows * 5)
    for c in CartesianIndices(mat2)
        mat2[c] = mod1(
            mat[mod1(c[1], nrows), mod1(c[2], ncols)] +
            (c[1] - 1) ÷ nrows +
            (c[2] - 1) ÷ ncols,
            9,
        )
    end
    part[2] = dijkstra_digraph(mat2, nrows * 5, ncols * 5, 1, length(mat2))

    return part
end

part = day15()
println("Part 1: ", part[1])
println("Part 2: ", part[2])


# Adaptive MvNormal random walk.
# See: https://www.tandfonline.com/doi/abs/10.1198/jcgs.2009.06134
mutable struct MvAdaptiveRWM{M <: AbstractVector{<:Real}, V <: AbstractMatrix{<:Real},
                             B <: Real} <: Metropolis
  sample_mean::M
  sample_cov::V
  beta::B
  iter::Int
  d::Int
end
function MvAdaptiveRWM(v::AbstractVector{<:Real}; beta=0.05, iter=1)
  d = length(v)
  return MvAdaptiveRWM(v, eye(d) * 0.01 / d, beta, iter, d)
end
function update_stats!(rwm::MvAdaptiveRWM, x::AbstractVector{<:Real})
  rwm.iter += 1
  update_mean!(rwm.sample_mean, x, rwm.iter)
  update_cov!(rwm.sample_cov, rwm.sample_mean, x, rwm.iter)
end
function _update(rng::Random.AbstractRNG, rwm::MvAdaptiveRWM, curr::AbstractVector{<:Real},
                 logprob::Function)
  # Update summary stats.
  update_stats!(rwm, curr)

  # Construct proposal covariance.
  proposal_cov = if rwm.iter <= 2 * rwm.d || rwm.beta > rand()
    0.01 * eye(rwm.d) / rwm.d
  else
    5.6644 * rwm.sample_cov / rwm.d
  end
  proposal_cov .= Matrix(LinearAlgebra.Symmetric(proposal_cov))

  return _update(rng, StaticRWM(MvNormal(proposal_cov)), curr, logprob)
end
const mvarwm = MvAdaptiveRWM


include("header.jl")

#=
# Dimensionless quantities
In this notebook, we will learn about making quantities dimensionless,
and the physical significance of this.
=#

# ### Set up the module
using MAE103
#-
using Plots

#=
In the previous notebook, we made a plot of the velocity field in a gap
between two walls. We'll regenerate the plot here for reference:
=#
u(y,Uc,H) = 4*Uc/H^2*y*(H-y)
H = 1u"cm"
Uc = 1u"m/s"
y = range(0u"cm",H,length=101)
v = u.(y,Uc,H)
plot(v,y,xlim=(0,Inf),ylim=(0,Inf),xlabel=L"u(y)",ylabel=L"y")

#=
In this plot, the data have units---they are *dimensional* data.
This is useful, since the plot not only conveys the velocities, but
also the gap size and the flow speed.

Let us also note that the plot is quite specific, since it only shows
the plot for *one* choice of gap size (1 cm) and *one* choice of flow speed (1 m/s).
Of course, it is not difficult for us to get plots at different gap sizes
and flow speeds by simply changing the parameters.

But suppose that these data were measured in an experiment. For the experiment,
let's say we constructed one gap $H$, we applied one flow speed $U_c$, and
we measured the resulting profile of fluid velocity across the gap. Then this plot,
at these specific conditions, would not be directly useful to someone else who would like
to know the velocity profile in a different size gap at a different flow speed.

If, instead, we divide the velocity data `v` by flow speed `Uc` and divide the position
data `y` by gap size `H`, then we "scale out" the specific choices for these parameters:
=#
plot(v/Uc,y/H,xlim=(0,Inf),ylim=(0,Inf),xlabel=L"u(y)/U_c",ylabel=L"y/H",yticks=0:0.25:1)

#=
This is a *dimensionless plot*. The values on each axis have no units: they are
called *dimensionless* or *non-dimensional* quantities.

This plot is more directly useful to another person, because they can read off
the values for a particular position *as a fraction of gap height*,
and find the velocity *as a fraction of the centerline velocity*. Then, they can
"scale" the value to their own situation.

For example, if a person whose gap size is $H = 3$ cm and centerline velocity is
$0.5$ m/s wishes to know the velocity at $y = 0.6$ cm, they calculate $y/H = 0.6/3 = 0.2$
and look up on the plot that $y/H = 0.2$ corresponds to $u/U_c = 0.64$. Thus,
for this person's case, $u = 0.64\times0.5$ m/s  $= 0.32$ m/s.
=#

#-
#=
### Unit as a reference quantity.
The principle of "scaling out" the parameters is very important. But before we
go deeper into this, let us make an observation on units.

A unit is a reference value, and this reference is standardized so we can communicate
results with one another on a common foundation. What does it mean to be a reference?

When we measure something, it is important to remember that the thing we're measuring is not
intrinsically tied to any unit. It is more abstract than that. For example, the
distance between two points is a physical fact. Our measurement of that distance with
a particular set of units is simply one way (of many) quantifying that fact.

With that measurement, we are simply comparing that something with another
thing that has been calibrated on a particular unit system. A ruler is the easiest
example of this: the ruler was constructed by marking off lines on a stick,
with each line determined from some other calibrated tool that the ruler manufacturer used.
Ultimately, all of these measurement tools refer back to some ultimate standard
for the unit (e.g. for a meter, it is length of the path travelled by light in a
vacuum in 1/(299,792,458) of a second.) As long as we all agree on this standard,
we can communicate with each other.
=#
#-
#=
However, we don't quite all agree. Most use SI units, but many still use imperial.
So we need conversion factors, and there is opportunity to make mistakes.
=#
#-
#=
### A more physically-meaningful reference quantity.
As we discussed earlier, when we measure something, we are making a comparison
with a standardized unit. We can think of this comparison as a ratio of the
physical quantity with the standard unit. For example, if we were to say the gap $H$ was
0.1 cm high, we could also write this statement as

$$\dfrac{H}{1\,\mathrm{cm}} = 0.1$$

However, this unit is a reference that we (humans) invented, so it is completely irrelevant
to physics.

On the other hand, if we instead choose as a reference some parameter
in the problem itself, then we are much more likely to have a *physically
meaningful* reference. In our example problem, the gap height is clearly physically
meaningful, so it serves as a natural *length scale*. The centerline velocity
is a natural *velocity scale*. These are our *reference scales*.

Actually, the statement that the dimensionless velocity plot has "no units" isn't
quite true. There *are* units on this plot. The units of position are "gap heights"
and the units of velocity are "centerline velocities". We can say, for example,
the: "the velocity at 0.2 gap heights from the lower wall is 0.64 times the centerline
velocity"). The huge advantage of this "local" system of units is that it requires no
standarization or international governing body. It is unambiguous to *anyone*,
including to an alien species.
=#

#-
#=
**In summary**, there are several advantages to the dimensionless view:
- There is no ambiguity about which system we used. It doesn't matter if
  the original parameters and values were in SI or imperial.
- Since the laws of physics hold regardless of our system of units, this sort of plot
    reveals true physical relationships between velocity and location.
- The dimensionless relationship between $u/U_c$ and $y/H$ is *universal*: it holds
  for *every* gap size and
  *every* flow speed. We know that this is true here because the exact velocity
  field was actually known to us:

  $$u(y) = \frac{4U_c}{H^2} y (H - y)$$

  But let's divide this by $U_c$ and group $y$ with $H$:

  $$\dfrac{u}{U_c} = 4 \dfrac{y}{H}\left(1 - \dfrac{y}{H}\right)$$

  This shows that $u/U_c$ depends on $y/H$ *and nothing
else*. There are no "free" parameters in this dimensionless relationship. We
will return to this point in the next notebook.
- We are lucky here because we already knew the mathematical equation. In most
cases, we aren't this lucky. But as we will discuss in the next notebook, once we
  ascertain the number of dimensionless quantities that fully describe a relationship,
  we can develop an *empirical correlation of data*. This correlation should *only* be
  pursued on dimensionless data. A correlation with dimensional coefficients is
  truly useless.
=#
#-
#=
### Dimensional homogeneity
One last point before move on. When we non-dimensionalize a quantity by a reference
quantity, they need to be in the same units. If they're not, then we need to
make sure the conversion factors are accounted for. For example, let's suppose
that the velocites in $u$ are measured in m/s, but the centerline velocity $U_c$ was
measured in ft/s. We obviously have to deal with the conversion between ft and
m to take the ratio $u/U_c$.

A conversion factor between two systems of units
is itself a dimensionless quantity. We can obtain the numerical factor with the
help of the `DimensionlessParameter` function. For example, let's consider the
conversion factor from feet to meters.
=#
DimensionlessParameter(1u"m"/1u"ft")

#=
We would multiply $U_c$ in ft/s by this factor to convert it to m/s.

But we can actually make use of a neat feature to skip this step, because
these software tools will carry out the conversion for us! For example, if
we divide $u = 0.5$ m/s by $U_c = 3$ ft/s:
=#
DimensionlessParameter(0.5u"m/s"/3u"ft/s")

#=
This name `DimensionlessParameter` isn't very descriptive! It might
be nice to use a more specific name for this quantity. Let's create
a new function that we can use to set this dimensionless velocity:
=#
@nondimvar DimensionlessVelocity
#=
Now we can use this to set the ratio of velocities:
=#
u_nd = DimensionlessVelocity(0.5u"m/s"/3u"ft/s")


#=
Geometry in 2d euclidean space
=#

"""
deltaangle(a::Real, b::Real)

Return the minimum δ such that
    a + δ = mod(b, 2π)
"""
deltaangle(a::Real, b::Real) = atan2(sin(b-a), cos(b-a))

"""
Distance between two angles
"""
angledist( a::Real, b::Real ) = abs(deltaangle(a,b))

"""
Linear interpolation between angles
"""
lerp_angle(a::Real, b::Real, t::AbstractFloat) = a + deltaangle(a, b)*t

"""
True if the values are collinear within a tolerance
"""
function are_collinear(a::AbstractVec, b::AbstractVec, c::AbstractVec, tol::Float64=1e-8)
    # http://mathworld.wolfram.com/Collinear.html
    # if val = 0 then they are collinear
    val = a.x*(b.y-c.y) + b.x*(c.y-a.y)+c.x*(a.y-b.y)
    abs(val) < tol
end

"""
To find orientation of ordered triplet (p, q, r).
The function returns following values
0 --> p, q and r are colinear
1 --> Clockwise
2 --> Counterclockwise
"""
function orientation(P::VecE2, Q::VecE2, R::VecE2)
    val = (Q.y - P.y)*(R.x - Q.x) - (Q.x - P.x)*(R.y - Q.y)
    if val ≈ 0
        return 0  # colinear
    end
    return (val > 0) ? 1 : 2
end

function inertial2body(point::VecE2, reference::VecSE2)

    #=
    Convert a point in an inertial cartesian coordinate frame
    to be relative to a body's coordinate frame

    The body's position is given relative to the same inertial coordinate frame
    =#

    s, c = sin(reference.θ), cos(reference.θ)
    Δx = point.x - reference.x
    Δy = point.y - reference.y
    VecE2(c*Δx + s*Δy, c*Δy - s*Δx)
end
function inertial2body(point::VecSE2, reference::VecSE2)

    #=
    Convert a point in an inertial cartesian coordinate frame
    to be relative to a body's coordinate frame

    The body's position is given relative to the same inertial coordinate frame
    =#

    s, c = sin(reference.θ), cos(reference.θ)
    Δx = point.x - reference.x
    Δy = point.y - reference.y
    VecSE2(c*Δx + s*Δy, c*Δy - s*Δx, point.θ - reference.θ)
end
function body2inertial(point::VecE2, reference::VecSE2)

    #=
    Convert a point in a body-relative cartesian coordinate frame
    to be relative to a the inertial coordinate frame the body is described by
    =#

    c, s = cos(reference.θ), sin(reference.θ)
    VecE2(c*point.x -s*point.y + reference.x, s*point.x +c*point.y + reference.y)
end
function body2inertial(point::VecSE2, reference::VecSE2)

    #=
    Convert a point in a body-relative cartesian coordinate frame
    to be relative to a the inertial coordinate frame the body is described by
    =#

    c, s = cos(reference.θ), sin(reference.θ)
    VecSE2(c*point.x -s*point.y + reference.x, s*point.x +c*point.y + reference.y, reference.θ + point.θ)
end

include("1d.jl")
include("lines.jl")
include("line_segments.jl")
include("rays.jl")
include("projectiles.jl")
include("solids.jl")
include("hyperplanes.jl")


##
cd(@__DIR__())
rot(ϕ) = Complex{Int}(round(Int,cos(deg2rad(ϕ))), round(Int,sin(deg2rad(ϕ))))
moves = Dict(
    'N' => α -> c -> begin (p,d) = c; (p + Complex{Int}(0,α), d) end,
    'S' => α -> c -> begin (p,d) = c; (p + Complex{Int}(0,-α), d) end,
    'E' => α -> c -> begin (p,d) = c; (p + Complex{Int}(α,0), d) end,
    'W' => α -> c -> begin (p,d) = c; (p + Complex{Int}(-α,0), d) end,
    'L' => ϕ -> c -> begin (p,d) = c; (p, d * rot(ϕ)) end,
    'R' => ϕ -> c -> begin (p,d) = c; (p, d * rot(-ϕ)) end,
    'F' => s -> c -> begin (p,d) = c; (p + s * d, d) end,
)
dist(p::Complex) = reim(p) .|> abs |> sum
dist(t::Tuple) = dist(first(t))
dist(t1::Tuple, t2::Tuple) = dist(t2[1] - t1[1]);

##
actions = map(eachline("test.txt")) do line
    moves[line[1]](parse(Int, line[2:end]))
end;

##
start = Complex{Int}(0,0), Complex{Int}(1,0)
finish = foldl(actions; init = start) do c, a
    a(c)
end
@show finish
dist(start, finish)

## PARTE 2
moves = Dict(
    'N' => α -> c -> begin (p,d) = c; (p, d + Complex{Int}(0,α)) end,
    'S' => α -> c -> begin (p,d) = c; (p, d + Complex{Int}(0,-α)) end,
    'E' => α -> c -> begin (p,d) = c; (p, d + Complex{Int}(α,0)) end,
    'W' => α -> c -> begin (p,d) = c; (p, d + Complex{Int}(-α,0)) end,
    'L' => ϕ -> c -> begin (p,d) = c; (p, d * rot(ϕ)) end,
    'R' => ϕ -> c -> begin (p,d) = c; (p, d * rot(-ϕ)) end,
    'F' => s -> c -> begin
        (p,d) = c
        (p + s * d, d)
    end,
)
actions = map(eachline("input.txt")) do line
    moves[line[1]](parse(Int, line[2:end]))
end;

##
start = Complex{Int}(0,0), Complex{Int}(10,1)
finish = foldl(actions; init = start) do c, a
    a(c)
end
@show finish
dist(start, finish)


## Multiply 2 matrices
# checks for invalid dimensions
# uses 1st method (element dot product method)

function outer(A,B)
  return A * B'
end

function matmul(A,B)

# convert any 1D column vectors into 1*x 2D matrices
  if ndims(A) = 1
    A = A[:,:]
  end

  if ndims(B) = 1
    B = B[:,:]
  end

# define row and column sizes
# for each matrix
  rowA, colA = size(A)
  rowB, colB = size(B)

# check that dimensions are valid for multiplication, otherwise throw error
  if colA != rowB
    throw(DimensionMismatch("Matrices of dimensions $(size(A)) and $(size(B)) cannot be multiplied together"))
  end

# initialise zero matrix to hold result
# matrix size is the outer dimensions of the two matrices
  result = zeros(rowA, colB)

# calculate A*B, populate zero matrix with results
# uses the outer products of each row with its corresponding column
# and adds the results together
  for i in 1:rowB # could also use colA, as colA = rowB
      result += outer(A[:, i], B[i, :])
  end

# return the result
  return result
end


logrange(x1, x2, n) = (10^y for y in range(log10(x1), log10(x2), length=n))

mutable struct LogRange

    lr::StepRangeLen

end

LogRange(first, last, n) = LogRange(range(log10(first), log10(last), length=n))

function Base.iterate(a::LogRange)

    v, s = iterate(a.lr)

    return 10^v, s

end

function Base.iterate(a::LogRange, b)

    if b > length(a.lr)
        return nothing
    end

    v, s = iterate(a.lr, b)

    return 10^v, s

end

Base.length(a::LogRange)      = length(a.lr)

Base.getindex(a::LogRange, i) = 10^getindex(a.lr, i)

Base.firstindex(a::LogRange)  = firstindex(a.lr)

Base.lastindex(a::LogRange)   = lastindex(a.lr)

Base.eltype(::Type{LogRange})  = Real


using SparseArrays: sparse

export constant_constraint
"""
    constant_constraint(comp::CellComplex{N},
        cells::AbstractVector{Cell{N}}, k::Int) where N

Find a sparse representation of the constraint that a `k-1` form is constant on given
`k-1` dimensional cells.
"""
function constant_constraint(comp::CellComplex{N},
    cells::AbstractVector{Cell{N}}, k::Int) where N
    row_inds, col_inds, vals = Int[], Int[], Float64[]
    for (row_ind, (c1, c2)) in enumerate(zip(cells[1:end-1], cells[2:end]))
        col1 = findfirst(isequal(c1), comp.cells[k])
        col2 = findfirst(isequal(c2), comp.cells[k])
        push!(row_inds, row_ind); push!(col_inds, col1); push!(vals, 1)
        push!(row_inds, row_ind); push!(col_inds, col2); push!(vals, -1)
    end
    return sparse(row_inds, col_inds, vals, length(cells)-1, length(comp.cells[k]))
end

export zero_constraint
"""
    zero_constraint(comp::CellComplex{N}, cells::AbstractVector{Cell{N}}, k::Int) where N

Find a sparse representation of the constraint that a `k-1` form is zero on given `k-1`
dimensional cells.
"""
function zero_constraint(comp::CellComplex{N}, cells::AbstractVector{Cell{N}},
    k::Int) where N
    col_inds = [findfirst(isequal(c), comp.cells[k]) for c in cells]
    num_rows = length(col_inds)
    return sparse(1:num_rows, col_inds, ones(num_rows), num_rows, length(comp.cells[k]))
end


function lagrange_poly_basis(nodes, evalPt)
    order = length(nodes)
    poly_coefs = zeros(order)
    for n in 1:order
        notn = [1:n-1;n+1:order]
        poly_coefs[n] = prod(evalPt.-nodes[notn])./prod(nodes[n].-nodes[notn])
    end
    return poly_coefs
end

function lagrange_interpolant()

end
function gauss_lobatto_quadrature(fun::Function,a::Float64,b::Float64,n_evals::Int)
    (b<=a)&&throw(DomainError("must have a<b"))
    (n_evals<1)&&throw(DomainError("n_evals must be > 0"))
    if n_evals > 1
        # the GL nodes
        # nodes in [-1,1]
        nodes = Jacobi.zglj(n_evals, 0, 0)
        weights = Jacobi.wglj(nodes,0,0)*(b-a)/2
        # shift nodes to [a,b]
        nodes *= 0.5*(b-a)
        nodes .+= 0.5*(a+b)
    else
        nodes = 0.5*(a+b)
        weights = (b-a)
    end
    fun_vals = fun.(nodes)
    quad = LinearAlgebra.dot(fun_vals,weights)
end


using RLinearAlgebra
using LinearAlgebra
using UnicodePlots

n = 10
d = 10
κ = 8.0

A = RLinearAlgebra.generate_matrix(n, d, d, κ)
b = randn(n)

sol = LinearSolver(TypeRPM())
x = rsolve(sol, A, b)
plt = lineplot(sol.log.residual_hist, title = "Default RPM", xlabel = "iteration", ylabel = "residual")
println(plt)

sol = LinearSolver(TypeRPM(SamplerKaczmarzCYC()))
x = rsolve(sol, A, b)
plt = lineplot(sol.log.residual_hist, title = "Cyclic Kaczmarz", xlabel = "iteration", ylabel = "residual")
println(plt)

sol = LinearSolver(TypeRPM(SamplerKaczmarzCYC(), ProjectionLowCore()))
x = rsolve(sol, A, b)
plt = lineplot(sol.log.residual_hist, title = "Cyclic with Low Core", xlabel = "iteration", ylabel = "residual")
println(plt)

sol = LinearSolver(TypeRPM(SamplerKaczmarzCYC(), ProjectionFullCore()))
x = rsolve(sol, A, b)
plt = lineplot(sol.log.residual_hist, title = "Cyclic with Full Core", xlabel = "iteration", ylabel = "residual")
println(plt)

sol = LinearSolver(TypeRPM(SamplerMotzkin()))
x = rsolve(sol, A, b)
plt = lineplot(sol.log.residual_hist, title = "Motzkin", xlabel = "iteration", ylabel = "residual")
println(plt)

sol = LinearSolver(TypeRPM(SamplerMotzkin(), ProjectionLowCore()))
x = rsolve(sol, A, b)
plt = lineplot(sol.log.residual_hist, title = "Motzkin low core", xlabel = "iteration", ylabel = "residual")
println(plt)

sol = LinearSolver(TypeRPM(SamplerMotzkin(), ProjectionFullCore()))
x = rsolve(sol, A, b)
plt = lineplot(sol.log.residual_hist, title = "Motzkin full core", xlabel = "iteration", ylabel = "residual")
println(plt)

sol = LinearSolver(TypeRPM(SamplerGaussSketch()))
x = rsolve(sol, A, b)
plt = lineplot(sol.log.residual_hist, title = "Gauss", xlabel = "iteration", ylabel = "residual")
println(plt)

sol = LinearSolver(TypeRPM(SamplerGaussSketch(), ProjectionLowCore()))
x = rsolve(sol, A, b)
plt = lineplot(sol.log.residual_hist, title = "Gauss Low Core", xlabel = "iteration", ylabel = "residual")
println(plt)

sol = LinearSolver(TypeRPM(SamplerGaussSketch(), ProjectionFullCore()))
x = rsolve(sol, A, b)
plt = lineplot(sol.log.residual_hist, title = "Gauss Full Core", xlabel = "iteration", ylabel = "residual")
println(plt)

samp = SamplerMotzkin()
samp.sampled = true
sol = LinearSolver(TypeRPM(samp))
x = rsolve(sol, A, b)
plt = lineplot(sol.log.residual_hist, title = "RPM Convergence", xlabel = "iteration", ylabel = "residual")
println(plt)

println("Plot collection of samplers")
plt = lineplot(0:500, zeros(501), width = 100, height = 20, xlabel = "iteration", ylabel = "residual",
               ylim = [0, norm(b)])
samplers = RPMSamplers()
for s in samplers
    solt = LinearSolver(TypeRPM(s))
    xx = rsolve(solt, A, b)
    lineplot!(plt, solt.log.residual_hist)
end
println(plt)

println("Randomized Gauss seidel")
sol = LinearSolver(TypeRGS())
x = rsolve(sol, A, b)
plt = lineplot(sol.log.residual_hist, title = "Randomized Gauss Seidel", xlabel = "iteration", ylabel = "residual")
println(plt)


#=
Problem 102
Three distinct points are plotted at random on a Cartesian plane, for which -1000 ≤ x, y ≤ 1000, such that a triangle is formed.

Consider the following two triangles:

A(-340,495), B(-153,-910), C(835,-947)
X(-175,41), Y(-421,-714), Z(574,-645)

It can be verified that triangle ABC contains the origin, whereas triangle XYZ does not.

Using triangles.txt (right click and 'Save Link/Target As...'), a 27K text file containing the co-ordinates of one thousand "random" triangles, find the number of triangles for which the interior contains the origin.

NOTE: The first two examples in the file represent the triangles in the example given above.
=#

using DataFrames

function containsOrigin(p1, p2, p3)
    # Change of basis approach...
    # Shift origin to p1, then write original origin as a linear combination. Problem gets very easy
    lin_comb = inv([(p2 - p1) (p3 - p1)]) * (-p1)
    if (lin_comb[1] > 0 && lin_comb[2] > 0 && (lin_comb[1] + lin_comb[2] < 1))
        return 1
    end
    return 0
end

df = readtable("appendices/p102_triangles.txt", header = false)
numTriangles = size(df)[1]

result = zeros(numTriangles)

for i = 1:numTriangles
    result[i] = containsOrigin(Array(df[i, 1:2])', Array(df[i, 3:4])', Array(df[i, 5:6])')
end

@show sum(result)


# Raised Cosine distribution
#
# Ref: http://en.wikipedia.org/wiki/Raised_cosine_distribution
#

immutable Cosine <: ContinuousUnivariateDistribution
    μ::Float64
    σ::Float64

    Cosine(μ::Real, σ::Real) = (@check_args(Cosine, σ > zero(σ)); new(μ, σ))
    Cosine(μ::Real) = new(μ, 1.0)
    Cosine() = new(0.0, 1.0)
end

@distr_support Cosine d.μ - d.σ d.μ + d.σ

#### Parameters

location(d::Cosine) = d.μ
scale(d::Cosine) = d.σ

params(d::Cosine) = (d.μ, d.σ)

#### Statistics

mean(d::Cosine) = d.μ

median(d::Cosine) = d.μ

mode(d::Cosine) = d.μ

var(d::Cosine) = d.σ^2 * 0.13069096604865779  # 0.130... = 1/3 - 2 / π^2

skewness(d::Cosine) = 0.0

kurtosis(d::Cosine) = -0.59376287559828102362

#### Evaluation

function pdf(d::Cosine, x::Float64)
    if insupport(d, x)
        z = (x - d.μ) / d.σ
        return (1.0 + cospi(z)) / (2 * d.σ)
    else
        return 0.0
    end
end

logpdf(d::Cosine, x::Float64) = insupport(d, x) ? log(pdf(d, x)) : -Inf

function cdf(d::Cosine, x::Float64)
    z = (x - d.μ) / d.σ
    0.5 * (1.0 + z + sinpi(z) * invπ)
end

function ccdf(d::Cosine, x::Float64)
    nz = (d.μ - x) / d.σ
    0.5 * (1.0 + nz + sinpi(nz) * invπ)
end

quantile(d::Cosine, p::Float64) = quantile_bisect(d, p)