### A Pluto.jl notebook ###
# v0.14.0

using Markdown
using InteractiveUtils

# This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error).
macro bind(def, element)
    quote
        local el = $(esc(element))
        global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : missing
        el
    end
end

# ╔═╡ 4ea0ccfa-9622-11eb-1cf0-e9ae2f927dd2
html"""

Section 2.8

 Linear Model, Data Science, & Simulations

Lecture Video

"""

# ╔═╡ d155ea12-9628-11eb-347f-7754a33fd403
begin
    import Pkg
    Pkg.activate(mktempdir())
    Pkg.add([
        Pkg.PackageSpec(name="Plots", version="1"),
        Pkg.PackageSpec(name="PlutoUI", version="0.7"),
        Pkg.PackageSpec(name="DataFrames", version="0.22"),
        Pkg.PackageSpec(name="CSV", version="0.8"),
        Pkg.PackageSpec(name="GLM", version="1"),
        Pkg.PackageSpec(name="Distributions", version="0.24"),
    ])
    using Plots, PlutoUI, DataFrames, CSV, GLM, Statistics, LinearAlgebra, Distributions
end

# ╔═╡ 01506de2-918a-11eb-2a4d-c554a6e54631
TableOfContents(title="📚 Table of Contents", aside=true)

# ╔═╡ 877deb2c-702b-457b-a54b-f27c277928d4
md"""
# Julia concepts for data science
- Data Frames (`DataFrames.jl`)
- `CSVread`, `CSVwrite` (`CSV.jl`)
- `lm` (linear model) (`GLM.jl`)
- `@formula` (formula macro to specify variables to analyze) (`GLM.jl`)

- Underscore as digits separator (e.g. `1_000` for 1000)

- The value of fast simulations.
"""

# ╔═╡ 36ce167f-382c-4b9a-be34-83250b10c4e5
md"""
In this lecture we will simulate a real world statistical application
for the purpose of understanding what statistics is about.  It is very
	helpful in simulations to be able to run many examples fast.
"""

# ╔═╡ 83912943-a847-420a-bfdb-450027b631e8
md"""
# Fahrenheit and Celsius Data Set
"""

# ╔═╡ 280d112f-d34a-4cc4-9e3a-4ebbfcd5eb51
n = 10

# ╔═╡ b5031c96-db57-4baf-b271-6bb12e29de9b
x = sort((rand( -10:100, n)))

# ╔═╡ c2f77e8f-a8c0-4144-a8b4-b25dd98ed234
y = 5/9 .* x  .- 17.7777777 #  same as y =  5/9 .* (x .- 32)

# ╔═╡ ad161b98-f4a1-42ac-ad4f-8b71fabcfde9
begin
	plot(x,y, m=:c, mc=:red,legend=false)
	xlabel!("°F")
	annotate!(-4,16,text("°C",11))
	# plot!( x, (x.-30)./2) Dave's cool approximation
end

# ╔═╡ 8e422886-74ef-4c0f-be1e-fda238c8db44
[x y]

# ╔═╡ ca21122a-2522-482a-b7ef-bd73e96cb5a9
md"""
## Julia: Data Frames
I like to think of a Data Frame as a matrix with labels.
"""

# ╔═╡ 41e05b1e-8b5e-45e3-91bb-01355ade9f3d
md"""
### Data Frame by Columns with labels
"""

# ╔═╡ 9d2e3861-ca36-406e-952d-831ca3947e44
data = DataFrame(°F=x,°C=y) # Label = data

# ╔═╡ e73854ed-3581-41c4-ada5-e48242033759
md"""
### Data Frame with a matrix
"""

# ╔═╡ 9873d944-b611-46f9-82a7-0cf714a3078c
begin
	data2 = DataFrame([x  y]) # convert Matrix to DataFrame
    rename!(data2,["°F","°C"]) # add column labels
end

# ╔═╡ 2be44753-afee-4125-b6bc-8866d2293dc2
Matrix(data2) # Convert back to a matrix (lose label information)

# ╔═╡ 6e07e8fb-fe51-4b37-bfb2-d1466e768754
md"""
## Julia:Comment about types

Notice that [x y] converts all the data to floats, but columns of a data frame can have different types.
"""

# ╔═╡ a755e58a-b16c-4d3b-a85f-81ccf374793f
md"""
# Reading/Writing CSV (comma separated values) Files
"""

# ╔═╡ f1e10fb7-adac-4083-8977-616a505fd591
md"""
  ## Writing Data to a CSV  file
readable by spreadsheet software.
"""

# ╔═╡ 2e42986c-2de3-49e6-9c29-a7313c0b1da8
CSV.write("testCSVwrite.csv", data)

# ╔═╡ 22758dd6-9d04-4616-ba99-1430f2dedf9a
md"""
 ## Reading Data from a CSV file to a DataFrame
"""

# ╔═╡ aff6a616-6d8b-4584-a6f2-195decef7774
data_again = CSV.read("testCSVwrite.csv", DataFrame )

# ╔═╡ 338da13a-3c26-4366-a669-ac3e24f31577
data_again[:,"°F" ] #or data_again[:,1]

# ╔═╡ 5a742546-1e4d-4aee-bed1-cb10c543e439
data_again[:,1]

# ╔═╡ 6a9c8c9a-fac7-42f7-976d-3168132cae48
md"""
# Noisy Data
## Add some random noise to the celsius readings
"""

# ╔═╡ 3c038b68-8676-4877-9720-38da7c4e0e0e
begin
	noisy_data = copy(data)  # Noisy DataFrame
	noisy_data[:, "°C" ] .+= noise * randn(n)
	yy = noisy_data[:, "°C" ]
	noisy_data
end

# ╔═╡ 5a877e40-a101-4f7d-b2a1-ef4cfe5d8807
begin

	scatter(x, yy,m=:c,mc=:red, label="noisy data", ylims=(-40,40))
	for i=1 : length(data[:,2])
		plot!([x[i],x[i]], [m*x[i]+b,yy[i]], color=:gray, ls=:dash, label=false)
	end
	xlabel!("°F")
	annotate!(-15,16,text("°C",11))
	plot!(x, m.*x .+ b,  color=:blue, label="best fit line")
	plot!(x,y,alpha=.5, color=:red, label="theory") # theoretical
	plot!(legend=:top)
end

# ╔═╡ 83c28c76-2eab-49f9-9999-05df85054520
md"""
# The noise slider (so I can find it easily)
"""

# ╔═╡ ba671804-dc6d-415c-89de-9cf6294907b3
md"""
noise = $(@bind noise Slider(0:.5:1000, show_value = true ))
"""

# ╔═╡ e8683a71-5822-4491-9ccd-20e0fc3bf531
md"""
## Statistics Software Outputs Mysterious Tables
example output from the "linear model"  (`lm`) which we store in the variable `ols` for ordinary least squares (by contrast weighted least squares treats the vertical displacements with unequally with differing weights.)
"""

# ╔═╡ 0489e5d8-51ca-4955-83e1-95ea353d9cf2
ols = lm(@formula(°C ~ °F), noisy_data)

# ╔═╡ 9a65aee4-ab8e-4ab7-be6f-cc2a2e9d5127
noisy_data

# ╔═╡ c3539f42-6ca7-47fb-9707-4d11c9e76643
md"""
This lecture is about explaining the meaning and significance to every part of this table.
"""

# ╔═╡ 469d809f-424f-4595-ad43-a5b2cc055304
md"""
# Regression a few ways
"""

# ╔═╡ 6128b8fd-9b85-4896-a0bf-934a0733fafb
md"""
## The "Coef." column in the table gives the slope and intercept of the best fit line
"""

# ╔═╡ 9eb7caaa-438d-4bcb-9c54-4a0fa72c61de
b, m = [ one.(x) x]\ yy  # The mysterious linear algebra solution using "least squares"

# ╔═╡ 0e8fce45-f1c0-41d4-996a-d6093182afee
function linear_regression(x,y)   # a direct computation from the data
	n = length(x)
	x0  = x.-mean(x)
	y0 = y.-mean(y)

	mᵉ = sum( x0 .* y0 ) / sum(  x0.^2 ) # slope estimate
	bᵉ = mean(y) - mᵉ * mean(x) # intercept estimate

	s2ᵉ = sum(  (mᵉ.*x .+ bᵉ .- y).^2 ) /(n-2) # noise estimate
	bᵉ,mᵉ,s2ᵉ
end

# ╔═╡ 71590890-38b6-440e-b61b-ece6c49ac602
linear_regression(x,yy)

# ╔═╡ f7cc7146-9ee6-4d87-b024-2a91863f4b24
md"""
[So why is it called "Regression" anyway?](http://blog.minitab.com/blog/statistics-and-quality-data-analysis/so-why-is-it-called-regression-anyway) Dalton's original meaning not quite what it means today.
"""

# ╔═╡ f64815e2-44b8-4585-9269-9a62655c984c
md"""
# Demystifying the word "Model"

    Step I:  The Model is y = m*x + b + σ*randn() .
    This means that out there in the real world are b, m, and σ.  You
    don't know them.

    Step II: You do, however, have data points x and y which allow you
    to compute an bᵉ,  mᵉ, and σᵉ.  A statistician would call these estimates
    based on your data points. If you ran the experiment again, you would
    get different data points.

    The computer lets us run the experiment as many times as we want just to see what happens.

        In summary, there are three kinds of variables.  The model variables b, m, and σ which are unknown.  The predictor variable x which is considered fixed and known.  The response variable y which is considered noisy.
"""

# ╔═╡ feb3c45e-88f4-4ffc-a4a0-e89489187c8d
md"""
## Understanding the relationship `°C ~ 1 + °F`
"""

# ╔═╡ 99069dd7-e088-4626-aa29-e48d6f9a474e
ols

# ╔═╡ 051a9e38-9a84-4ead-96fa-24c86c2b9f2d
md"""
`°C ~ 1 + °F` means the celsius (y) is (Coef1)*1 + (Coef2)*(°F),

in general `y ~ 1 + x1 + x2 + x3` is shorthand for
``y = c_0 + c_1 x_1 + c_2 x_2 + c_3 x_3``, etc.
"""

# ╔═╡ 2f33ee51-0725-46c2-9f1b-a61cd68abab1
md"""
# Simulating the real world: running many noisy models
"""

# ╔═╡ e4acd97b-22f7-4812-9898-1a485887a5f2
function simulate(σ,howmany)
	[linear_regression(x,y .+ σ * randn(length(x)))   for i=1:howmany]
	#[linear_regression(x,y .+ (σ * sqrt(12)) * (-.5 .+ rand(length(x))))   for i=1:howmany]
	# [linear_regression(x,y .+ (σ ) * ( rand([-1,1],length(x))))   for i=1:howmany]

end

# ╔═╡ 4e413b40-81c4-4160-9d01-046c2d179a06
howmany = 100_000

# ╔═╡ 7b94db0d-f46b-4621-9413-1dc787ae9a39
md"""
## Julia: underscore as a digits separator
"""

# ╔═╡ 51a28b67-ad64-4cf2-a0e6-a78fb101eb15
s = simulate(σ, howmany)

# ╔═╡ d451af49-3139-4329-a885-a210b1760f74
s[1] # first simulation,  intercept, slope, estimation of noise σ

# ╔═╡ c7455f7a-9c72-42f5-8238-1799cad96f6c
md"""
## Simulated intercepts ($howmany simulations)
"""

# ╔═╡ d2971801-2cdb-4b9f-8ec8-c74cbb2a0b31
md"""
σ = $(@bind σ Slider(0:.1:3, show_value=true, default=1))
"""

# ╔═╡ e1e8c140-bc4e-400d-beb2-0986e071c3a3
begin
	histogram( first.(s) , alpha=.6, bins=100, norm=true)
	vline!([-17.777777],color=:white)
	title!("intercept")
	xlims!(-17.7777-3,-17.7777+3)
	ylims!(0,1)
	plot!(legend=false)

end

# ╔═╡ 1429be09-a31f-415f-9c3d-f32b085ef68d
md"""
Experimental mean of the intercept
"""

# ╔═╡ da321202-0dc5-44ad-aac0-f3ea0d229243
mean(first.(s)), -17.777777

# ╔═╡ 2aceb366-a067-4271-9362-c320f4735ed1
md"""
Experimental std of the intercept
"""

# ╔═╡ 58f548fd-f6d0-479d-8469-bc886783f9a7
std( first.(s))

# ╔═╡ 07be9435-bc07-4a18-aad8-3ff19f5bcce4
md"""
Statisticians know an exact formula for the theoretical std of the intercept
"""

# ╔═╡ 1a6ad08d-c3bb-47e7-bdee-156bbff3aeda
    sb = σ * norm(x)  / norm(x.-mean(x)) / sqrt(n)

# ╔═╡ c55e4894-db71-4729-a1a1-5f68b45e3bf5
md"""
## Simulated slopes ($howmany simulations)
"""

# ╔═╡ f50d66eb-0357-4017-ac9b-99e63cd52dc0
begin
	histogram( getindex.(s,2), alpha=.6, bins=100, norm=true, legend=false )
	title!("slope")
	vline!([5/9],color=:white)
	xlims!(5/9-.1, 5/9+.1)
	ylims!(0,100)
end

# ╔═╡ 5c7a7361-f0e7-473a-9e38-226828aa00ca
md"""
Sample mean of the slope
"""

# ╔═╡ acf0e90e-8f1f-451f-9f0f-70a0bcc7efca
mean(getindex.(s,2)), .555555

# ╔═╡ c9f65e15-f222-4a88-98c2-9e1d8b5ec3eb
md"""
Sample std of the slope.
"""

# ╔═╡ 2589a369-8b21-406d-906d-71b18e4c7895
std( getindex.(s,2))

# ╔═╡ ed6a0e6a-2d0c-4f77-9b08-1a5b5d56dd34
md"""
Statisticians know a formula for the theoretical std of the slope.
"""

# ╔═╡ 61d1c1f7-e070-413b-8a92-76f44d237206
 σ  / norm(x.-mean(x))

# ╔═╡ 94d80ad6-0403-4322-aa9f-647c291c19d7
md"""
## Simulated σ ($howmany simulations)
"""

# ╔═╡ ce89b805-39a2-49e6-8781-c557aa73ed27
begin
	histogram( last.(s) ./ (σ^2/(n-2)) , alpha=.6, bins=100, norm=true,legend=false)
	vline!([1],color=:white)
	title!("residual")
	vline!([n-2],color=:white, lw=4)
	#xlims!(0,20)
	#ylims!(0,.13)
	plot!( x-> pdf(Chisq(n-2),x) , lw=4 , color=:red )
	plot!()

end

# ╔═╡ 75f9b5e9-775d-4767-9da6-222f977da686
mean( last.(s)  )

# ╔═╡ 797c9f2f-0b85-4435-b1c0-edc8cf67f738
σ^2

# ╔═╡ 6e0b2452-9f8b-4730-8072-a663704893c5
std(last.(s))

# ╔═╡ bf537a3a-b7c6-4c64-8b44-85511c3d492e
 (σ^2/ sqrt((n-2)/2))

# ╔═╡ 1340818c-3391-420b-aa94-acaea8a47d7d
md"""
# The Linear Model Table
"""

# ╔═╡ 829607ff-25e0-4585-9c5c-d132ecb86cc8
ols # = lm(@formula(°C ~ °F), noisy_data)

# ╔═╡ 3fc0a4a8-6719-4920-99c7-bd576225214e
-24.3784  / 19.0397

# ╔═╡ 24a7ad28-936c-47dc-bc53-d1ddbf39d05d
0.686156 / 0.330459

# ╔═╡ 9233dc6a-7578-4d72-b0c2-c3bb110a9fbe
md"""
## The Coef column is just the regression formula for the best line
"""

# ╔═╡ 07e02bb6-380d-40dd-86ad-19d713cd1657
mᵉ, bᵉ, σ²ᵉ =  linear_regression(x, yy)

# ╔═╡ b14593ba-cb8c-4f28-8fb0-2d2df479357b
md"""
## The Std. error column
"""

# ╔═╡ ac204681-b9df-471b-a22e-9d8f68679151
md"""
Above we saw that statisticians had formulas for the exact std of the slope and intercept:

 `std(intercept) = σ * norm(x)  / norm(x.-mean(x)) / sqrt(n)`

` std(slope) =  σ  / norm(x.-mean(x))`
"""

# ╔═╡ 08f43fff-fbd8-468f-8b3b-efd1829f4fc0
md"""
Let's replace σ with our estimate √σ²ᵉ
"""

# ╔═╡ 43ec6124-c3e5-4f34-b0d9-1a0b069aa3e0
sqrt(σ²ᵉ) * norm(x) / norm(x.-mean(x)) / sqrt(n)

# ╔═╡ 3fe71215-bbf2-40e9-bcfc-0bc9b3ac94c8
sqrt(σ²ᵉ) / norm(x.-mean(x))

# ╔═╡ a2b27841-256e-4898-aeca-04c4f44138fb
md"""
See those are the numbers in the magic table above.  I always love when I can reproduce the numbers myself.  It makes me feel I understand it.
"""

# ╔═╡ 8851dca3-e1a6-46b2-9745-f175ef0b0fae
md"""
## The t column
"""

# ╔═╡ ccfcb4d9-5a88-48fb-9568-1147a74f6eec
md"""
The t column, is simply the Coeff column divided by the Std. error column which we will use in a hypothesis test in the upcoming column.
"""

# ╔═╡ 13858c0a-3e7a-4742-a821-97dd9a45109d
md"""
### The t-distribution
"""

# ╔═╡ b2c3c1e5-e569-4c6f-bad9-055a25d73dce
md"""
In a statistics class you will likely see a random variable known as a t-distribution.
(with parameter k). It is the ratio of a standard normal to a χ distribution with parameter k. Let's just use `randn` to simulate. For the data sets of most of today's experiments, the normal distribution is close enough to t, that nobody needs to even use t much anymore.  In any event, with a t or a normal we are using this distribution because we are cognizant of the fact that the true σ is unknown and is merely being estimated.
"""

# ╔═╡ 305e4dfc-af7d-4667-8da8-a7ba5fd20fa6
rand_t(k) = sqrt(k)* randn() / norm( randn(k))

# ╔═╡ a648ba4f-fec4-4fa7-b328-1b52070224eb
md"""
k = $(@bind k Slider(3:100, show_value=true))
"""

# ╔═╡ d652df7d-7364-4da4-b51e-9fc88b978cda
begin

	histogram([rand_t(k) for i=1:100000], norm=true, bins=500, label=false)
	plot!( x-> pdf(TDist(k),x) , lw=4 , color=:red, label="t dist" )
	plot!( x->pdf(Normal(),x), color=:green, lw=2, label="normal dist")
	xlims!(-3, 3)
	ylims!(0, .4)
end

# ╔═╡ 2e530106-57a8-46a9-8f99-49a871d43255
md"""
## The Pr(>|t|)  column
is the area of the curve outside of the interval [-t,t].
"""

# ╔═╡ a990b133-ce50-4edf-81e1-1e78aeff8cd6
md"""
In statistics we ask if the coefficient ought to be considered 0 (which means in this case the data has no intercept or does not depend on x) or whether the coefficients are signficant with some probability.  The Pr(>|t|) column gives us the probablity that we should accept the hypothesis that the coefficients might reasonably be just 0.

In a proper statistical test, you should decide at what level you might be willing to accept the hypothesis, example .99, .95, or .9  might be a reasonable level, and if the test gives a smaller probability, you will accept that the coefficients are signficant.  It is not proper to produce the able and then decide whether to use .99, say.
"""

# ╔═╡ 3d0ea801-d66b-4e4e-90da-3a7dce28140d
md"""
# Degrees of Freedom
"""

# ╔═╡ 6fb223bb-f193-414d-9144-df180d09bea1
md"""
It is interesting to see that the sum of squares of a demeaned Gaussian vector is the size -1.  This is the reason for the (n-1) in the sample mean for variance.
"""

# ╔═╡ fb495ba4-52e6-4e0d-bd9c-981700edfebc
md"""
How many degrees of freedom are in a "demeaned" vector of normals?
"""

# ╔═╡ cdc4b25d-d05f-40c8-9c79-265876f01523

mean([ (v = randn(17);v.-=mean(v);sum(v.^2)) for i=1:1_000_000])

# ╔═╡ 967c5e3e-ab4c-45de-953c-aff6d16229af
md"""
If you ever wondered why the sample variance always has you dividing by (n-1)
and not n, this is the crux of the reason.
"""

# ╔═╡ Cell order:
# ╟─4ea0ccfa-9622-11eb-1cf0-e9ae2f927dd2
# ╠═d155ea12-9628-11eb-347f-7754a33fd403
# ╠═01506de2-918a-11eb-2a4d-c554a6e54631
# ╟─877deb2c-702b-457b-a54b-f27c277928d4
# ╟─36ce167f-382c-4b9a-be34-83250b10c4e5
# ╟─83912943-a847-420a-bfdb-450027b631e8
# ╠═280d112f-d34a-4cc4-9e3a-4ebbfcd5eb51
# ╠═b5031c96-db57-4baf-b271-6bb12e29de9b
# ╠═c2f77e8f-a8c0-4144-a8b4-b25dd98ed234
# ╟─ad161b98-f4a1-42ac-ad4f-8b71fabcfde9
# ╠═8e422886-74ef-4c0f-be1e-fda238c8db44
# ╟─ca21122a-2522-482a-b7ef-bd73e96cb5a9
# ╟─41e05b1e-8b5e-45e3-91bb-01355ade9f3d
# ╠═9d2e3861-ca36-406e-952d-831ca3947e44
# ╟─e73854ed-3581-41c4-ada5-e48242033759
# ╠═9873d944-b611-46f9-82a7-0cf714a3078c
# ╠═2be44753-afee-4125-b6bc-8866d2293dc2
# ╟─6e07e8fb-fe51-4b37-bfb2-d1466e768754
# ╟─a755e58a-b16c-4d3b-a85f-81ccf374793f
# ╟─f1e10fb7-adac-4083-8977-616a505fd591
# ╠═2e42986c-2de3-49e6-9c29-a7313c0b1da8
# ╟─22758dd6-9d04-4616-ba99-1430f2dedf9a
# ╠═aff6a616-6d8b-4584-a6f2-195decef7774
# ╠═338da13a-3c26-4366-a669-ac3e24f31577
# ╠═5a742546-1e4d-4aee-bed1-cb10c543e439
# ╟─6a9c8c9a-fac7-42f7-976d-3168132cae48
# ╟─3c038b68-8676-4877-9720-38da7c4e0e0e
# ╟─5a877e40-a101-4f7d-b2a1-ef4cfe5d8807
# ╟─83c28c76-2eab-49f9-9999-05df85054520
# ╟─ba671804-dc6d-415c-89de-9cf6294907b3
# ╟─e8683a71-5822-4491-9ccd-20e0fc3bf531
# ╠═0489e5d8-51ca-4955-83e1-95ea353d9cf2
# ╠═9a65aee4-ab8e-4ab7-be6f-cc2a2e9d5127
# ╟─c3539f42-6ca7-47fb-9707-4d11c9e76643
# ╟─469d809f-424f-4595-ad43-a5b2cc055304
# ╟─6128b8fd-9b85-4896-a0bf-934a0733fafb
# ╠═9eb7caaa-438d-4bcb-9c54-4a0fa72c61de
# ╠═0e8fce45-f1c0-41d4-996a-d6093182afee
# ╠═71590890-38b6-440e-b61b-ece6c49ac602
# ╟─f7cc7146-9ee6-4d87-b024-2a91863f4b24
# ╟─f64815e2-44b8-4585-9269-9a62655c984c
# ╟─feb3c45e-88f4-4ffc-a4a0-e89489187c8d
# ╠═99069dd7-e088-4626-aa29-e48d6f9a474e
# ╟─051a9e38-9a84-4ead-96fa-24c86c2b9f2d
# ╟─2f33ee51-0725-46c2-9f1b-a61cd68abab1
# ╠═e4acd97b-22f7-4812-9898-1a485887a5f2
# ╠═4e413b40-81c4-4160-9d01-046c2d179a06
# ╟─7b94db0d-f46b-4621-9413-1dc787ae9a39
# ╠═51a28b67-ad64-4cf2-a0e6-a78fb101eb15
# ╠═d451af49-3139-4329-a885-a210b1760f74
# ╟─c7455f7a-9c72-42f5-8238-1799cad96f6c
# ╟─d2971801-2cdb-4b9f-8ec8-c74cbb2a0b31
# ╠═e1e8c140-bc4e-400d-beb2-0986e071c3a3
# ╟─1429be09-a31f-415f-9c3d-f32b085ef68d
# ╠═da321202-0dc5-44ad-aac0-f3ea0d229243
# ╟─2aceb366-a067-4271-9362-c320f4735ed1
# ╠═58f548fd-f6d0-479d-8469-bc886783f9a7
# ╟─07be9435-bc07-4a18-aad8-3ff19f5bcce4
# ╠═1a6ad08d-c3bb-47e7-bdee-156bbff3aeda
# ╟─c55e4894-db71-4729-a1a1-5f68b45e3bf5
# ╠═f50d66eb-0357-4017-ac9b-99e63cd52dc0
# ╟─5c7a7361-f0e7-473a-9e38-226828aa00ca
# ╠═acf0e90e-8f1f-451f-9f0f-70a0bcc7efca
# ╟─c9f65e15-f222-4a88-98c2-9e1d8b5ec3eb
# ╠═2589a369-8b21-406d-906d-71b18e4c7895
# ╟─ed6a0e6a-2d0c-4f77-9b08-1a5b5d56dd34
# ╠═61d1c1f7-e070-413b-8a92-76f44d237206
# ╟─94d80ad6-0403-4322-aa9f-647c291c19d7
# ╠═ce89b805-39a2-49e6-8781-c557aa73ed27
# ╠═75f9b5e9-775d-4767-9da6-222f977da686
# ╠═797c9f2f-0b85-4435-b1c0-edc8cf67f738
# ╠═6e0b2452-9f8b-4730-8072-a663704893c5
# ╠═bf537a3a-b7c6-4c64-8b44-85511c3d492e
# ╟─1340818c-3391-420b-aa94-acaea8a47d7d
# ╠═829607ff-25e0-4585-9c5c-d132ecb86cc8
# ╠═3fc0a4a8-6719-4920-99c7-bd576225214e
# ╠═24a7ad28-936c-47dc-bc53-d1ddbf39d05d
# ╟─9233dc6a-7578-4d72-b0c2-c3bb110a9fbe
# ╠═07e02bb6-380d-40dd-86ad-19d713cd1657
# ╟─b14593ba-cb8c-4f28-8fb0-2d2df479357b
# ╟─ac204681-b9df-471b-a22e-9d8f68679151
# ╟─08f43fff-fbd8-468f-8b3b-efd1829f4fc0
# ╠═43ec6124-c3e5-4f34-b0d9-1a0b069aa3e0
# ╠═3fe71215-bbf2-40e9-bcfc-0bc9b3ac94c8
# ╟─a2b27841-256e-4898-aeca-04c4f44138fb
# ╟─8851dca3-e1a6-46b2-9745-f175ef0b0fae
# ╟─ccfcb4d9-5a88-48fb-9568-1147a74f6eec
# ╟─13858c0a-3e7a-4742-a821-97dd9a45109d
# ╟─b2c3c1e5-e569-4c6f-bad9-055a25d73dce
# ╠═305e4dfc-af7d-4667-8da8-a7ba5fd20fa6
# ╟─a648ba4f-fec4-4fa7-b328-1b52070224eb
# ╠═d652df7d-7364-4da4-b51e-9fc88b978cda
# ╟─2e530106-57a8-46a9-8f99-49a871d43255
# ╟─a990b133-ce50-4edf-81e1-1e78aeff8cd6
# ╟─3d0ea801-d66b-4e4e-90da-3a7dce28140d
# ╟─6fb223bb-f193-414d-9144-df180d09bea1
# ╟─fb495ba4-52e6-4e0d-bd9c-981700edfebc
# ╠═cdc4b25d-d05f-40c8-9c79-265876f01523
# ╟─967c5e3e-ab4c-45de-953c-aff6d16229af


#---------------------------------------------------------
# # [LazyGrids ndgrid](@id 1-ndgrid)
#---------------------------------------------------------

# This page explains the `ndgrid` method(s) in the Julia package
# [`LazyGrids`](https://github.com/JuliaArrays/LazyGrids.jl).

# ### Setup

# Packages needed here.

using LazyGrids: ndgrid, ndgrid_array
using LazyGrids: btime, @timeo # not exported; just for timing tests here
using BenchmarkTools: @benchmark
using InteractiveUtils: versioninfo

# ### Overview

# We begin with simple illustrations.

# The Julia method `ndgrid_array` in this package
# is comparable to Matlab's `ndgrid` function.
# It is given a long name here
# to discourage its use,
# because the lazy `ndgrid` version is preferable.
# The package provides `ndgrid_array`
# mainly for testing and timing comparisons.

(xa, ya) = ndgrid_array(1.0:3.0, 1:2)

#
xa

#
ya

# This package provides a "lazy" version of `ndgrid` that appears to the user
# to be the same, but under the hood it is not storing huge arrays.

(xl, yl) = ndgrid(1.0:3.0, 1:2)

#
xl

#
yl

# The following example illustrates the memory savings
# (thanks to Julia's powerful `AbstractArray` type):

(xl, yl) = ndgrid(1:100, 1:200)
(xa, ya) = ndgrid_array(1:100, 1:200)
sizeof(xl), sizeof(xa)

# One can do everything with a lazy array that one would expect
# from a "normal" array, e.g., multiplication and summation:

sum(xl * xl'), sum(xa * xa')

# ### Using lazy `ndgrid`

# Many applications with multiple variables
# involve evaluating functions over a *grid* of values.

# As a simple example (for illustration),
# one can numerically approximate the area of the unit circle
# by sampling that circle over a grid of x,y values,
# corresponding to numerical evaluation of the double integral
# ``∫ ∫ 1_{\{x^2 + y^2 < 1\}} \, dx \, dy``.
# There are many ways to implement this approximation in Julia,
# given a vector of `x` and `y` samples.

Δ = 1/2^10
x = range(-1, stop=1, step=Δ)
y = copy(x)

@inline circle(x::Real, y::Real) = abs2(x) + abs2(y) < 1
@inline circle(xy::NTuple{2}) = circle(xy...)

# The documentation below has many timing comparisons.
# The times in the Julia comments are on a 2017 iMac with Julia 1.6.1;
# the times printed out are whatever server GitHub actions uses.
# Using a trick to [capture output](https://fredrikekre.github.io/Literate.jl/v2/generated/example/#Output-Capturing),
# let's find out:

io = IOBuffer()
versioninfo(io)
split(String(take!(io)), '\n')

# A basic double loop is the C/Fortran way.
# It uses minimal memory (only 48 bytes) but is somewhat slow.

function method0(x,y) # basic double loop
    sum = 0.0
    for x in x, y in y
        sum += circle(x,y)
    end
    return sum * Δ^2
end

area0 = method0(x,y)
t = @benchmark method0($x,$y) # 10.5 ms (3 allocations: 48 bytes)
btime(t)

# The loop version does not look much like the math.
# It often seems natural to think of a grid of x,y values
# and simply sum over that grid, accounting for the grid spacing,
# using a function like this:

area(xx,yy) = sum(circle.(xx,yy)) * Δ^2

# Users coming from Matlab who are unfamiliar with its newer broadcast
# capabilities might use an `ndgrid` of arrays, like in the following code,
# to compute the area.
# But this array approach is much slower and uses much more memory,
# so it does not scale well to higher dimensions.

function area_array(x, y)
    (xa, ya) = ndgrid_array(x, y)
    return area(xa, ya)
end
@assert area_array(x, y) ≈ area0
t = @benchmark area_array($x, $y) # 21.4 ms (11 allocations: 64.57 MiB)
btime(t)

# To be fair, one might have multiple uses of the grids `xa,ya`
# so perhaps they should be excluded from the timing.
# Separating that allocation makes the timing look faster,
# but it still uses a lot of memory,
# both for allocating the grids, and for the `circle.` broadcast
# in the `area` function above:

(xa, ya) = ndgrid_array(x, y)
@assert area(xa, ya) ≈ area0
t = @benchmark area($xa, $ya) # 5.2 ms (7 allocations: 516.92 KiB)
btime(t)

# The main point of this package is to provide
# a lazy version of `ndgrid` that uses minimal memory.

(xl, yl) = ndgrid(x, y)
@assert xl == xa
@assert yl == ya
sizeof(xa), sizeof(xl)

# Now there is essentially no memory overhead for the grids,
# but memory is still used for the `circle.` broadcast.

@assert area(xl,yl) ≈ area0
t = @benchmark area($xl,$yl) # 3.7 ms (7 allocations: 516.92 KiB)
btime(t)

# Furthermore, creating this lazy ndgrid is so efficient
# that we can include its construction time
# and still have performance comparable to the array version
# that had pre-allocated arrays.

function area_lazy(x, y)
    (xl, yl) = ndgrid(x, y)
    return area(xl, yl)
end
@assert area_lazy(x, y) ≈ area0
t = @benchmark area_lazy($x, $y) # 3.7 ms (7 allocations: 516.92 KiB)
btime(t)

# ### More details

# The comparisons below here might be more
# for the curiosity of the package developers
# than for most users...

# One can preallocate memory to store the `circle.` array,
# to avoid additional memory during the area calculation:

out = Array{Float64}(undef, length(x), length(y))
function area!(xx, yy)
    global out .= circle.(xx,yy)
    return sum(out) * Δ^2
end
@assert area!(xl,yl) ≈ area0
t = @benchmark area!(xl,yl) # 4.8 ms (4 allocations: 128 bytes)
btime(t)

# Interestingly, the lazy version is *faster* than the array version,
# presumably because of the overheard of moving data from RAM to CPU:

@assert area!(xa,ya) ≈ area0
t = @benchmark area!(xa,ya) # 6.2 ms (4 allocations: 80 bytes)
btime(t)

# One can avoid allocating the output array by using a loop
# with [CartesianIndices](https://julialang.org/blog/2016/02/iteration):

function area_ci(xx, yy)
    size(xx) == size(yy) || throw("size")
    sum = 0.0
    @inbounds for c in CartesianIndices(xx)
        sum += circle(xx[c], yy[c])
    end
    return sum * Δ^2
end

# With this approach the lazy version is a bit faster
# than the array version:

@assert area_ci(xl,yl) ≈ area0
t = @benchmark area_ci(xl,yl) # 5.2 ms (3 allocations: 48 bytes)
btime(t)

#
@assert area_ci(xa,ya) ≈ area0
t = @benchmark area_ci(xa,ya) # 5.9 ms (3 allocations: 48 bytes)
btime(t)

# Alternatively one can use a linear index for loop,
# that also avoids the extra memory of `circle.` above,
# but is slower, especially for the lazy arrays
# that are optimized for Cartesian indexing:

function area_for2(xx,yy)
    size(xx) == size(yy) || throw("size")
    sum = 0.0
    @inbounds for i in 1:length(xx)
        sum += circle(xx[i], yy[i])
    end
    return sum * Δ^2
end
@assert area_for2(xa, ya) ≈ area0
t = @benchmark area_for2($xa, $ya) # 5.9 ms (3 allocations: 48 bytes)
btime(t)

#
@assert area_for2(xl, yl) ≈ area0
t = @benchmark area_for2($xl, $yl) # 15.4 ms (3 allocations: 48 bytes)
btime(t)

# Some Julia users would
# [recommend using broadcast](https://discourse.julialang.org/t/meshgrid-function-in-julia/48679/25).
# In this case, broadcast is reasonably fast, but still uses a lot of memory
# for the `circle.` output in the simplest implementation.

areab(x,y) = sum(circle.(x,y')) * Δ^2
@assert areab(x,y) ≈ area0
t = @benchmark areab($x,$y) # 11.6 ms (7 allocations: 516.92 KiB)
btime(t)

# Using `zip` can avoid the "extra" memory beyond the grids,
# but seems to have some undesirable overhead,
# presumably because `zip` uses linear indexing:

areaz(xa, ya) = sum(circle, zip(xa,ya)) * Δ^2
@assert areaz(xa, ya) ≈ area0
t = @benchmark areaz($xa, $ya) # 3.9 ms (3 allocations: 48 bytes)
btime(t)

#
@assert areaz(xl, yl) ≈ area0
t = @benchmark areaz($xl, $yl) # 12.2 ms (3 allocations: 48 bytes)
btime(t)

# One can also ensure low memory by using a product iterator,
# but the code starts to look pretty different from the math at this point
# and it is not much faster than broadcast here.

areap(x,y) = sum(circle, Iterators.product(x, y)) * Δ^2
@assert areap(x,y) ≈ area0
t = @benchmark areap($x, $y) # 9.9 ms (3 allocations: 48 bytes)
btime(t)

# ### 3D case

# A 3D example is finding (verifying) the volume of a unit sphere.

sphere(x::Real,y::Real,z::Real) = abs2(x) + abs2(y) + abs2(z) < 1
sphere(r::NTuple) = sum(abs2, r) < 1

# Storing three 3D arrays of size 2049^3 Float64 would take 192GB,
# so already we must greatly reduce the sampling to use either
# `broadcast` or `ndgrid_array`.
# Furthermore, the `broadcast` requires annoying `reshape` steps:

Δc = 1/2^8 # coarse grid
xc = range(-1, stop=1, step=Δc)
yc = xc
zc = xc
nc = length(zc)
3 * nc^3 * 8 / 1024^3 # GB prediction

# Here is broadcast in 3D (yuch!):

vol_br(x,y,z,Δ) = sum(sphere.(
        repeat(x, 1, length(y), length(z)),
        repeat(reshape(y, (1, :, 1)), length(x), 1, length(z)),
        repeat(reshape(z, (1, 1, :)), length(x), length(y), 1),
    )) * Δ^3
vol_br([0.],[0.],[0.],Δc) # warm-up
@timeo vol0 = vol_br(xc,yc,zc,Δc) # 2.7 sec, 3.0 GiB, roughly (4/3)π

function vol_ci(xx, yy, zz, Δ)
    size(xx) == size(yy) == size(zz) || throw("size")
    sum = 0.0
    @inbounds for c in CartesianIndices(xx)
        sum += sphere(xx[c], yy[c], zz[c])
    end
    return sum * Δ^3
end

# Here is the lazy version:
(xlc, ylc, zlc) = ndgrid(xc, yc, zc) # warm-up
@timeo (xlc, ylc, zlc) = ndgrid(xc, yc, zc); # 0.000022 sec (1.8 KiB)

#
vol_ci([0.], [0.], [0.], Δc) # warm-up
@timeo vol_ci(xlc, ylc, zlc, Δc) # 0.2 sec, 1.4 MiB

# Creating the grid of arrays itself is quite slow, even for the coarse grid:
(xac, yac, zac) = ndgrid_array(xc, yc, zc) # warm-up
@timeo (xac, yac, zac) = ndgrid_array(xc, yc, zc) # 1.8 sec 3.0GiB

# Once created, the array version is no faster than the lazy version:
@timeo vol_ci(xac, yac, zac, Δc) # 0.2 seconds (1.1 MiB)

# Using `zip` is more concise (but slower):
vol_zip(xx, yy, zz, Δ) = sum(sphere, zip(xx,yy,zz)) * Δ^3

#
vol_zip([0.], [0.], [0.], Δc) # warm-up
@timeo vol_zip(xlc, ylc, zlc, Δc) # 1.0 sec, 26 MiB

# Using zip for the array version seems to have less overhead
# so that is a potential for future improvement:

@assert vol_zip(xac, yac, zac, Δc) ≈ vol0
@timeo vol_zip(xac, yac, zac, Δc) # 0.19 sec, 16 byte

# Importantly, with the lazy ndgrid now we can return to the fine scale;
# it takes a few seconds, but it is feasible because of the low memory.
z = copy(x)
@timeo (xlf, ylf, zlf) = ndgrid(x, y, z) # 0.000023 sec

#
@timeo vol_ci(xlf, ylf, zlf, Δ) # 12.7 sec, 16 bytes

# I was hoping that with a lazy grid, now we could explore
# higher-dimensional spheres.  But with the current `zip` overhead
# it was too slow, even with coarse grid.
# @timeo (π^2/2, sum(sphere, zip(ndgrid(xc,xc,xc,xc)...)) * Δc^4)

# Probably I need to learn more about stuff like `pairs(IndexCartesian(), A)`
# [e.g., this PR](https://github.com/JuliaLang/julia/pull/38150).
# Another day...


using JuMP
using Clp

CARBS = 50
PROTEIN = 40
FAT = 60

m = Model(Clp.Optimizer)

@variable(m, steak ≥ 0)
@variable(m, potatoes ≥ 0)

@objective(m, Min, 8 * steak + 4 * potatoes)

@constraint(m, 5 * steak + 15 * potatoes ≥ CARBS)
@constraint(m, 20 * steak + 5 * potatoes ≥ PROTEIN)
@constraint(m, 15 * steak + 2 * potatoes ≤ FAT)

print(m)

optimize!(m)

status = termination_status(m)

println("Solution status: ", status)

println("Objective value: ", objective_value(m))
println("potatoes = ", value(steak))
println("steak = ", value(potatoes))


export RandomPatch

"""
    RandomPatch()

A random patch. For this we first draw entries of a vector `v` independently from a
complex normal distribution (`randn(ComplexF64)`). And then normalize `v` with respect
to the 2-norm.
"""
struct RandomPatch <: AbstractAffinePatch end

struct RandomPatchState{T,N} <: AbstractAffinePatchState{N}
    v::PVector{T,N}
end

is_global_patch(::RandomPatch) = true

function state(::RandomPatch, x::PVector)
    v = similar(x, ComplexF64)
    Random.randn!(v)
    LinearAlgebra.normalize!(v)
    RandomPatchState(v)
end

onpatch!(x::AbstractVector, state::RandomPatchState) = onpatch!(x, state.v)
evaluate!(u, state::RandomPatchState, x::PVector) = evaluate_patch!(u, state.v, x)
jacobian!(U, state::RandomPatchState, x::PVector) = jacobian_patch!(U, state.v, x)


### A Pluto.jl notebook ###
# v0.17.4

using Markdown
using InteractiveUtils

# ╔═╡ 0810c98f-2526-4ab4-9cd5-a662434e332d
begin
	using Statistics
	using StatsBase
	using Plots
end

# ╔═╡ a4283cac-c253-11ec-1d76-495287bbab72
md"""
# Bootstrap Method Notes

I'm working through Lorna Yen's article [An Introduction to the Bootstrap Method](https://towardsdatascience.com/an-introduction-to-the-bootstrap-method-58bcb51b4d60). The Bootstrap Method was introduced by Bradley Efron in 1979.

Date: 2022 April 22

## The Bootstrap Sampling Method

We need to make an inference about an estimate (e.g. sample mean) for a population parameter $$\theta$$ (e.g. population mean).

Basic Steps in a bootstrap method:

1. Start with a sample of size $$n$$.
2. Create $$B$$ Bootstrap Samples of size $$n$$ by drawing from the original sample with replacement.
3. Evaluate the statistic $$\theta$$ on each sample, giving us $$B$$ estimates.
4. Constrtuct a sampling distribution with these bootstrap statistics and use it to make further statistical inference, such as estimating the standard error for $$\theta$$ or obtaining a confidence interval for $$\theta$$.
"""

# ╔═╡ c33fde47-ac17-4194-973b-cdb277907b60
md"""
## Motivation - The Esimator's Standard Error

How can we evaluate the accuracy of an estimator?

Example: We want to esimate the number of times your 100 student pick up their phones in a day. 30 students installed an app which tells you how many times they picked up their phones. We will calculate the mean of these 30 students and get an estimate for pickups.
"""

# ╔═╡ e42b8dfb-1a75-448c-acc1-3255c87a0edf
pickups = rand(0:500, 100)

# ╔═╡ 0e14ff27-5d05-4367-947e-d97e96ddfa8e
md"""

The population mean is **$(mean(pickups))** pickups. Now, let's get our sample of 30 students. Note that this is performed without replacement.

"""

# ╔═╡ fbbc6413-0afb-4531-8113-bb85afd2710d
responses = sample(pickups, 30)

# ╔═╡ 0ed657f2-8f2b-4e8f-a375-9c8986448340
md"""

Our sample mean is **$(round(mean(responses); digits=2))** with standard deviation **$(round(std(responses); digits=2))**.

This is called a point estimate, but what we want is the true number of pickups in the whole lab.

An **estimator** is a rule for calculating an estimate. In this case, we are using sample mean, denoted $$\bar{X}$$. A **population parameter** is a numeric summary about a population. In this case, this is the average time of phone pickups per day, denoted $$\mu$$.

But how accurate is the estimate result? (How far is $$\bar{X}$$ from $$\mu$$?) The common measure is the standard error.
"""

# ╔═╡ c30c49a1-2821-4ada-ab10-5cac70a99e87
md"""
## Standard Error

**The standard error of an estimator is it's standard deviation.** Often times we do not know the parameters, and must use an estimated standard error.

### Standard Error of Sample Mean

$$\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}$$

where $$\sigma$$ is the standard deviation of the population and $$n$$ is the size of the sample. However, we do not know the standard devation of the population. Thus, we use estimated standard error, where we use the sample standard deviation $$S$$ in place of $$\sigma$$.

$$\sigma_{\bar{x}} \approx \frac{S}{\sqrt{n}}$$

where

$$S = \sum_{i=1}^n\frac{(x_i-\bar{x})^2}{n-1}$$

This means that for us, **$$S=$$ $(round(std(responses);digits=2))** and **$$\sigma_{\bar{x}}\approx$$ $(round(std(responses) / sqrt(length(responses));digits=2))**.

With the assumption that the estimator as a normal distribution, then we expect our estimate to be within $$\bar{x}\pm\sigma_{\bar{x}}$$ 68% of the time and within $$\bar{x}\pm2\sigma_{\bar{x}}$$ 95% of the time.
"""

# ╔═╡ 520524ab-53ef-4edb-bc58-f6a86620a126
md"""
### The Real World

Assumptions we have used so far:

 - We can easily estimate the standard error of our statistic of interest.
 - We assume the estimator follows a normal distribution.

But in the real world:

 - It is hard to know information about the population or it's distribution.
 - The standard error is hard to evaluate standard error in general. For instance, the standard error of the median is hard to know.

The bootstrap method helps us in these cases.
"""

# ╔═╡ fda6b303-a082-4f44-8341-49446b100609
md"""
## The Bootstrap Method Explained

### Scenario

Assume want to estimate the standard error of a statistic to make an inference about a population parameter, such as for construction the corresponding confidence interval. Assume further that:

 - We don't know anything about the population.
 - There is no precise formula for estimating the standard error of the statistic.

Let $$X_1, X_2, \dots, X_n$$ be a random sample from a population $$P$$ with distribution function $$F$$ Let $$M=g(X_1, X_2, \dots, X_n)$$ be our statistic for the parameter of interest. We want to know the variance of $$M$$,denoted $$Var(M)$$.

Challenges:

 - We can't determine the variance of the population since we don't know it. Thus, we need to estimate the variance and estimate the standard error.
 - In the real world, we don't always have a simple formula for evaluting the estimated variance or than sample means.

Before we get to esimating the variance, we'll introduce a common practical way, through simulation, assuming we know $$P$$.
"""

# ╔═╡ 4453da4c-12fc-427d-9f9c-89476bf65541
md"""
### Simulation

If we know $$P$$, we can generate $$B$$ random samples of size $$n$$, and calculate the statistic on each. Using these $$B$$ measurements, we can get a mean and a variance. The law of large numbers tells us the sample mean and variance of these $$B$$ statistics converges to the true variance as more samples are taken.

Below, we will use $$B=100000$$ for the phone problem above. First we'll look at  the distribution of the sample means, and then we will look at what the average mean converges to.
"""

# ╔═╡ f9153c34-d26b-4e54-847e-c3bae8cdb4df
sims = [mean(sample(pickups, 30; replace=false)) for x in 1:100000]

# ╔═╡ d273d1b8-e66c-4461-ab55-b2a4fe578d0c
histogram(sims, fillcolor=:black, plot_title="Histogram of Sample Means", legend = false)

# ╔═╡ 14b9e9b0-d67b-4376-80a5-cb0b76602174
convergence = [mean(sims[1:i]) for i in 1:100000]

# ╔═╡ ceb8a078-4c55-40e7-a1bf-41bc357eaf86
plot(convergence,
	linecolor=:black,
	plottitle="Convergence of Sample Means",
	xlabel="Number of Simulations",
	ylabel="Mean of first N Simulations",
	legend=false)

# ╔═╡ 3c1f21fb-be76-446d-b27f-54119ee4835a
md"""
In the distribution chart, we see that the means are normally distributed, and centered just below 250.

In the bottom chart, after running 1000000 experiments, we see that the mean quickly approaches a stable value. The final value is $(round(convergence[100000]; digits=2)) which is very close to the true value of $(round(mean(pickups);digits=2)).
"""

# ╔═╡ 1e05bcbc-cf0a-47ba-a1be-cb2e9bafff78
md"""
### The Empirical Distribution Function and Plug-in Principle

But we can't use simulation because we don't know $$P$$.

We will use the empirical distribution function to approximate the distribution function of the population and will apply the plug-in principle to get an estimate for Var(M), which is the plug-in estimator.

#### The Empirical Distribution Function

We'll build a CDF from data. The EDF is a discrete distribution that weights each point equally. It forms a step function that increases by $$1/n$$ at each data point. (The data points are placed on the x-axis and the y-value incrases by $$1/n$$ at each.)

#### The Statistical Functional

The bootstram method uses the EDF as an estimator for the CDF of the population. However, to apply the EDF as an estimator for our statistic $$M$$, we need to formulate M as a function of a CDF. Functions of CDFs are known as **statistical functionals**.

Examples:
If a random variable $$X$$ has CDF $$F$$ and PDF $$p(x)$$,

$$E(x) = \int x dF(x) = \int xp(x)dx,$$

noting that $$dF(x) = p(x)dx$$.

When $$X$$ is discrete, we instead have

$$E(X) = \int xdF(x) = \sum_x P(x)$$

How about Variance?

$$\sigma^2 = \int x^2dF(x) - (\int xdF(x))^2$$

What we're doing is rewriting statistics as functions of their CDFs. Thus we can rewrite our statistic as $$M=g(F)$$ for population with CDF $$F$$.

#### The Plug-In Principle

Combining the EDF and the Statistical Functional gives us the Plug-In Principle.

Simply plug in the EDF for the true CDF $$F$$, enabling us to evaluate $$M=g(F).$$

Formally, we say that for a parameter of interest $$\theta=g(F)$$, if $$\hat{F}$$ is an estimator for $$F$$, the plug-in estimator for $$\theta$$ is defined to be $$\hat{\theta} = g(\hat{F})$$.
"""

# ╔═╡ 56116619-fd06-4af1-9e6c-d78b8f166542
md"""
#### Plugging in the EDF for an unknown distribution
Say we have some statistic defined as follows

$$g(F) = \int r(x)dF(x)$$.

The EDF for our data is

$$\hat{F} = \frac{1}{n}\sum_{i=1}^n1(x_i\leq x)$$

Its PMF is

$$f(x) = \frac{1}{n}, x\in {\{x_1, \dots, x_n\}}$$

Thus, the esimator for $$M$$ is

$$\hat{M} = g(\hat{F}) = \int r(x)d\hat{F}(x).$$

Using our knowledge of the PDF (I guess), we can translate this into:

$$\hat{M} = \frac{1}{n}\sum_{i=1}^n r(x_i)$$.

Note that if we let $$r(x)=x$$, we get the plug-in estimator for the population mean, which is just the sample mean.

---
Ok, so we're almost there, but we still can't use this method to compute $$Var(\hat{M})$$, I assume because we can't make the jump to discrete like we did above. So we're returning to simulation.
"""

# ╔═╡ 06fef3c2-21cf-41d4-b645-6d3dbb75450c
md"""

#### Bootstrap Variance Estimation

Let's use simulation to approximate the variance of our estimator. The original procedure has us draw a sample from $$F$$, $$B$$ times. We don't have $$F$$, we'll use the EDF instead.

Steps for simulation:

1. Draw a sample from the EDF.
2. Obtain a plug-in statistic $$\hat{M} = g(\hat{F})$$
3. Replicate the two procedures $$B$$ times.
4. Get the variance in these $$B$$ statistics and use it to **approximate the true variance of the plug-in statistic**.
"""

# ╔═╡ a2a3d276-f33e-40e3-bf66-bfeeb0c99f95
md"""

**Drawing from the EDF**

By defintion of the EDF, each point is equally weighted. Thus, sampling from the EDF is just drawing n observations with replacement, as introduced at the beginning.
"""

# ╔═╡ 58ac0581-bde8-4940-9020-a738f695a2fd
md"""

## Assumptions, Applicability

Questions:

1. Will the variance result obtained from this process, $$S^2$$, be a good estimator for $$Var(\hat{M})$$?
2. Will $$Var(\hat{M})$$ be a good approximation of $$Var(M)$$?

$$S^2 = \frac{1}{B}\sum_{j=1}^B(\hat{M_j})^2 - (\frac{1}{B}\sum_{j=1}^B\hat{M_j})^2 \approx Var(\hat{M};\hat{F}) \approx Var(M;F)$$.

These are two distinct types of error. The difference between $$S^2$$ and $$Var(\hat{M})$$ is **simulation error** (the first approximate equality sign above). The difference between $$Var(\hat{M})$$ and $$Var(M)$$ is **approximation error** (the second approximate equality sign above).

Simulation error is improved with larger $$B$$.

Approximation error will be small if the following conditions are met:
1. The EDF must use enough data to approximate $$F$$ well.
2. There are some smoothness conditions on the statistic g.

1 is easy but 2 is concerning. The important thing is that variance does in fact meet these smoothness conditions so the bootstrap method can be used to estimate it.

## Implementation

Let's actually do the thing on the original problem.
"""

# ╔═╡ dd9e01ed-64c6-43cf-bde2-add77d470770
bootmeans = [mean(sample(responses, 30; replace=true)) for x in 1:100000]

# ╔═╡ 38c6b0b4-08a9-49ed-8268-1cdfd3861d2a
bootmean = mean(bootmeans)

# ╔═╡ fddd7c16-90b7-45dc-bcd0-8fbae1b64bb4
truemean = mean(pickups)

# ╔═╡ cb88575d-719b-44d6-aebe-e071e5a90069
bootmeanstd = sqrt(var(bootmeans))

# ╔═╡ 57c0deb6-feb6-4e79-bc1d-a57a6e678eda
truestderror = std(pickups) / (30^(.5))

# ╔═╡ 5b9bc0f6-c43d-4ea5-9ce6-2aa41d0ec74a
md"""
And that's it, we're done. We see that we have calculated an estimate for the standard deviation. The bootstrap simulation gave us **$(round(bootmeanstd;digits=2))** and the true standerd error for a sample size of 30 was **$(round(truestderror;digits=2))**.

Before, we needed to assume that the standard error of the sample was a good estimate for the standard deviation of the population, but we had to assume our estimator followed a normal distribution and that it was easy to calculate the standard error for our statistic. The bootstrap method let us sidestep that, and we only needed the following assumtions:
1. We need a smooth measure of error, formulated as a statistical functional (we used variance)
2. We needed a good estimate of $$F$$

**How can we reduce the error in our error estimate?**

With higher a higher sample size, (we used 30) we will have a better EDF and less approximation error. With more simulations, we will have less simulation error. This will give us a better estimate of the error.
"""

# ╔═╡ d9166184-8705-49c4-a7e9-0bba61adcaae
md"""

### Oh and one more thing...

Did we get a better mean? We had only one sample, which had a mean of $(mean(responses)). The bootstrap method provided us with $(round(bootmean;digits=2)), which is nearly identical. So we didn't use this to get a better mean, merely to to an estimate of the error in that measurement. Perhaps however, this would not be true with a different statistic. I don't know, this was just my own observation.
"""

# ╔═╡ e41995a5-1591-4404-b585-3c4e9aac49bf
md"""

## Thoughts

- How do we know if we have a good enough estimate of $$F$$?
- How do we apply this to statistical simulation of computer architecture?
  - Can we use this to get estimates on the error for our Markov models?
- How do we use this in an online fashion?

"""

# ╔═╡ 00000000-0000-0000-0000-000000000001
PLUTO_PROJECT_TOML_CONTENTS = """
[deps]
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
StatsBase = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"

[compat]
Plots = "~1.27.6"
StatsBase = "~0.33.16"
"""

# ╔═╡ 00000000-0000-0000-0000-000000000002
PLUTO_MANIFEST_TOML_CONTENTS = """
# This file is machine-generated - editing it directly is not advised

julia_version = "1.7.1"
manifest_format = "2.0"

[[deps.Adapt]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "af92965fb30777147966f58acb05da51c5616b5f"
uuid = "79e6a3ab-5dfb-504d-930d-738a2a938a0e"
version = "3.3.3"

[[deps.ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"

[[deps.Artifacts]]
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"

[[deps.Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[deps.Bzip2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "19a35467a82e236ff51bc17a3a44b69ef35185a2"
uuid = "6e34b625-4abd-537c-b88f-471c36dfa7a0"
version = "1.0.8+0"

[[deps.Cairo_jll]]
deps = ["Artifacts", "Bzip2_jll", "Fontconfig_jll", "FreeType2_jll", "Glib_jll", "JLLWrappers", "LZO_jll", "Libdl", "Pixman_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll", "Zlib_jll", "libpng_jll"]
git-tree-sha1 = "4b859a208b2397a7a623a03449e4636bdb17bcf2"
uuid = "83423d85-b0ee-5818-9007-b63ccbeb887a"
version = "1.16.1+1"

[[deps.ChainRulesCore]]
deps = ["Compat", "LinearAlgebra", "SparseArrays"]
git-tree-sha1 = "9950387274246d08af38f6eef8cb5480862a435f"
uuid = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
version = "1.14.0"

[[deps.ChangesOfVariables]]
deps = ["ChainRulesCore", "LinearAlgebra", "Test"]
git-tree-sha1 = "bf98fa45a0a4cee295de98d4c1462be26345b9a1"
uuid = "9e997f8a-9a97-42d5-a9f1-ce6bfc15e2c0"
version = "0.1.2"

[[deps.ColorSchemes]]
deps = ["ColorTypes", "Colors", "FixedPointNumbers", "Random"]
git-tree-sha1 = "12fc73e5e0af68ad3137b886e3f7c1eacfca2640"
uuid = "35d6a980-a343-548e-a6ea-1d62b119f2f4"
version = "3.17.1"

[[deps.ColorTypes]]
deps = ["FixedPointNumbers", "Random"]
git-tree-sha1 = "024fe24d83e4a5bf5fc80501a314ce0d1aa35597"
uuid = "3da002f7-5984-5a60-b8a6-cbb66c0b333f"
version = "0.11.0"

[[deps.Colors]]
deps = ["ColorTypes", "FixedPointNumbers", "Reexport"]
git-tree-sha1 = "417b0ed7b8b838aa6ca0a87aadf1bb9eb111ce40"
uuid = "5ae59095-9a9b-59fe-a467-6f913c188581"
version = "0.12.8"

[[deps.Compat]]
deps = ["Base64", "Dates", "DelimitedFiles", "Distributed", "InteractiveUtils", "LibGit2", "Libdl", "LinearAlgebra", "Markdown", "Mmap", "Pkg", "Printf", "REPL", "Random", "SHA", "Serialization", "SharedArrays", "Sockets", "SparseArrays", "Statistics", "Test", "UUIDs", "Unicode"]
git-tree-sha1 = "b153278a25dd42c65abbf4e62344f9d22e59191b"
uuid = "34da2185-b29b-5c13-b0c7-acf172513d20"
version = "3.43.0"

[[deps.CompilerSupportLibraries_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"

[[deps.Contour]]
deps = ["StaticArrays"]
git-tree-sha1 = "9f02045d934dc030edad45944ea80dbd1f0ebea7"
uuid = "d38c429a-6771-53c6-b99e-75d170b6e991"
version = "0.5.7"

[[deps.DataAPI]]
git-tree-sha1 = "cc70b17275652eb47bc9e5f81635981f13cea5c8"
uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a"
version = "1.9.0"

[[deps.DataStructures]]
deps = ["Compat", "InteractiveUtils", "OrderedCollections"]
git-tree-sha1 = "3daef5523dd2e769dad2365274f760ff5f282c7d"
uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
version = "0.18.11"

[[deps.DataValueInterfaces]]
git-tree-sha1 = "bfc1187b79289637fa0ef6d4436ebdfe6905cbd6"
uuid = "e2d170a0-9d28-54be-80f0-106bbe20a464"
version = "1.0.0"

[[deps.Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[deps.DelimitedFiles]]
deps = ["Mmap"]
uuid = "8bb1440f-4735-579b-a4ab-409b98df4dab"

[[deps.Distributed]]
deps = ["Random", "Serialization", "Sockets"]
uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b"

[[deps.DocStringExtensions]]
deps = ["LibGit2"]
git-tree-sha1 = "b19534d1895d702889b219c382a6e18010797f0b"
uuid = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae"
version = "0.8.6"

[[deps.Downloads]]
deps = ["ArgTools", "LibCURL", "NetworkOptions"]
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"

[[deps.EarCut_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "3f3a2501fa7236e9b911e0f7a588c657e822bb6d"
uuid = "5ae413db-bbd1-5e63-b57d-d24a61df00f5"
version = "2.2.3+0"

[[deps.Expat_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "bad72f730e9e91c08d9427d5e8db95478a3c323d"
uuid = "2e619515-83b5-522b-bb60-26c02a35a201"
version = "2.4.8+0"

[[deps.FFMPEG]]
deps = ["FFMPEG_jll"]
git-tree-sha1 = "b57e3acbe22f8484b4b5ff66a7499717fe1a9cc8"
uuid = "c87230d0-a227-11e9-1b43-d7ebe4e7570a"
version = "0.4.1"

[[deps.FFMPEG_jll]]
deps = ["Artifacts", "Bzip2_jll", "FreeType2_jll", "FriBidi_jll", "JLLWrappers", "LAME_jll", "Libdl", "Ogg_jll", "OpenSSL_jll", "Opus_jll", "Pkg", "Zlib_jll", "libass_jll", "libfdk_aac_jll", "libvorbis_jll", "x264_jll", "x265_jll"]
git-tree-sha1 = "d8a578692e3077ac998b50c0217dfd67f21d1e5f"
uuid = "b22a6f82-2f65-5046-a5b2-351ab43fb4e5"
version = "4.4.0+0"

[[deps.FixedPointNumbers]]
deps = ["Statistics"]
git-tree-sha1 = "335bfdceacc84c5cdf16aadc768aa5ddfc5383cc"
uuid = "53c48c17-4a7d-5ca2-90c5-79b7896eea93"
version = "0.8.4"

[[deps.Fontconfig_jll]]
deps = ["Artifacts", "Bzip2_jll", "Expat_jll", "FreeType2_jll", "JLLWrappers", "Libdl", "Libuuid_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "21efd19106a55620a188615da6d3d06cd7f6ee03"
uuid = "a3f928ae-7b40-5064-980b-68af3947d34b"
version = "2.13.93+0"

[[deps.Formatting]]
deps = ["Printf"]
git-tree-sha1 = "8339d61043228fdd3eb658d86c926cb282ae72a8"
uuid = "59287772-0a20-5a39-b81b-1366585eb4c0"
version = "0.4.2"

[[deps.FreeType2_jll]]
deps = ["Artifacts", "Bzip2_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "87eb71354d8ec1a96d4a7636bd57a7347dde3ef9"
uuid = "d7e528f0-a631-5988-bf34-fe36492bcfd7"
version = "2.10.4+0"

[[deps.FriBidi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "aa31987c2ba8704e23c6c8ba8a4f769d5d7e4f91"
uuid = "559328eb-81f9-559d-9380-de523a88c83c"
version = "1.0.10+0"

[[deps.GLFW_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libglvnd_jll", "Pkg", "Xorg_libXcursor_jll", "Xorg_libXi_jll", "Xorg_libXinerama_jll", "Xorg_libXrandr_jll"]
git-tree-sha1 = "51d2dfe8e590fbd74e7a842cf6d13d8a2f45dc01"
uuid = "0656b61e-2033-5cc2-a64a-77c0f6c09b89"
version = "3.3.6+0"

[[deps.GR]]
deps = ["Base64", "DelimitedFiles", "GR_jll", "HTTP", "JSON", "Libdl", "LinearAlgebra", "Pkg", "Printf", "Random", "RelocatableFolders", "Serialization", "Sockets", "Test", "UUIDs"]
git-tree-sha1 = "af237c08bda486b74318c8070adb96efa6952530"
uuid = "28b8d3ca-fb5f-59d9-8090-bfdbd6d07a71"
version = "0.64.2"

[[deps.GR_jll]]
deps = ["Artifacts", "Bzip2_jll", "Cairo_jll", "FFMPEG_jll", "Fontconfig_jll", "GLFW_jll", "JLLWrappers", "JpegTurbo_jll", "Libdl", "Libtiff_jll", "Pixman_jll", "Pkg", "Qt5Base_jll", "Zlib_jll", "libpng_jll"]
git-tree-sha1 = "cd6efcf9dc746b06709df14e462f0a3fe0786b1e"
uuid = "d2c73de3-f751-5644-a686-071e5b155ba9"
version = "0.64.2+0"

[[deps.GeometryBasics]]
deps = ["EarCut_jll", "IterTools", "LinearAlgebra", "StaticArrays", "StructArrays", "Tables"]
git-tree-sha1 = "83ea630384a13fc4f002b77690bc0afeb4255ac9"
uuid = "5c1252a2-5f33-56bf-86c9-59e7332b4326"
version = "0.4.2"

[[deps.Gettext_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "XML2_jll"]
git-tree-sha1 = "9b02998aba7bf074d14de89f9d37ca24a1a0b046"
uuid = "78b55507-aeef-58d4-861c-77aaff3498b1"
version = "0.21.0+0"

[[deps.Glib_jll]]
deps = ["Artifacts", "Gettext_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Libiconv_jll", "Libmount_jll", "PCRE_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "a32d672ac2c967f3deb8a81d828afc739c838a06"
uuid = "7746bdde-850d-59dc-9ae8-88ece973131d"
version = "2.68.3+2"

[[deps.Graphite2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "344bf40dcab1073aca04aa0df4fb092f920e4011"
uuid = "3b182d85-2403-5c21-9c21-1e1f0cc25472"
version = "1.3.14+0"

[[deps.Grisu]]
git-tree-sha1 = "53bb909d1151e57e2484c3d1b53e19552b887fb2"
uuid = "42e2da0e-8278-4e71-bc24-59509adca0fe"
version = "1.0.2"

[[deps.HTTP]]
deps = ["Base64", "Dates", "IniFile", "Logging", "MbedTLS", "NetworkOptions", "Sockets", "URIs"]
git-tree-sha1 = "0fa77022fe4b511826b39c894c90daf5fce3334a"
uuid = "cd3eb016-35fb-5094-929b-558a96fad6f3"
version = "0.9.17"

[[deps.HarfBuzz_jll]]
deps = ["Artifacts", "Cairo_jll", "Fontconfig_jll", "FreeType2_jll", "Glib_jll", "Graphite2_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Pkg"]
git-tree-sha1 = "129acf094d168394e80ee1dc4bc06ec835e510a3"
uuid = "2e76f6c2-a576-52d4-95c1-20adfe4de566"
version = "2.8.1+1"

[[deps.IniFile]]
git-tree-sha1 = "f550e6e32074c939295eb5ea6de31849ac2c9625"
uuid = "83e8ac13-25f8-5344-8a64-a9f2b223428f"
version = "0.5.1"

[[deps.InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[deps.InverseFunctions]]
deps = ["Test"]
git-tree-sha1 = "91b5dcf362c5add98049e6c29ee756910b03051d"
uuid = "3587e190-3f89-42d0-90ee-14403ec27112"
version = "0.1.3"

[[deps.IrrationalConstants]]
git-tree-sha1 = "7fd44fd4ff43fc60815f8e764c0f352b83c49151"
uuid = "92d709cd-6900-40b7-9082-c6be49f344b6"
version = "0.1.1"

[[deps.IterTools]]
git-tree-sha1 = "fa6287a4469f5e048d763df38279ee729fbd44e5"
uuid = "c8e1da08-722c-5040-9ed9-7db0dc04731e"
version = "1.4.0"

[[deps.IteratorInterfaceExtensions]]
git-tree-sha1 = "a3f24677c21f5bbe9d2a714f95dcd58337fb2856"
uuid = "82899510-4779-5014-852e-03e436cf321d"
version = "1.0.0"

[[deps.JLLWrappers]]
deps = ["Preferences"]
git-tree-sha1 = "abc9885a7ca2052a736a600f7fa66209f96506e1"
uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210"
version = "1.4.1"

[[deps.JSON]]
deps = ["Dates", "Mmap", "Parsers", "Unicode"]
git-tree-sha1 = "3c837543ddb02250ef42f4738347454f95079d4e"
uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
version = "0.21.3"

[[deps.JpegTurbo_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b53380851c6e6664204efb2e62cd24fa5c47e4ba"
uuid = "aacddb02-875f-59d6-b918-886e6ef4fbf8"
version = "2.1.2+0"

[[deps.LAME_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "f6250b16881adf048549549fba48b1161acdac8c"
uuid = "c1c5ebd0-6772-5130-a774-d5fcae4a789d"
version = "3.100.1+0"

[[deps.LERC_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "bf36f528eec6634efc60d7ec062008f171071434"
uuid = "88015f11-f218-50d7-93a8-a6af411a945d"
version = "3.0.0+1"

[[deps.LZO_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "e5b909bcf985c5e2605737d2ce278ed791b89be6"
uuid = "dd4b983a-f0e5-5f8d-a1b7-129d4a5fb1ac"
version = "2.10.1+0"

[[deps.LaTeXStrings]]
git-tree-sha1 = "f2355693d6778a178ade15952b7ac47a4ff97996"
uuid = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f"
version = "1.3.0"

[[deps.Latexify]]
deps = ["Formatting", "InteractiveUtils", "LaTeXStrings", "MacroTools", "Markdown", "Printf", "Requires"]
git-tree-sha1 = "6f14549f7760d84b2db7a9b10b88cd3cc3025730"
uuid = "23fbe1c1-3f47-55db-b15f-69d7ec21a316"
version = "0.15.14"

[[deps.LibCURL]]
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"

[[deps.LibCURL_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"

[[deps.LibGit2]]
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"

[[deps.LibSSH2_jll]]
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"

[[deps.Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[deps.Libffi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "0b4a5d71f3e5200a7dff793393e09dfc2d874290"
uuid = "e9f186c6-92d2-5b65-8a66-fee21dc1b490"
version = "3.2.2+1"

[[deps.Libgcrypt_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgpg_error_jll", "Pkg"]
git-tree-sha1 = "64613c82a59c120435c067c2b809fc61cf5166ae"
uuid = "d4300ac3-e22c-5743-9152-c294e39db1e4"
version = "1.8.7+0"

[[deps.Libglvnd_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll", "Xorg_libXext_jll"]
git-tree-sha1 = "7739f837d6447403596a75d19ed01fd08d6f56bf"
uuid = "7e76a0d4-f3c7-5321-8279-8d96eeed0f29"
version = "1.3.0+3"

[[deps.Libgpg_error_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "c333716e46366857753e273ce6a69ee0945a6db9"
uuid = "7add5ba3-2f88-524e-9cd5-f83b8a55f7b8"
version = "1.42.0+0"

[[deps.Libiconv_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "42b62845d70a619f063a7da093d995ec8e15e778"
uuid = "94ce4f54-9a6c-5748-9c1c-f9c7231a4531"
version = "1.16.1+1"

[[deps.Libmount_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "9c30530bf0effd46e15e0fdcf2b8636e78cbbd73"
uuid = "4b2f31a3-9ecc-558c-b454-b3730dcb73e9"
version = "2.35.0+0"

[[deps.Libtiff_jll]]
deps = ["Artifacts", "JLLWrappers", "JpegTurbo_jll", "LERC_jll", "Libdl", "Pkg", "Zlib_jll", "Zstd_jll"]
git-tree-sha1 = "c9551dd26e31ab17b86cbd00c2ede019c08758eb"
uuid = "89763e89-9b03-5906-acba-b20f662cd828"
version = "4.3.0+1"

[[deps.Libuuid_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "7f3efec06033682db852f8b3bc3c1d2b0a0ab066"
uuid = "38a345b3-de98-5d2b-a5d3-14cd9215e700"
version = "2.36.0+0"

[[deps.LinearAlgebra]]
deps = ["Libdl", "libblastrampoline_jll"]
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[[deps.LogExpFunctions]]
deps = ["ChainRulesCore", "ChangesOfVariables", "DocStringExtensions", "InverseFunctions", "IrrationalConstants", "LinearAlgebra"]
git-tree-sha1 = "a970d55c2ad8084ca317a4658ba6ce99b7523571"
uuid = "2ab3a3ac-af41-5b50-aa03-7779005ae688"
version = "0.3.12"

[[deps.Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[deps.MacroTools]]
deps = ["Markdown", "Random"]
git-tree-sha1 = "3d3e902b31198a27340d0bf00d6ac452866021cf"
uuid = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"
version = "0.5.9"

[[deps.Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[deps.MbedTLS]]
deps = ["Dates", "MbedTLS_jll", "Random", "Sockets"]
git-tree-sha1 = "1c38e51c3d08ef2278062ebceade0e46cefc96fe"
uuid = "739be429-bea8-5141-9913-cc70e7f3736d"
version = "1.0.3"

[[deps.MbedTLS_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"

[[deps.Measures]]
git-tree-sha1 = "e498ddeee6f9fdb4551ce855a46f54dbd900245f"
uuid = "442fdcdd-2543-5da2-b0f3-8c86c306513e"
version = "0.3.1"

[[deps.Missings]]
deps = ["DataAPI"]
git-tree-sha1 = "bf210ce90b6c9eed32d25dbcae1ebc565df2687f"
uuid = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28"
version = "1.0.2"

[[deps.Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"

[[deps.MozillaCACerts_jll]]
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"

[[deps.NaNMath]]
git-tree-sha1 = "737a5957f387b17e74d4ad2f440eb330b39a62c5"
uuid = "77ba4419-2d1f-58cd-9bb1-8ffee604a2e3"
version = "1.0.0"

[[deps.NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"

[[deps.Ogg_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "887579a3eb005446d514ab7aeac5d1d027658b8f"
uuid = "e7412a2a-1a6e-54c0-be00-318e2571c051"
version = "1.3.5+1"

[[deps.OpenBLAS_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "Libdl"]
uuid = "4536629a-c528-5b80-bd46-f80d51c5b363"

[[deps.OpenSSL_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "ab05aa4cc89736e95915b01e7279e61b1bfe33b8"
uuid = "458c3c95-2e84-50aa-8efc-19380b2a3a95"
version = "1.1.14+0"

[[deps.Opus_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "51a08fb14ec28da2ec7a927c4337e4332c2a4720"
uuid = "91d4177d-7536-5919-b921-800302f37372"
version = "1.3.2+0"

[[deps.OrderedCollections]]
git-tree-sha1 = "85f8e6578bf1f9ee0d11e7bb1b1456435479d47c"
uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
version = "1.4.1"

[[deps.PCRE_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b2a7af664e098055a7529ad1a900ded962bca488"
uuid = "2f80f16e-611a-54ab-bc61-aa92de5b98fc"
version = "8.44.0+0"

[[deps.Parsers]]
deps = ["Dates"]
git-tree-sha1 = "3b429f37de37f1fc603cc1de4a799dc7fbe4c0b6"
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
version = "2.3.0"

[[deps.Pixman_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b4f5d02549a10e20780a24fce72bea96b6329e29"
uuid = "30392449-352a-5448-841d-b1acce4e97dc"
version = "0.40.1+0"

[[deps.Pkg]]
deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"

[[deps.PlotThemes]]
deps = ["PlotUtils", "Statistics"]
git-tree-sha1 = "8162b2f8547bc23876edd0c5181b27702ae58dce"
uuid = "ccf2f8ad-2431-5c83-bf29-c5338b663b6a"
version = "3.0.0"

[[deps.PlotUtils]]
deps = ["ColorSchemes", "Colors", "Dates", "Printf", "Random", "Reexport", "Statistics"]
git-tree-sha1 = "bb16469fd5224100e422f0b027d26c5a25de1200"
uuid = "995b91a9-d308-5afd-9ec6-746e21dbc043"
version = "1.2.0"

[[deps.Plots]]
deps = ["Base64", "Contour", "Dates", "Downloads", "FFMPEG", "FixedPointNumbers", "GR", "GeometryBasics", "JSON", "Latexify", "LinearAlgebra", "Measures", "NaNMath", "Pkg", "PlotThemes", "PlotUtils", "Printf", "REPL", "Random", "RecipesBase", "RecipesPipeline", "Reexport", "Requires", "Scratch", "Showoff", "SparseArrays", "Statistics", "StatsBase", "UUIDs", "UnicodeFun", "Unzip"]
git-tree-sha1 = "6f2dd1cf7a4bbf4f305a0d8750e351cb46dfbe80"
uuid = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
version = "1.27.6"

[[deps.Preferences]]
deps = ["TOML"]
git-tree-sha1 = "47e5f437cc0e7ef2ce8406ce1e7e24d44915f88d"
uuid = "21216c6a-2e73-6563-6e65-726566657250"
version = "1.3.0"

[[deps.Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[deps.Qt5Base_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "Fontconfig_jll", "Glib_jll", "JLLWrappers", "Libdl", "Libglvnd_jll", "OpenSSL_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libxcb_jll", "Xorg_xcb_util_image_jll", "Xorg_xcb_util_keysyms_jll", "Xorg_xcb_util_renderutil_jll", "Xorg_xcb_util_wm_jll", "Zlib_jll", "xkbcommon_jll"]
git-tree-sha1 = "c6c0f690d0cc7caddb74cef7aa847b824a16b256"
uuid = "ea2cea3b-5b76-57ae-a6ef-0a8af62496e1"
version = "5.15.3+1"

[[deps.REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[[deps.Random]]
deps = ["SHA", "Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[deps.RecipesBase]]
git-tree-sha1 = "6bf3f380ff52ce0832ddd3a2a7b9538ed1bcca7d"
uuid = "3cdcf5f2-1ef4-517c-9805-6587b60abb01"
version = "1.2.1"

[[deps.RecipesPipeline]]
deps = ["Dates", "NaNMath", "PlotUtils", "RecipesBase"]
git-tree-sha1 = "dc1e451e15d90347a7decc4221842a022b011714"
uuid = "01d81517-befc-4cb6-b9ec-a95719d0359c"
version = "0.5.2"

[[deps.Reexport]]
git-tree-sha1 = "45e428421666073eab6f2da5c9d310d99bb12f9b"
uuid = "189a3867-3050-52da-a836-e630ba90ab69"
version = "1.2.2"

[[deps.RelocatableFolders]]
deps = ["SHA", "Scratch"]
git-tree-sha1 = "cdbd3b1338c72ce29d9584fdbe9e9b70eeb5adca"
uuid = "05181044-ff0b-4ac5-8273-598c1e38db00"
version = "0.1.3"

[[deps.Requires]]
deps = ["UUIDs"]
git-tree-sha1 = "838a3a4188e2ded87a4f9f184b4b0d78a1e91cb7"
uuid = "ae029012-a4dd-5104-9daa-d747884805df"
version = "1.3.0"

[[deps.SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"

[[deps.Scratch]]
deps = ["Dates"]
git-tree-sha1 = "0b4b7f1393cff97c33891da2a0bf69c6ed241fda"
uuid = "6c6a2e73-6563-6170-7368-637461726353"
version = "1.1.0"

[[deps.Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[deps.SharedArrays]]
deps = ["Distributed", "Mmap", "Random", "Serialization"]
uuid = "1a1011a3-84de-559e-8e89-a11a2f7dc383"

[[deps.Showoff]]
deps = ["Dates", "Grisu"]
git-tree-sha1 = "91eddf657aca81df9ae6ceb20b959ae5653ad1de"
uuid = "992d4aef-0814-514b-bc4d-f2e9a6c4116f"
version = "1.0.3"

[[deps.Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"

[[deps.SortingAlgorithms]]
deps = ["DataStructures"]
git-tree-sha1 = "b3363d7460f7d098ca0912c69b082f75625d7508"
uuid = "a2af1166-a08f-5f64-846c-94a0d3cef48c"
version = "1.0.1"

[[deps.SparseArrays]]
deps = ["LinearAlgebra", "Random"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"

[[deps.StaticArrays]]
deps = ["LinearAlgebra", "Random", "Statistics"]
git-tree-sha1 = "cd56bf18ed715e8b09f06ef8c6b781e6cdc49911"
uuid = "90137ffa-7385-5640-81b9-e52037218182"
version = "1.4.4"

[[deps.Statistics]]
deps = ["LinearAlgebra", "SparseArrays"]
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[[deps.StatsAPI]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "c82aaa13b44ea00134f8c9c89819477bd3986ecd"
uuid = "82ae8749-77ed-4fe6-ae5f-f523153014b0"
version = "1.3.0"

[[deps.StatsBase]]
deps = ["DataAPI", "DataStructures", "LinearAlgebra", "LogExpFunctions", "Missings", "Printf", "Random", "SortingAlgorithms", "SparseArrays", "Statistics", "StatsAPI"]
git-tree-sha1 = "8977b17906b0a1cc74ab2e3a05faa16cf08a8291"
uuid = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
version = "0.33.16"

[[deps.StructArrays]]
deps = ["Adapt", "DataAPI", "StaticArrays", "Tables"]
git-tree-sha1 = "57617b34fa34f91d536eb265df67c2d4519b8b98"
uuid = "09ab397b-f2b6-538f-b94a-2f83cf4a842a"
version = "0.6.5"

[[deps.TOML]]
deps = ["Dates"]
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"

[[deps.TableTraits]]
deps = ["IteratorInterfaceExtensions"]
git-tree-sha1 = "c06b2f539df1c6efa794486abfb6ed2022561a39"
uuid = "3783bdb8-4a98-5b6b-af9a-565f29a5fe9c"
version = "1.0.1"

[[deps.Tables]]
deps = ["DataAPI", "DataValueInterfaces", "IteratorInterfaceExtensions", "LinearAlgebra", "OrderedCollections", "TableTraits", "Test"]
git-tree-sha1 = "5ce79ce186cc678bbb5c5681ca3379d1ddae11a1"
uuid = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
version = "1.7.0"

[[deps.Tar]]
deps = ["ArgTools", "SHA"]
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"

[[deps.Test]]
deps = ["InteractiveUtils", "Logging", "Random", "Serialization"]
uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[[deps.URIs]]
git-tree-sha1 = "97bbe755a53fe859669cd907f2d96aee8d2c1355"
uuid = "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4"
version = "1.3.0"

[[deps.UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[[deps.Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"

[[deps.UnicodeFun]]
deps = ["REPL"]
git-tree-sha1 = "53915e50200959667e78a92a418594b428dffddf"
uuid = "1cfade01-22cf-5700-b092-accc4b62d6e1"
version = "0.4.1"

[[deps.Unzip]]
git-tree-sha1 = "34db80951901073501137bdbc3d5a8e7bbd06670"
uuid = "41fe7b60-77ed-43a1-b4f0-825fd5a5650d"
version = "0.1.2"

[[deps.Wayland_jll]]
deps = ["Artifacts", "Expat_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Pkg", "XML2_jll"]
git-tree-sha1 = "3e61f0b86f90dacb0bc0e73a0c5a83f6a8636e23"
uuid = "a2964d1f-97da-50d4-b82a-358c7fce9d89"
version = "1.19.0+0"

[[deps.Wayland_protocols_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4528479aa01ee1b3b4cd0e6faef0e04cf16466da"
uuid = "2381bf8a-dfd0-557d-9999-79630e7b1b91"
version = "1.25.0+0"

[[deps.XML2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "1acf5bdf07aa0907e0a37d3718bb88d4b687b74a"
uuid = "02c8fc9c-b97f-50b9-bbe4-9be30ff0a78a"
version = "2.9.12+0"

[[deps.XSLT_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgcrypt_jll", "Libgpg_error_jll", "Libiconv_jll", "Pkg", "XML2_jll", "Zlib_jll"]
git-tree-sha1 = "91844873c4085240b95e795f692c4cec4d805f8a"
uuid = "aed1982a-8fda-507f-9586-7b0439959a61"
version = "1.1.34+0"

[[deps.Xorg_libX11_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll", "Xorg_xtrans_jll"]
git-tree-sha1 = "5be649d550f3f4b95308bf0183b82e2582876527"
uuid = "4f6342f7-b3d2-589e-9d20-edeb45f2b2bc"
version = "1.6.9+4"

[[deps.Xorg_libXau_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4e490d5c960c314f33885790ed410ff3a94ce67e"
uuid = "0c0b7dd1-d40b-584c-a123-a41640f87eec"
version = "1.0.9+4"

[[deps.Xorg_libXcursor_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXfixes_jll", "Xorg_libXrender_jll"]
git-tree-sha1 = "12e0eb3bc634fa2080c1c37fccf56f7c22989afd"
uuid = "935fb764-8cf2-53bf-bb30-45bb1f8bf724"
version = "1.2.0+4"

[[deps.Xorg_libXdmcp_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4fe47bd2247248125c428978740e18a681372dd4"
uuid = "a3789734-cfe1-5b06-b2d0-1dd0d9d62d05"
version = "1.1.3+4"

[[deps.Xorg_libXext_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "b7c0aa8c376b31e4852b360222848637f481f8c3"
uuid = "1082639a-0dae-5f34-9b06-72781eeb8cb3"
version = "1.3.4+4"

[[deps.Xorg_libXfixes_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "0e0dc7431e7a0587559f9294aeec269471c991a4"
uuid = "d091e8ba-531a-589c-9de9-94069b037ed8"
version = "5.0.3+4"

[[deps.Xorg_libXi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll", "Xorg_libXfixes_jll"]
git-tree-sha1 = "89b52bc2160aadc84d707093930ef0bffa641246"
uuid = "a51aa0fd-4e3c-5386-b890-e753decda492"
version = "1.7.10+4"

[[deps.Xorg_libXinerama_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll"]
git-tree-sha1 = "26be8b1c342929259317d8b9f7b53bf2bb73b123"
uuid = "d1454406-59df-5ea1-beac-c340f2130bc3"
version = "1.1.4+4"

[[deps.Xorg_libXrandr_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll"]
git-tree-sha1 = "34cea83cb726fb58f325887bf0612c6b3fb17631"
uuid = "ec84b674-ba8e-5d96-8ba1-2a689ba10484"
version = "1.5.2+4"

[[deps.Xorg_libXrender_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "19560f30fd49f4d4efbe7002a1037f8c43d43b96"
uuid = "ea2f1a96-1ddc-540d-b46f-429655e07cfa"
version = "0.9.10+4"

[[deps.Xorg_libpthread_stubs_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "6783737e45d3c59a4a4c4091f5f88cdcf0908cbb"
uuid = "14d82f49-176c-5ed1-bb49-ad3f5cbd8c74"
version = "0.1.0+3"

[[deps.Xorg_libxcb_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "XSLT_jll", "Xorg_libXau_jll", "Xorg_libXdmcp_jll", "Xorg_libpthread_stubs_jll"]
git-tree-sha1 = "daf17f441228e7a3833846cd048892861cff16d6"
uuid = "c7cfdc94-dc32-55de-ac96-5a1b8d977c5b"
version = "1.13.0+3"

[[deps.Xorg_libxkbfile_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "926af861744212db0eb001d9e40b5d16292080b2"
uuid = "cc61e674-0454-545c-8b26-ed2c68acab7a"
version = "1.1.0+4"

[[deps.Xorg_xcb_util_image_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "0fab0a40349ba1cba2c1da699243396ff8e94b97"
uuid = "12413925-8142-5f55-bb0e-6d7ca50bb09b"
version = "0.4.0+1"

[[deps.Xorg_xcb_util_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll"]
git-tree-sha1 = "e7fd7b2881fa2eaa72717420894d3938177862d1"
uuid = "2def613f-5ad1-5310-b15b-b15d46f528f5"
version = "0.4.0+1"

[[deps.Xorg_xcb_util_keysyms_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "d1151e2c45a544f32441a567d1690e701ec89b00"
uuid = "975044d2-76e6-5fbe-bf08-97ce7c6574c7"
version = "0.4.0+1"

[[deps.Xorg_xcb_util_renderutil_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "dfd7a8f38d4613b6a575253b3174dd991ca6183e"
uuid = "0d47668e-0667-5a69-a72c-f761630bfb7e"
version = "0.3.9+1"

[[deps.Xorg_xcb_util_wm_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "e78d10aab01a4a154142c5006ed44fd9e8e31b67"
uuid = "c22f9ab0-d5fe-5066-847c-f4bb1cd4e361"
version = "0.4.1+1"

[[deps.Xorg_xkbcomp_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxkbfile_jll"]
git-tree-sha1 = "4bcbf660f6c2e714f87e960a171b119d06ee163b"
uuid = "35661453-b289-5fab-8a00-3d9160c6a3a4"
version = "1.4.2+4"

[[deps.Xorg_xkeyboard_config_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xkbcomp_jll"]
git-tree-sha1 = "5c8424f8a67c3f2209646d4425f3d415fee5931d"
uuid = "33bec58e-1273-512f-9401-5d533626f822"
version = "2.27.0+4"

[[deps.Xorg_xtrans_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "79c31e7844f6ecf779705fbc12146eb190b7d845"
uuid = "c5fb5394-a638-5e4d-96e5-b29de1b5cf10"
version = "1.4.0+3"

[[deps.Zlib_jll]]
deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"

[[deps.Zstd_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "e45044cd873ded54b6a5bac0eb5c971392cf1927"
uuid = "3161d3a3-bdf6-5164-811a-617609db77b4"
version = "1.5.2+0"

[[deps.libass_jll]]
deps = ["Artifacts", "Bzip2_jll", "FreeType2_jll", "FriBidi_jll", "HarfBuzz_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "5982a94fcba20f02f42ace44b9894ee2b140fe47"
uuid = "0ac62f75-1d6f-5e53-bd7c-93b484bb37c0"
version = "0.15.1+0"

[[deps.libblastrampoline_jll]]
deps = ["Artifacts", "Libdl", "OpenBLAS_jll"]
uuid = "8e850b90-86db-534c-a0d3-1478176c7d93"

[[deps.libfdk_aac_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "daacc84a041563f965be61859a36e17c4e4fcd55"
uuid = "f638f0a6-7fb0-5443-88ba-1cc74229b280"
version = "2.0.2+0"

[[deps.libpng_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "94d180a6d2b5e55e447e2d27a29ed04fe79eb30c"
uuid = "b53b4c65-9356-5827-b1ea-8c7a1a84506f"
version = "1.6.38+0"

[[deps.libvorbis_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Ogg_jll", "Pkg"]
git-tree-sha1 = "b910cb81ef3fe6e78bf6acee440bda86fd6ae00c"
uuid = "f27f6e37-5d2b-51aa-960f-b287f2bc3b7a"
version = "1.3.7+1"

[[deps.nghttp2_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"

[[deps.p7zip_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"

[[deps.x264_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4fea590b89e6ec504593146bf8b988b2c00922b2"
uuid = "1270edf5-f2f9-52d2-97e9-ab00b5d0237a"
version = "2021.5.5+0"

[[deps.x265_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "ee567a171cce03570d77ad3a43e90218e38937a9"
uuid = "dfaa095f-4041-5dcd-9319-2fabd8486b76"
version = "3.5.0+0"

[[deps.xkbcommon_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Wayland_jll", "Wayland_protocols_jll", "Xorg_libxcb_jll", "Xorg_xkeyboard_config_jll"]
git-tree-sha1 = "ece2350174195bb31de1a63bea3a41ae1aa593b6"
uuid = "d8fb68d0-12a3-5cfd-a85a-d49703b185fd"
version = "0.9.1+5"
"""

# ╔═╡ Cell order:
# ╟─0810c98f-2526-4ab4-9cd5-a662434e332d
# ╟─a4283cac-c253-11ec-1d76-495287bbab72
# ╟─c33fde47-ac17-4194-973b-cdb277907b60
# ╟─e42b8dfb-1a75-448c-acc1-3255c87a0edf
# ╟─0e14ff27-5d05-4367-947e-d97e96ddfa8e
# ╠═fbbc6413-0afb-4531-8113-bb85afd2710d
# ╟─0ed657f2-8f2b-4e8f-a375-9c8986448340
# ╟─c30c49a1-2821-4ada-ab10-5cac70a99e87
# ╟─520524ab-53ef-4edb-bc58-f6a86620a126
# ╟─fda6b303-a082-4f44-8341-49446b100609
# ╟─4453da4c-12fc-427d-9f9c-89476bf65541
# ╟─f9153c34-d26b-4e54-847e-c3bae8cdb4df
# ╟─d273d1b8-e66c-4461-ab55-b2a4fe578d0c
# ╟─14b9e9b0-d67b-4376-80a5-cb0b76602174
# ╟─ceb8a078-4c55-40e7-a1bf-41bc357eaf86
# ╟─3c1f21fb-be76-446d-b27f-54119ee4835a
# ╟─1e05bcbc-cf0a-47ba-a1be-cb2e9bafff78
# ╟─56116619-fd06-4af1-9e6c-d78b8f166542
# ╟─06fef3c2-21cf-41d4-b645-6d3dbb75450c
# ╟─a2a3d276-f33e-40e3-bf66-bfeeb0c99f95
# ╟─58ac0581-bde8-4940-9020-a738f695a2fd
# ╠═dd9e01ed-64c6-43cf-bde2-add77d470770
# ╠═38c6b0b4-08a9-49ed-8268-1cdfd3861d2a
# ╠═fddd7c16-90b7-45dc-bcd0-8fbae1b64bb4
# ╠═cb88575d-719b-44d6-aebe-e071e5a90069
# ╠═57c0deb6-feb6-4e79-bc1d-a57a6e678eda
# ╟─5b9bc0f6-c43d-4ea5-9ce6-2aa41d0ec74a
# ╟─d9166184-8705-49c4-a7e9-0bba61adcaae
# ╟─e41995a5-1591-4404-b585-3c4e9aac49bf
# ╟─00000000-0000-0000-0000-000000000001
# ╟─00000000-0000-0000-0000-000000000002


@testset "Data" begin

@test artificialInSARImage(2) == S1Point.(zeros(2,2))
@test artificialInSARImage(2,RnPoint) == RnPoint.(2*π*ones(2,2))
@test artificialInSARImage(2,SnPoint) == repeat([SnPoint([1.,0.])],2,2)

@test artificialS1SlopeSignal(20,0.) == repeat([S1Point(-π/2)],20)

@test ismissing( artificialS1Signal(-1.) )
@test ismissing( artificialS1Signal(2.) )
@test artificialS1Signal(2) == S1Point.([-3*π/4,-3*π/4])

# for the remainder check data types only
@test length( artificialS1Signal(20) ) == 20

@test size( artificialS2WhirlImage(64) ) == (64,64)
@test length(getValue(artificialS2WhirlImage(64)[1,1])) == 3
@test eltype( getValue(artificialS2WhirlImage(64)) ) == SnPoint{Float64}

@test size( artificialS2RotationsImage(64) ) == (64,64)
@test length(getValue(artificialS2RotationsImage(64)[1,1])) == 3
@test eltype( getValue(artificialS2RotationsImage(64)) ) == SnPoint{Float64}

@test size( artificialS2WhirlPatch(8) ) == (8,8)
@test length(getValue(artificialS2WhirlPatch(8)[1,1])) == 3
@test eltype( getValue(artificialS2WhirlPatch(8)) ) == SnPoint{Float64}

@test size( artificialSPDImage(8) ) == (8,8)
@test size(getValue(artificialSPDImage(8)[1,1])) == (3,3)
@test eltype( getValue(artificialSPDImage(8)) ) == SPDPoint{Float64}

@test size( artificialSPDImage2(8) ) == (8,8)
@test size(getValue(artificialSPDImage2(8)[1,1])) == (3,3)
@test eltype( getValue(artificialSPDImage2(8)) ) == SPDPoint{Float64}

@test length( artificialS2Lemniscate(SnPoint([0.,0.,1.]),20) ) == 20
@test length( getValue( artificialS2Lemniscate(SnPoint([0.,0.,1.]),20)[1])) == 3
@test eltype( artificialS2Lemniscate(SnPoint([0.,0.,1.]),20)) == SnPoint{Float64}

end


function linsolve_pcg( Lmat::SparseMatrixCSC{Float64,Int64},
                       prec::SparseMatrixCSC{Float64,Int64},
                       b::Array{Float64,1};
                       x0 = nothing,
                       NiterMax = 1000, TOL=5.e-10,
                       convmsg=false, showprogress=false )
    #
    Npoints = size(b)[1]
    if x0 == nothing
        x = zeros( Float64, Npoints )
    else
        x = copy(x0)
    end
    #
    r = zeros( Float64, Npoints )
    p = zeros( Float64, Npoints )
    z = zeros( Float64, Npoints )
    #
    nabla2_x = Lmat*x
    r = b - nabla2_x
    z = apply_prec_ilu0( prec, r)
    p = copy(z)

    rsold = dot( r, z )

    for iter = 1 : NiterMax
        #
        nabla2_x = Lmat*p
        #
        alpha = rsold/dot( p, nabla2_x )
        #
        x = x + alpha * p
        r = r - alpha * nabla2_x
        z = apply_prec_ilu0( prec, r)
        #
        rsnew = dot( z, r )
        # deltars = rsold - rsnew
        if showprogress
            @printf("%8d %18.10e\n", iter, sqrt(abs(rsnew)))
        end
        #
        if sqrt(abs(rsnew)) < TOL
            if convmsg
                @printf("#Convergence achieved in linsolve_pcg: %8d iterations.\n", iter)
            end
            break
        end
        #
        p = z + (rsnew/rsold) * p
        #
        rsold = rsnew
    end
    #
    return x
    #
end # of function


function trinary_to_decimal(str)
    dec = 0
    for (i, ch) in enumerate(reverse(str))
        d = Int(ch) - 48
        if !(0 <= d <= 2)
            return 0
        end
        dec += d * 3^(i - 1)
    end
    return dec
end


n = rand(1:10)
m = n + rand(0:5)
A = rand(m, n)

r = polarfact(A, alg = :svd);

# Test unitary matrix U

U = r.U
H = r.H

@test_approx_eq_eps U'*U eye(n) 1e-7

# Test Hermitian positive semifefinite matrix H

@test issym(H)

for i in eigvals(H)
    @test i >= 0.
end

@test_approx_eq A U*H

println("SVD method passed test...")


import Base.randperm

"The path graph on n vertices"
function pathGraph(n::Int64)
  x = append!(collect(1:(n-1)), collect(2:n))
  y = append!(collect(2:n), collect(1:(n-1)))
  w = ones(2 * (n - 1))
  return sparse(x, y, w)
end # pathGraph

"The complete graph"
function completeGraph(n::Int64)
  return sparse(ones(n,n) - eye(n))
end # completeGraph

"""The simple ring on n vertices"""
function ringGraph(n::Int64)
    a = spdiagm(ones(n-1),1,n,n)
    a[1,n] = 1
    a = a + a'
end

"""A generalization of a ring graph.
The vertices are integers modulo n.
Two are connected if their difference is in gens.
For example,

```
generalizedRing(17, [1 5])
```
"""
function generalizedRing(n::Int64, gens)
    k = length(gens)
    m = 2*n*k
    ai = zeros(Int64,m)
    aj = zeros(Int64,m)
    ind = 1
    for i in 0:(n-1)
        for j in 1:k
            ai[ind] = i
            aj[ind] = mod(i+gens[j],n)
            ind = ind + 1
            ai[ind] = i
            aj[ind] = mod(i-gens[j],n)
            ind = ind + 1
        end
    end
    return sparse(1+ai,1+aj,ones(m),n,n)
    #return ai, aj
end

"""A random generalized ring graph of degree k.
Gens always contains 1, and the other k-1 edge types
are chosen from an exponential distribution"""
function randGenRing(n::Int64, k::Integer)
    # if any of n, 2n, 3n etc. is in gens we will have self loops
    gens = [0]
    while 0 in (gens % n)
        gens = [1; 1 + ceil(Integer,exp(rand(k-1)*log(n-1)))]
    end

    return generalizedRing(n, gens)
end

"""
~~~
generalizedNecklace{Tv, Ti}(A::SparseMatrixCSC{Tv, Ti}, H::SparseMatrixCSC, k::Int64)
~~~

 Constructs a generalized necklace graph starting with two graphs A and H. The
resulting new graph will be constructed by expanding each vertex in H to an
instance of A. k random edges will be generated between components. Thus, the
resulting graph may have weighted edges.
"""

function generalizedNecklace{Tv, Ti}(A::SparseMatrixCSC{Tv, Ti}, H::SparseMatrixCSC, k::Int64)
  a = findnz(A)
  h = findnz(H)

  # these are square matrices
  n = A.n
  m = H.n

  newI = Ti[]
  newJ = Ti[]
  newW = Tv[]

  # duplicate the vertices in A so that each vertex in H corresponds to a copy of A
  for i in 1:m
    newI = append!(newI, a[1] + n * (i - 1))
    newJ = append!(newJ, a[2] + n * (i - 1))
    newW = append!(newW, a[3])
  end

  # for each edge in H, add k random edges between two corresponding components
  # multiedges will be concatenated to a single edge with higher cost
  for i in 1:length(h[1])
    u = h[1][i]
    v = h[2][i]

    if (u < v)
      #component x is from 1 + (x - 1) * n to n + (x - 1) * n
      for edgeToAdd in 1:k
        newU = rand(1:n) + n * (u - 1)
        newV = rand(1:n) + n * (v - 1)
        append!(newI, [newU, newV])
        append!(newJ, [newV, newU])
        append!(newW, [1, 1])
      end
    end
  end

  return sparse(newI, newJ, newW)
end # generalizedNecklace

"""The d dimensional hypercube.  Has 2^d vertices"""
function hyperCube(d::Int64)
  a = sparse([0 1; 1 0])

  for i = 1:(d-1)
    k = 2^i
    D = speye(k)
    a = [a D; D a]
  end

  return a
end # hyperCube

"""The complete binary tree on n vertices"""
function completeBinaryTree(n::Int64)

  k = div(n-1,2)
  a = sparse(collect(1:k),2*collect(1:k),1,n,n) + sparse(collect(1:k),2*collect(1:k)+1,1,n,n)

  if 2*k+1 < n
    a[n-1,n] = 1
  end

  a = a + a'

  return a
end # completeBinaryTree

""" An n by n grid with random weights. User can specify the weighting scheme. """
function wGrid2(n::Int64; weightGen::Function=rand)
    gr2 = sparse(grid2(n));

    gr2.nzval = Float64[weightGen() for i in 1:nnz(gr2)]

    # symmetrize
    gr2 = tril(gr2) + tril(gr2)'

    return gr2
end

""" An n^3 grid with random weights. User can specify the weighting scheme. """
function wGrid3(n::Int64; weightGen::Function=rand)
    gr2 = grid2(n);

    a = kron(speye(n), gr2);
    b = kron(gr2, speye(n));

    gr3 = sparse(a + b);
    gr3.nzval = Float64[weightGen() for i in 1:nnz(gr3)]

    # symmetrize
    gr3 = tril(gr3) + tril(gr3)'

    return gr3
end

"""An n-by-m grid graph.  iostropy is the weighting on edges in one direction."""
function grid2(n::Int64, m::Int64; isotropy=1)
  a = kron(speye(n),spdiagm(ones(m-1),1,m,m))
  a = a + isotropy*kron(spdiagm(ones(n-1),1,n,n), speye(m))
  a = a + a'
  return a
end # grid2

grid2(n::Int64) = grid2(n,n)

"""Coordinates for plotting the vertices of the n-by-m grid graph"""
function grid2coords(n::Int64, m::Int64)
  x = kron(collect(1:n),ones(m))
  y = kron(ones(n),collect(1:m))
  return x, y
end # grid2coords

grid2coords(n) = grid2coords(n, n)

"""A random matching on n vertices"""
function randMatching(n::Int64)

  p = randperm(n)
  n1 = convert(Int64,floor(n/2))
  n2 = 2*n1
  a = sparse(p[1:n1],p[(n1+1):n2],ones(n1),n,n)

  a = a + a'

  return a

end # randMatching

"""A sum of k random matchings on n vertices"""
function randRegular(n::Int64, k::Int64)
  a = randMatching(n)
  for i in 2:k
    a = a + randMatching(n)
  end

  return a
end # randRegular

"""Create a graph on n vertices.
For each vertex, give it k edges to randomly chosen prior
vertices.
This is a variety of a preferential attachment graph.
"""
function grownGraph(n::Int64, k::Int64)
  a = spzeros(n,n)

  for i = 1:k
    a = a + sparse(2:n,ceil(Integer,collect(1:n-1).*rand(n-1)),1,n,n)
  end

  a = a + a'
end # grownGraph

# used in grownGraphD
function randSet(n::Integer,k::Integer)
    if n == k
        return collect(1:n)
    elseif n < k
        error("n must be at least k")
    else

        s = sort(ceil(Integer,n*rand(k)))
        good = (minimum(s[2:end]-s[1:(end-1)]) > 0)
        while good == false
            s = sort(ceil(Integer,n*rand(k)))
            good = (minimum(s[2:end]-s[1:(end-1)]) > 0)
        end

        return s

    end
end

"""Like a grownGraph, but it forces the edges to all be distinct.
It starts out with a k+1 clique on the first k vertices"""
function grownGraphD(n::Int64, k::Int64)
    a = spzeros(n,n)

    u = zeros(Int64, k*(n-k-1))
    v = zeros(Int64, k*(n-k-1))

    for i in (k+2):n
        nb = randSet(i-1,k)
        u[(i-k-2)*k + collect(1:k)] = i
        v[(i-k-2)*k + collect(1:k)] = nb
    end

    a = sparse(u,v,1,n,n)

    (ai,aj) = findnz(triu(ones(k+1,k+1),1))
    a = a + sparse(ai,aj,1,n,n)
    a = a + a'

end # grownGraphD

"""A preferential attachment graph in which each vertex has k edges to those
that come before.  These are chosen with probability p to be from a random vertex,
and with probability 1-p to come from the endpoint of a random edge.
It begins with a k-clique on the first k+1 vertices."""
function prefAttach(n::Int64, k::Int64, p::Float64)
    if n == (k+1)
        return sparse(ones(Float64,n,n) - eye(Float64,n))
    elseif n <= k
        error("n must be more than k")
    else

        u = zeros(Int64,n*k)
        v = zeros(Int64,n*k)

        # fill in the initial clique
        # this will accidentally double every edge in the clique
        # we clean it up at the end
        ind = 1
        for i in 1:(k+1)
            for j in 1:(k+1)
                if i != j
                    u[ind] = i
                    v[ind] = j
                    ind += 1
                end
            end
        end

        s = zeros(Int64,k)
        for i in (k+2):n
            distinct = false
            while distinct == false
                for j in 1:k
                    if rand(Float64) < p
                        s[j] = rand(1:(i-1))
                    else
                        s[j] = v[rand(1:(k*(i-1)))]
                    end
                end
                s = sort(s)
                distinct = true
                for ii in 1:(k-1)
                    if s[ii] == s[ii+1]
                        distinct = false
                    end
                end
                # distinct = (minimum(s[2:end]-s[1:(end-1)]) > 0)

            end

            for j in 1:k
                u[ind] = i
                v[ind] = s[j]
                ind += 1
            end

        end # for i

        w = ones(Float64,n*k)

        w[1:(k*(k+1))] = 1/2

        a = sparse(u,v,w,n,n)
        a = a + a'
        return a
    end
end

"""Randomly permutes the vertex indices"""
function randperm(mat::AbstractMatrix)
    perm = randperm(mat.n)
    return mat[perm,perm]
end

randperm(f::Expr) = randperm(eval(f))

"""Generate a random graph on n vertices with m edges.
The actual number of edges will probably be smaller, as we sample
with replacement"""
function ErdosRenyi(n::Integer, m::Integer)
    ai = rand(1:n, m)
    aj = rand(1:n, m)
    ind = (ai .!= aj)
    mat = sparse(ai[ind],aj[ind],1,n,n)
    uniformWeight!(mat)
    return mat
end

"""Generate an ER graph with average degree k,
and then return the largest component.
Will probably have fewer than n vertices.
If you want to add a tree to bring it back to n,
try ErdosRenyiClusterFix."""
function ErdosRenyiCluster(n::Integer, k::Integer)
    m = ceil(Integer,n*k/2)
    ai = rand(1:n, m)
    aj = rand(1:n, m)
    ind = (ai .!= aj)
    mat = sparse(ai[ind],aj[ind],1,n,n)
    mat = mat + mat'

    return biggestComp(mat)
end

"""Like an Erdos-Renyi cluster, but add back a tree so
it has n vertices"""
function ErdosRenyiClusterFix(n::Integer, k::Integer)
    m1 = ErdosRenyiCluster(n, k)
    n2 = n - size(m1)[1]
    if (n2 > 0)
        m2 = completeBinaryTree(n2)
        return joinGraphs(m1,m2,1)
    else
        return m1
    end
end

"""Generate a random graph with n vertices from one of our natural distributions"""
function pureRandomGraph(n::Integer)

    gr = []
    wt = []

    push!(gr,:(pathGraph($n)))
    push!(wt,1)

    push!(gr,:(ringGraph($n)))
    push!(wt,3)

    push!(gr,:(completeBinaryTree($n)))
    push!(wt,3)

    push!(gr,:(grownGraph($n,2)))
    push!(wt,6)

    push!(gr,:(grid2(ceil(Integer,sqrt($n)))[1:$n,1:$n]))
    push!(wt,6)

    push!(gr,:(randRegular($n,3)))
    push!(wt,6)

    push!(gr,:(ErdosRenyiClusterFix($n,2)))
    push!(wt,6)

    if n >= 4
        push!(gr,:(randGenRing($n,4)))
        push!(wt,6)
    end

    i = sampleByWeight(wt)

    # make sure get a connected graph
    its = 0
    mat = eval(gr[i])

    while (~isConnected(mat)) && (its < 100)
        i = sampleByWeight(wt)
        mat = eval(gr[i])
        its += 1
    end
    if its == 100
        error("Getting a disconnected graph from $(gr[i])")
    end

    if (sum(diag(mat)) > 0)
        error("nonzero diag from $(gr[i])")
    end

    return floatGraph(mat)

end

"""sample an index with probability proportional to its weight given here"""
function sampleByWeight(wt)
    r = rand(1)*sum(wt)
    find(cumsum(wt) .> r)[1]
end

"""A Chimera graph with some weights.  The weights just appear when graphs are combined.
For more interesting weights, use `wtedChimera`"""
function semiWtedChimera(n::Integer)

    if (n < 2)
        gr = sparse([0.0])

        return randperm(gr)
    end

    r = rand()^2

    if (n < 30) || (rand() < .2)

        gr = pureRandomGraph(n)

        return randperm(gr)
    end

    if (n < 200)
        # just join disjoint copies of graphs

        n1 = 10 + floor(Integer,(n-20)*rand())
        n2 = n - n1
        k = ceil(Integer,exp(rand()*log(min(n1,n2)/2)))

        gr = joinGraphs(r*chimera(n1),chimera(n2),k)

        return randperm(gr)
    end

    # split with probability .7

    if (rand() < .7)
        n1 = ceil(Integer,10*exp(rand()*log(n/20)))

        n2 = n - n1
        k = floor(Integer,1+exp(rand()*log(min(n1,n2)/2)))

        gr = joinGraphs(r*chimera(n1),chimera(n2),k)

        return randperm(gr)

    else
        n1 = floor(Integer,10*exp(rand()*log(n/100)))

        n2 = floor(Integer, n / n1)

        if (rand() < .5)

            gr = productGraph(r*chimera(n1),chimera(n2))

        else

            k = floor(Integer,1+exp(rand()*log(min(n1,n2)/10)))
            gr = generalizedNecklace(r*chimera(n1),chimera(n2),k)

        end

        n3 = n - size(gr)[1]
        if (n3 > 0)
            gr = joinGraphs(gr,chimera(n3),2)

        end

        return randperm(gr)

    end
end

"""Builds a chimeric graph on n vertices.
The components come from pureRandomGraph,
connected by joinGraphs, productGraph and generalizedNecklace"""
function chimera(n::Integer)

    gr = semiWtedChimera(n)
    unweight!(gr)

    return gr

end

"""Builds the kth chimeric graph on n vertices.
It does this by resetting the random number generator seed.
It should captute the state of the generator before that and then
return it, but it does not yet."""
function chimera(n::Integer, k::Integer)
    srand(100*n+k)
    g = chimera(n)
    return g
end

"""Applies one of a number of random weighting schemes to the edges of the graph"""
function randWeight(a)

    if (rand() < .2)
        return a
    end

    n = a.n
    (ai,aj) = findnz(a)
    m = length(ai)

    # potentials or edge-based

    if (rand() < .3)
        w = rand(m)

    else
        v = randn(a.n)

        # mult by matrix ?
        if (rand() < .5)

            invdeg = spdiagm(1./(a*ones(size(a)[1])))
            if (rand() < .5)
                for i in 1:10
                    v = a * (invdeg * v)
                    v = v - mean(v)
                end
            else
                for i in 1:10
                    v = v - a * (invdeg * v)
                    v = v - mean(v)
                end
            end
        end

        w = abs(v[ai]-v[aj])

    end

    # reciprocate or not?

    w[w.==0] = 1
    w[isnan(w)] = 1

    if (rand() < .5)
        w = 1./w
    end

    w = w / mean(w)

    ar = sparse(ai,aj,w,n,n)
    ar = ar + ar';
    return ar
end

"""Builds the kth wted chimeric graph on n vertices.
It does this by resetting the random number generator seed.
It should captute the state of the generator before that and then
return it, but it does not yet."""
function wtedChimera(n::Integer, k::Integer)
    srand(100*n+k)
    g = wtedChimera(n)
    return g
end

function semiWtedChimera(n::Integer, k::Integer)
    srand(100*n+k)
    g = semiWtedChimera(n)
    return g
end

"""Generate a chimera, and then apply a random weighting scheme"""
function wtedChimera(n::Integer)
    return randWeight(semiWtedChimera(n))
end


using HomotopyContinuation
using Base.Test
import DynamicPolynomials
const PolyImpl = DynamicPolynomials

include("base_utilities_test.jl")

include("straight_line_test.jl")
include("gamma_trick_test.jl")
include("fixedhomotopy_test.jl")
include("spherical_test.jl")
include("affine_test.jl")
include("test_systems_test.jl")
include("cauchyendgame_test.jl")


using ..Galerkin:
    galerkin_matrix,
    galerkin_projection!,
    galerkin_projection  # for Documenter only...

using LinearAlgebra:
    cholesky!,
    Cholesky,
    ldiv!

@doc raw"""
    MinimiseL2Error <: AbstractApproxMethod

Approximate a given function ``f(x)`` by minimisation of the ``L^2`` distance
between ``f`` and its spline approximation ``g(x)``.

# Extended help

Minimises the ``L^2`` distance between the two functions:

```math
{\left\lVert f - g \right\rVert}^2 = \left< f - g, f - g \right>,
```

where

```math
\left< u, v \right> = ∫_a^b u(x) \, v(x) \, \mathrm{d}x
```

is the inner product between two functions, and ``a`` and ``b`` are the
boundaries of the prescribed B-spline basis.
Here, ``g`` is the spline ``g(x) = ∑_{i = 1}^N c_i \, b_i(x)``, and
``\{ b_i \}_{i = 1}^N`` is a prescribed B-spline basis.

One can show that the optimal coefficients ``c_i`` minimising the ``L^2`` error
are the solution to the linear system ``\bm{M} \bm{c} = \bm{φ}``,
where ``M_{ij} = \left< b_i, b_j \right>`` and ``φ_i = \left< b_i, f \right>``.
These two terms are respectively computed by [`galerkin_matrix`](@ref) and
[`galerkin_projection`](@ref).

The integrals associated to ``\bm{M}`` and ``\bm{φ}`` are computed via
Gauss--Legendre quadrature.
The number of quadrature nodes is chosen as a function of the order ``k`` of the
prescribed B-spline basis, ensuring that ``\bm{M}`` is computed exactly (see
also [`galerkin_matrix`](@ref)).
In the particular case where ``f`` is a polynomial of degree ``k - 1``, this
also results in an exact computation of ``\bm{φ}``.
In more general cases, as long as ``f`` is smooth enough, this is still expected
to yield a very good approximation of the integral, and thus of the optimal coefficients ``c_i``.

"""
struct MinimiseL2Error <: AbstractApproxMethod end

function approximate(f, B::AbstractBSplineBasis, m::MinimiseL2Error)
    T = typeof(f(first(knots(B))))
    S = Spline{T}(undef, B)
    M = galerkin_matrix(B)  # by default it's a BandedMatrix

    # We annotate the return type to avoid inference issue in ArrayLayouts...
    # https://github.com/JuliaMatrices/ArrayLayouts.jl/issues/66
    Mfact = cholesky!(M) :: Cholesky{eltype(M), typeof(parent(M))}

    data = (; M = Mfact)
    A = SplineApproximation(m, S, data)
    approximate!(f, A)
end

function _approximate!(f, A, m::MinimiseL2Error)
    @assert method(A) === m
    S = spline(A)
    cs = coefficients(S)
    galerkin_projection!(f, cs, basis(S))  # computes rhs onto cs
    ldiv!(data(A).M, cs)  # now cs = M \ rhs
    A
end