### A Pluto.jl notebook ###
# v0.18.4

using Markdown
using InteractiveUtils

# ╔═╡ 57c67334-a30d-11ec-3019-012043761a13
begin
	using Plots
	using Random
	import Distributions as di
	using Statistics
end

# ╔═╡ 287d1794-fe2c-49ba-ad5a-71a130ee14be
md"""
# Midterm BME 502 Spring 2022
put names of your group here.  A maxiumum of three students can be in a group.  The class is allowed to discuss the problem with each other.  I expect one report per group.  Don't copy from each other.
"""

# ╔═╡ cb547e46-51fb-47ae-97ed-482d89d9c5e5
md"""
# Problem 1 (30 Points)
Estimate Pi using random numbers.  The basic idea is to play darts on a 2x2 square in which a circle of radius one is placed.  You now throw random darts at this board and you have to figure out how to calculate ``\pi`` by counting how many darts hit the circle and how many darts miss.  You can use any Julia function that you like, but I want you to be conscious of computational and coding efficiency.  To illustrate your approach we want you to create a figure of the circle with the hits and misses indicated by different colors.  We provided a template.
Also, please calculate how accurately you can estimate Pi using this method given that you throw N darts (Remember what I told you about counting errors).
"""

# ╔═╡ b93fcb5e-64b3-4bda-abd4-ade9cedf56ca
function makeacircle(x1,y1,r)
    theta = LinRange(0,2*pi,500)
    return x1.+r*cos.(theta), y1.+r*sin.(theta)
end

# ╔═╡ 61d7c512-c0a0-4341-b97c-ffaf26944b6a
md"""
Here is some help with the figure.
All you need to do is to add the hits and misses with different colors
"""

# ╔═╡ 6aa0d42e-e778-48eb-a2a7-d923fa80daea
begin
	plot(makeacircle(1,1,1),seriestype = :shape,c = :blue,legend = false,fillalpha = 0.2,aspect_ratio = 1)
	# add random hits here
end

# ╔═╡ ab0da14c-2b52-44d6-a91d-795c669687fb
md"""
# Problem 2 (total 70 Points)
In the lecture on "least squares", we discussed the case where we used the wrong model (linear vs. quadratic).  We showed that in this case the chi-squared distribution does not follow the theoretical prediction but showed a different distribution.  Here I want to demonstrate that if your error is bimodal (as compared to Gaussian), and even if you are using the correct model function, the statistics do not agree with a least-squares fit.  This is not unsurprising but the subject of this midterm is to attempt to develop a Bayesian model to properly fit data that come from a bimodal distribution.
"""

# ╔═╡ 68c4f411-d5a0-48f0-bbba-8e6901b0408f
md"""
1) (30 points) Develop a maximum likelihood model for a bi-modal error distribution that is composed of two Gaussian distributions of width $\sigma$ at a y-distance of $\pm \Delta y$ around the real data point.  For this, you need to formulate the likelihood probability distribution of finding a particular dataset $\{y_{i}\}$ of N datapoints with this bi-modal error distribution assuming a “true” model $f(x_{i},\theta)$.  Here I expect an analytic expression.

2) (20 points) Develop a method to fit a straight line through actual data points taken from a line with added bi-modal errors as described in the paragraph above.  Do this by maximizing the likelihood above.  Can you develop a similar procedure as for Gaussian errors (taking the log of the likelihood turns the maximization into a minimization problem)? First try an analytical approach by taking the derivative and trying to find the maximum of the function.  If analytical math is too difficult you may approach the problem using numerical or approximate methods.

3) (20 points) Consider the same scenario as in (1), but instead of fitting to a single line, fit to two parallel lines using a least squares approach (fit parameters should be slope, intercept and Δy). Be very careful how you calculate chi-squared (each data point should only count once).  Please produce a similar graph (Out[7]) to confirm that the fit indeed behaves properly and fits the theoretical chi-squared distribution.
"""

# ╔═╡ 00000000-0000-0000-0000-000000000001
PLUTO_PROJECT_TOML_CONTENTS = """
[deps]
Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[compat]
Distributions = "~0.25.49"
Plots = "~1.27.0"
"""

# ╔═╡ 00000000-0000-0000-0000-000000000002
PLUTO_MANIFEST_TOML_CONTENTS = """
# This file is machine-generated - editing it directly is not advised

[[Adapt]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "af92965fb30777147966f58acb05da51c5616b5f"
uuid = "79e6a3ab-5dfb-504d-930d-738a2a938a0e"
version = "3.3.3"

[[ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"

[[Artifacts]]
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"

[[Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[Bzip2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "19a35467a82e236ff51bc17a3a44b69ef35185a2"
uuid = "6e34b625-4abd-537c-b88f-471c36dfa7a0"
version = "1.0.8+0"

[[Cairo_jll]]
deps = ["Artifacts", "Bzip2_jll", "Fontconfig_jll", "FreeType2_jll", "Glib_jll", "JLLWrappers", "LZO_jll", "Libdl", "Pixman_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll", "Zlib_jll", "libpng_jll"]
git-tree-sha1 = "4b859a208b2397a7a623a03449e4636bdb17bcf2"
uuid = "83423d85-b0ee-5818-9007-b63ccbeb887a"
version = "1.16.1+1"

[[Calculus]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "f641eb0a4f00c343bbc32346e1217b86f3ce9dad"
uuid = "49dc2e85-a5d0-5ad3-a950-438e2897f1b9"
version = "0.5.1"

[[ChainRulesCore]]
deps = ["Compat", "LinearAlgebra", "SparseArrays"]
git-tree-sha1 = "c9a6160317d1abe9c44b3beb367fd448117679ca"
uuid = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
version = "1.13.0"

[[ChangesOfVariables]]
deps = ["ChainRulesCore", "LinearAlgebra", "Test"]
git-tree-sha1 = "bf98fa45a0a4cee295de98d4c1462be26345b9a1"
uuid = "9e997f8a-9a97-42d5-a9f1-ce6bfc15e2c0"
version = "0.1.2"

[[ColorSchemes]]
deps = ["ColorTypes", "Colors", "FixedPointNumbers", "Random"]
git-tree-sha1 = "12fc73e5e0af68ad3137b886e3f7c1eacfca2640"
uuid = "35d6a980-a343-548e-a6ea-1d62b119f2f4"
version = "3.17.1"

[[ColorTypes]]
deps = ["FixedPointNumbers", "Random"]
git-tree-sha1 = "024fe24d83e4a5bf5fc80501a314ce0d1aa35597"
uuid = "3da002f7-5984-5a60-b8a6-cbb66c0b333f"
version = "0.11.0"

[[Colors]]
deps = ["ColorTypes", "FixedPointNumbers", "Reexport"]
git-tree-sha1 = "417b0ed7b8b838aa6ca0a87aadf1bb9eb111ce40"
uuid = "5ae59095-9a9b-59fe-a467-6f913c188581"
version = "0.12.8"

[[Compat]]
deps = ["Base64", "Dates", "DelimitedFiles", "Distributed", "InteractiveUtils", "LibGit2", "Libdl", "LinearAlgebra", "Markdown", "Mmap", "Pkg", "Printf", "REPL", "Random", "SHA", "Serialization", "SharedArrays", "Sockets", "SparseArrays", "Statistics", "Test", "UUIDs", "Unicode"]
git-tree-sha1 = "96b0bc6c52df76506efc8a441c6cf1adcb1babc4"
uuid = "34da2185-b29b-5c13-b0c7-acf172513d20"
version = "3.42.0"

[[CompilerSupportLibraries_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"

[[Contour]]
deps = ["StaticArrays"]
git-tree-sha1 = "9f02045d934dc030edad45944ea80dbd1f0ebea7"
uuid = "d38c429a-6771-53c6-b99e-75d170b6e991"
version = "0.5.7"

[[DataAPI]]
git-tree-sha1 = "cc70b17275652eb47bc9e5f81635981f13cea5c8"
uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a"
version = "1.9.0"

[[DataStructures]]
deps = ["Compat", "InteractiveUtils", "OrderedCollections"]
git-tree-sha1 = "3daef5523dd2e769dad2365274f760ff5f282c7d"
uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
version = "0.18.11"

[[DataValueInterfaces]]
git-tree-sha1 = "bfc1187b79289637fa0ef6d4436ebdfe6905cbd6"
uuid = "e2d170a0-9d28-54be-80f0-106bbe20a464"
version = "1.0.0"

[[Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[DelimitedFiles]]
deps = ["Mmap"]
uuid = "8bb1440f-4735-579b-a4ab-409b98df4dab"

[[DensityInterface]]
deps = ["InverseFunctions", "Test"]
git-tree-sha1 = "80c3e8639e3353e5d2912fb3a1916b8455e2494b"
uuid = "b429d917-457f-4dbc-8f4c-0cc954292b1d"
version = "0.4.0"

[[Distributed]]
deps = ["Random", "Serialization", "Sockets"]
uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b"

[[Distributions]]
deps = ["ChainRulesCore", "DensityInterface", "FillArrays", "LinearAlgebra", "PDMats", "Printf", "QuadGK", "Random", "SparseArrays", "SpecialFunctions", "Statistics", "StatsBase", "StatsFuns", "Test"]
git-tree-sha1 = "9d3c0c762d4666db9187f363a76b47f7346e673b"
uuid = "31c24e10-a181-5473-b8eb-7969acd0382f"
version = "0.25.49"

[[DocStringExtensions]]
deps = ["LibGit2"]
git-tree-sha1 = "b19534d1895d702889b219c382a6e18010797f0b"
uuid = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae"
version = "0.8.6"

[[Downloads]]
deps = ["ArgTools", "LibCURL", "NetworkOptions"]
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"

[[DualNumbers]]
deps = ["Calculus", "NaNMath", "SpecialFunctions"]
git-tree-sha1 = "90b158083179a6ccbce2c7eb1446d5bf9d7ae571"
uuid = "fa6b7ba4-c1ee-5f82-b5fc-ecf0adba8f74"
version = "0.6.7"

[[EarCut_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "3f3a2501fa7236e9b911e0f7a588c657e822bb6d"
uuid = "5ae413db-bbd1-5e63-b57d-d24a61df00f5"
version = "2.2.3+0"

[[Expat_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "ae13fcbc7ab8f16b0856729b050ef0c446aa3492"
uuid = "2e619515-83b5-522b-bb60-26c02a35a201"
version = "2.4.4+0"

[[FFMPEG]]
deps = ["FFMPEG_jll"]
git-tree-sha1 = "b57e3acbe22f8484b4b5ff66a7499717fe1a9cc8"
uuid = "c87230d0-a227-11e9-1b43-d7ebe4e7570a"
version = "0.4.1"

[[FFMPEG_jll]]
deps = ["Artifacts", "Bzip2_jll", "FreeType2_jll", "FriBidi_jll", "JLLWrappers", "LAME_jll", "Libdl", "Ogg_jll", "OpenSSL_jll", "Opus_jll", "Pkg", "Zlib_jll", "libass_jll", "libfdk_aac_jll", "libvorbis_jll", "x264_jll", "x265_jll"]
git-tree-sha1 = "d8a578692e3077ac998b50c0217dfd67f21d1e5f"
uuid = "b22a6f82-2f65-5046-a5b2-351ab43fb4e5"
version = "4.4.0+0"

[[FillArrays]]
deps = ["LinearAlgebra", "Random", "SparseArrays", "Statistics"]
git-tree-sha1 = "0dbc5b9683245f905993b51d2814202d75b34f1a"
uuid = "1a297f60-69ca-5386-bcde-b61e274b549b"
version = "0.13.1"

[[FixedPointNumbers]]
deps = ["Statistics"]
git-tree-sha1 = "335bfdceacc84c5cdf16aadc768aa5ddfc5383cc"
uuid = "53c48c17-4a7d-5ca2-90c5-79b7896eea93"
version = "0.8.4"

[[Fontconfig_jll]]
deps = ["Artifacts", "Bzip2_jll", "Expat_jll", "FreeType2_jll", "JLLWrappers", "Libdl", "Libuuid_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "21efd19106a55620a188615da6d3d06cd7f6ee03"
uuid = "a3f928ae-7b40-5064-980b-68af3947d34b"
version = "2.13.93+0"

[[Formatting]]
deps = ["Printf"]
git-tree-sha1 = "8339d61043228fdd3eb658d86c926cb282ae72a8"
uuid = "59287772-0a20-5a39-b81b-1366585eb4c0"
version = "0.4.2"

[[FreeType2_jll]]
deps = ["Artifacts", "Bzip2_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "87eb71354d8ec1a96d4a7636bd57a7347dde3ef9"
uuid = "d7e528f0-a631-5988-bf34-fe36492bcfd7"
version = "2.10.4+0"

[[FriBidi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "aa31987c2ba8704e23c6c8ba8a4f769d5d7e4f91"
uuid = "559328eb-81f9-559d-9380-de523a88c83c"
version = "1.0.10+0"

[[GLFW_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libglvnd_jll", "Pkg", "Xorg_libXcursor_jll", "Xorg_libXi_jll", "Xorg_libXinerama_jll", "Xorg_libXrandr_jll"]
git-tree-sha1 = "51d2dfe8e590fbd74e7a842cf6d13d8a2f45dc01"
uuid = "0656b61e-2033-5cc2-a64a-77c0f6c09b89"
version = "3.3.6+0"

[[GR]]
deps = ["Base64", "DelimitedFiles", "GR_jll", "HTTP", "JSON", "Libdl", "LinearAlgebra", "Pkg", "Printf", "Random", "RelocatableFolders", "Serialization", "Sockets", "Test", "UUIDs"]
git-tree-sha1 = "9f836fb62492f4b0f0d3b06f55983f2704ed0883"
uuid = "28b8d3ca-fb5f-59d9-8090-bfdbd6d07a71"
version = "0.64.0"

[[GR_jll]]
deps = ["Artifacts", "Bzip2_jll", "Cairo_jll", "FFMPEG_jll", "Fontconfig_jll", "GLFW_jll", "JLLWrappers", "JpegTurbo_jll", "Libdl", "Libtiff_jll", "Pixman_jll", "Pkg", "Qt5Base_jll", "Zlib_jll", "libpng_jll"]
git-tree-sha1 = "a6c850d77ad5118ad3be4bd188919ce97fffac47"
uuid = "d2c73de3-f751-5644-a686-071e5b155ba9"
version = "0.64.0+0"

[[GeometryBasics]]
deps = ["EarCut_jll", "IterTools", "LinearAlgebra", "StaticArrays", "StructArrays", "Tables"]
git-tree-sha1 = "83ea630384a13fc4f002b77690bc0afeb4255ac9"
uuid = "5c1252a2-5f33-56bf-86c9-59e7332b4326"
version = "0.4.2"

[[Gettext_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "XML2_jll"]
git-tree-sha1 = "9b02998aba7bf074d14de89f9d37ca24a1a0b046"
uuid = "78b55507-aeef-58d4-861c-77aaff3498b1"
version = "0.21.0+0"

[[Glib_jll]]
deps = ["Artifacts", "Gettext_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Libiconv_jll", "Libmount_jll", "PCRE_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "a32d672ac2c967f3deb8a81d828afc739c838a06"
uuid = "7746bdde-850d-59dc-9ae8-88ece973131d"
version = "2.68.3+2"

[[Graphite2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "344bf40dcab1073aca04aa0df4fb092f920e4011"
uuid = "3b182d85-2403-5c21-9c21-1e1f0cc25472"
version = "1.3.14+0"

[[Grisu]]
git-tree-sha1 = "53bb909d1151e57e2484c3d1b53e19552b887fb2"
uuid = "42e2da0e-8278-4e71-bc24-59509adca0fe"
version = "1.0.2"

[[HTTP]]
deps = ["Base64", "Dates", "IniFile", "Logging", "MbedTLS", "NetworkOptions", "Sockets", "URIs"]
git-tree-sha1 = "0fa77022fe4b511826b39c894c90daf5fce3334a"
uuid = "cd3eb016-35fb-5094-929b-558a96fad6f3"
version = "0.9.17"

[[HarfBuzz_jll]]
deps = ["Artifacts", "Cairo_jll", "Fontconfig_jll", "FreeType2_jll", "Glib_jll", "Graphite2_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Pkg"]
git-tree-sha1 = "129acf094d168394e80ee1dc4bc06ec835e510a3"
uuid = "2e76f6c2-a576-52d4-95c1-20adfe4de566"
version = "2.8.1+1"

[[HypergeometricFunctions]]
deps = ["DualNumbers", "LinearAlgebra", "SpecialFunctions", "Test"]
git-tree-sha1 = "65e4589030ef3c44d3b90bdc5aac462b4bb05567"
uuid = "34004b35-14d8-5ef3-9330-4cdb6864b03a"
version = "0.3.8"

[[IniFile]]
git-tree-sha1 = "f550e6e32074c939295eb5ea6de31849ac2c9625"
uuid = "83e8ac13-25f8-5344-8a64-a9f2b223428f"
version = "0.5.1"

[[InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[InverseFunctions]]
deps = ["Test"]
git-tree-sha1 = "91b5dcf362c5add98049e6c29ee756910b03051d"
uuid = "3587e190-3f89-42d0-90ee-14403ec27112"
version = "0.1.3"

[[IrrationalConstants]]
git-tree-sha1 = "7fd44fd4ff43fc60815f8e764c0f352b83c49151"
uuid = "92d709cd-6900-40b7-9082-c6be49f344b6"
version = "0.1.1"

[[IterTools]]
git-tree-sha1 = "fa6287a4469f5e048d763df38279ee729fbd44e5"
uuid = "c8e1da08-722c-5040-9ed9-7db0dc04731e"
version = "1.4.0"

[[IteratorInterfaceExtensions]]
git-tree-sha1 = "a3f24677c21f5bbe9d2a714f95dcd58337fb2856"
uuid = "82899510-4779-5014-852e-03e436cf321d"
version = "1.0.0"

[[JLLWrappers]]
deps = ["Preferences"]
git-tree-sha1 = "abc9885a7ca2052a736a600f7fa66209f96506e1"
uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210"
version = "1.4.1"

[[JSON]]
deps = ["Dates", "Mmap", "Parsers", "Unicode"]
git-tree-sha1 = "3c837543ddb02250ef42f4738347454f95079d4e"
uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
version = "0.21.3"

[[JpegTurbo_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b53380851c6e6664204efb2e62cd24fa5c47e4ba"
uuid = "aacddb02-875f-59d6-b918-886e6ef4fbf8"
version = "2.1.2+0"

[[LAME_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "f6250b16881adf048549549fba48b1161acdac8c"
uuid = "c1c5ebd0-6772-5130-a774-d5fcae4a789d"
version = "3.100.1+0"

[[LERC_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "bf36f528eec6634efc60d7ec062008f171071434"
uuid = "88015f11-f218-50d7-93a8-a6af411a945d"
version = "3.0.0+1"

[[LZO_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "e5b909bcf985c5e2605737d2ce278ed791b89be6"
uuid = "dd4b983a-f0e5-5f8d-a1b7-129d4a5fb1ac"
version = "2.10.1+0"

[[LaTeXStrings]]
git-tree-sha1 = "f2355693d6778a178ade15952b7ac47a4ff97996"
uuid = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f"
version = "1.3.0"

[[Latexify]]
deps = ["Formatting", "InteractiveUtils", "LaTeXStrings", "MacroTools", "Markdown", "Printf", "Requires"]
git-tree-sha1 = "4f00cc36fede3c04b8acf9b2e2763decfdcecfa6"
uuid = "23fbe1c1-3f47-55db-b15f-69d7ec21a316"
version = "0.15.13"

[[LibCURL]]
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"

[[LibCURL_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"

[[LibGit2]]
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"

[[LibSSH2_jll]]
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"

[[Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[Libffi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "0b4a5d71f3e5200a7dff793393e09dfc2d874290"
uuid = "e9f186c6-92d2-5b65-8a66-fee21dc1b490"
version = "3.2.2+1"

[[Libgcrypt_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgpg_error_jll", "Pkg"]
git-tree-sha1 = "64613c82a59c120435c067c2b809fc61cf5166ae"
uuid = "d4300ac3-e22c-5743-9152-c294e39db1e4"
version = "1.8.7+0"

[[Libglvnd_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll", "Xorg_libXext_jll"]
git-tree-sha1 = "7739f837d6447403596a75d19ed01fd08d6f56bf"
uuid = "7e76a0d4-f3c7-5321-8279-8d96eeed0f29"
version = "1.3.0+3"

[[Libgpg_error_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "c333716e46366857753e273ce6a69ee0945a6db9"
uuid = "7add5ba3-2f88-524e-9cd5-f83b8a55f7b8"
version = "1.42.0+0"

[[Libiconv_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "42b62845d70a619f063a7da093d995ec8e15e778"
uuid = "94ce4f54-9a6c-5748-9c1c-f9c7231a4531"
version = "1.16.1+1"

[[Libmount_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "9c30530bf0effd46e15e0fdcf2b8636e78cbbd73"
uuid = "4b2f31a3-9ecc-558c-b454-b3730dcb73e9"
version = "2.35.0+0"

[[Libtiff_jll]]
deps = ["Artifacts", "JLLWrappers", "JpegTurbo_jll", "LERC_jll", "Libdl", "Pkg", "Zlib_jll", "Zstd_jll"]
git-tree-sha1 = "c9551dd26e31ab17b86cbd00c2ede019c08758eb"
uuid = "89763e89-9b03-5906-acba-b20f662cd828"
version = "4.3.0+1"

[[Libuuid_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "7f3efec06033682db852f8b3bc3c1d2b0a0ab066"
uuid = "38a345b3-de98-5d2b-a5d3-14cd9215e700"
version = "2.36.0+0"

[[LinearAlgebra]]
deps = ["Libdl"]
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[[LogExpFunctions]]
deps = ["ChainRulesCore", "ChangesOfVariables", "DocStringExtensions", "InverseFunctions", "IrrationalConstants", "LinearAlgebra"]
git-tree-sha1 = "56ad13e26b7093472eba53b418eba15ad830d6b5"
uuid = "2ab3a3ac-af41-5b50-aa03-7779005ae688"
version = "0.3.9"

[[Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[MacroTools]]
deps = ["Markdown", "Random"]
git-tree-sha1 = "3d3e902b31198a27340d0bf00d6ac452866021cf"
uuid = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"
version = "0.5.9"

[[Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[MbedTLS]]
deps = ["Dates", "MbedTLS_jll", "Random", "Sockets"]
git-tree-sha1 = "1c38e51c3d08ef2278062ebceade0e46cefc96fe"
uuid = "739be429-bea8-5141-9913-cc70e7f3736d"
version = "1.0.3"

[[MbedTLS_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"

[[Measures]]
git-tree-sha1 = "e498ddeee6f9fdb4551ce855a46f54dbd900245f"
uuid = "442fdcdd-2543-5da2-b0f3-8c86c306513e"
version = "0.3.1"

[[Missings]]
deps = ["DataAPI"]
git-tree-sha1 = "bf210ce90b6c9eed32d25dbcae1ebc565df2687f"
uuid = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28"
version = "1.0.2"

[[Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"

[[MozillaCACerts_jll]]
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"

[[NaNMath]]
git-tree-sha1 = "737a5957f387b17e74d4ad2f440eb330b39a62c5"
uuid = "77ba4419-2d1f-58cd-9bb1-8ffee604a2e3"
version = "1.0.0"

[[NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"

[[Ogg_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "887579a3eb005446d514ab7aeac5d1d027658b8f"
uuid = "e7412a2a-1a6e-54c0-be00-318e2571c051"
version = "1.3.5+1"

[[OpenLibm_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "05823500-19ac-5b8b-9628-191a04bc5112"

[[OpenSSL_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "648107615c15d4e09f7eca16307bc821c1f718d8"
uuid = "458c3c95-2e84-50aa-8efc-19380b2a3a95"
version = "1.1.13+0"

[[OpenSpecFun_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "13652491f6856acfd2db29360e1bbcd4565d04f1"
uuid = "efe28fd5-8261-553b-a9e1-b2916fc3738e"
version = "0.5.5+0"

[[Opus_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "51a08fb14ec28da2ec7a927c4337e4332c2a4720"
uuid = "91d4177d-7536-5919-b921-800302f37372"
version = "1.3.2+0"

[[OrderedCollections]]
git-tree-sha1 = "85f8e6578bf1f9ee0d11e7bb1b1456435479d47c"
uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
version = "1.4.1"

[[PCRE_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b2a7af664e098055a7529ad1a900ded962bca488"
uuid = "2f80f16e-611a-54ab-bc61-aa92de5b98fc"
version = "8.44.0+0"

[[PDMats]]
deps = ["LinearAlgebra", "SparseArrays", "SuiteSparse"]
git-tree-sha1 = "e8185b83b9fc56eb6456200e873ce598ebc7f262"
uuid = "90014a1f-27ba-587c-ab20-58faa44d9150"
version = "0.11.7"

[[Parsers]]
deps = ["Dates"]
git-tree-sha1 = "85b5da0fa43588c75bb1ff986493443f821c70b7"
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
version = "2.2.3"

[[Pixman_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b4f5d02549a10e20780a24fce72bea96b6329e29"
uuid = "30392449-352a-5448-841d-b1acce4e97dc"
version = "0.40.1+0"

[[Pkg]]
deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"

[[PlotThemes]]
deps = ["PlotUtils", "Requires", "Statistics"]
git-tree-sha1 = "a3a964ce9dc7898193536002a6dd892b1b5a6f1d"
uuid = "ccf2f8ad-2431-5c83-bf29-c5338b663b6a"
version = "2.0.1"

[[PlotUtils]]
deps = ["ColorSchemes", "Colors", "Dates", "Printf", "Random", "Reexport", "Statistics"]
git-tree-sha1 = "6f1b25e8ea06279b5689263cc538f51331d7ca17"
uuid = "995b91a9-d308-5afd-9ec6-746e21dbc043"
version = "1.1.3"

[[Plots]]
deps = ["Base64", "Contour", "Dates", "Downloads", "FFMPEG", "FixedPointNumbers", "GR", "GeometryBasics", "JSON", "Latexify", "LinearAlgebra", "Measures", "NaNMath", "Pkg", "PlotThemes", "PlotUtils", "Printf", "REPL", "Random", "RecipesBase", "RecipesPipeline", "Reexport", "Requires", "Scratch", "Showoff", "SparseArrays", "Statistics", "StatsBase", "UUIDs", "UnicodeFun", "Unzip"]
git-tree-sha1 = "9213b4c18b57b7020ee20f33a4ba49eb7bef85e0"
uuid = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
version = "1.27.0"

[[Preferences]]
deps = ["TOML"]
git-tree-sha1 = "d3538e7f8a790dc8903519090857ef8e1283eecd"
uuid = "21216c6a-2e73-6563-6e65-726566657250"
version = "1.2.5"

[[Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[Qt5Base_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "Fontconfig_jll", "Glib_jll", "JLLWrappers", "Libdl", "Libglvnd_jll", "OpenSSL_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libxcb_jll", "Xorg_xcb_util_image_jll", "Xorg_xcb_util_keysyms_jll", "Xorg_xcb_util_renderutil_jll", "Xorg_xcb_util_wm_jll", "Zlib_jll", "xkbcommon_jll"]
git-tree-sha1 = "ad368663a5e20dbb8d6dc2fddeefe4dae0781ae8"
uuid = "ea2cea3b-5b76-57ae-a6ef-0a8af62496e1"
version = "5.15.3+0"

[[QuadGK]]
deps = ["DataStructures", "LinearAlgebra"]
git-tree-sha1 = "78aadffb3efd2155af139781b8a8df1ef279ea39"
uuid = "1fd47b50-473d-5c70-9696-f719f8f3bcdc"
version = "2.4.2"

[[REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[[Random]]
deps = ["Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[RecipesBase]]
git-tree-sha1 = "6bf3f380ff52ce0832ddd3a2a7b9538ed1bcca7d"
uuid = "3cdcf5f2-1ef4-517c-9805-6587b60abb01"
version = "1.2.1"

[[RecipesPipeline]]
deps = ["Dates", "NaNMath", "PlotUtils", "RecipesBase"]
git-tree-sha1 = "995a812c6f7edea7527bb570f0ac39d0fb15663c"
uuid = "01d81517-befc-4cb6-b9ec-a95719d0359c"
version = "0.5.1"

[[Reexport]]
git-tree-sha1 = "45e428421666073eab6f2da5c9d310d99bb12f9b"
uuid = "189a3867-3050-52da-a836-e630ba90ab69"
version = "1.2.2"

[[RelocatableFolders]]
deps = ["SHA", "Scratch"]
git-tree-sha1 = "cdbd3b1338c72ce29d9584fdbe9e9b70eeb5adca"
uuid = "05181044-ff0b-4ac5-8273-598c1e38db00"
version = "0.1.3"

[[Requires]]
deps = ["UUIDs"]
git-tree-sha1 = "838a3a4188e2ded87a4f9f184b4b0d78a1e91cb7"
uuid = "ae029012-a4dd-5104-9daa-d747884805df"
version = "1.3.0"

[[Rmath]]
deps = ["Random", "Rmath_jll"]
git-tree-sha1 = "bf3188feca147ce108c76ad82c2792c57abe7b1f"
uuid = "79098fc4-a85e-5d69-aa6a-4863f24498fa"
version = "0.7.0"

[[Rmath_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "68db32dff12bb6127bac73c209881191bf0efbb7"
uuid = "f50d1b31-88e8-58de-be2c-1cc44531875f"
version = "0.3.0+0"

[[SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"

[[Scratch]]
deps = ["Dates"]
git-tree-sha1 = "0b4b7f1393cff97c33891da2a0bf69c6ed241fda"
uuid = "6c6a2e73-6563-6170-7368-637461726353"
version = "1.1.0"

[[Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[SharedArrays]]
deps = ["Distributed", "Mmap", "Random", "Serialization"]
uuid = "1a1011a3-84de-559e-8e89-a11a2f7dc383"

[[Showoff]]
deps = ["Dates", "Grisu"]
git-tree-sha1 = "91eddf657aca81df9ae6ceb20b959ae5653ad1de"
uuid = "992d4aef-0814-514b-bc4d-f2e9a6c4116f"
version = "1.0.3"

[[Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"

[[SortingAlgorithms]]
deps = ["DataStructures"]
git-tree-sha1 = "b3363d7460f7d098ca0912c69b082f75625d7508"
uuid = "a2af1166-a08f-5f64-846c-94a0d3cef48c"
version = "1.0.1"

[[SparseArrays]]
deps = ["LinearAlgebra", "Random"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"

[[SpecialFunctions]]
deps = ["ChainRulesCore", "IrrationalConstants", "LogExpFunctions", "OpenLibm_jll", "OpenSpecFun_jll"]
git-tree-sha1 = "5ba658aeecaaf96923dce0da9e703bd1fe7666f9"
uuid = "276daf66-3868-5448-9aa4-cd146d93841b"
version = "2.1.4"

[[StaticArrays]]
deps = ["LinearAlgebra", "Random", "Statistics"]
git-tree-sha1 = "74fb527333e72ada2dd9ef77d98e4991fb185f04"
uuid = "90137ffa-7385-5640-81b9-e52037218182"
version = "1.4.1"

[[Statistics]]
deps = ["LinearAlgebra", "SparseArrays"]
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[[StatsAPI]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "c3d8ba7f3fa0625b062b82853a7d5229cb728b6b"
uuid = "82ae8749-77ed-4fe6-ae5f-f523153014b0"
version = "1.2.1"

[[StatsBase]]
deps = ["DataAPI", "DataStructures", "LinearAlgebra", "LogExpFunctions", "Missings", "Printf", "Random", "SortingAlgorithms", "SparseArrays", "Statistics", "StatsAPI"]
git-tree-sha1 = "8977b17906b0a1cc74ab2e3a05faa16cf08a8291"
uuid = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
version = "0.33.16"

[[StatsFuns]]
deps = ["ChainRulesCore", "HypergeometricFunctions", "InverseFunctions", "IrrationalConstants", "LogExpFunctions", "Reexport", "Rmath", "SpecialFunctions"]
git-tree-sha1 = "25405d7016a47cf2bd6cd91e66f4de437fd54a07"
uuid = "4c63d2b9-4356-54db-8cca-17b64c39e42c"
version = "0.9.16"

[[StructArrays]]
deps = ["Adapt", "DataAPI", "StaticArrays", "Tables"]
git-tree-sha1 = "57617b34fa34f91d536eb265df67c2d4519b8b98"
uuid = "09ab397b-f2b6-538f-b94a-2f83cf4a842a"
version = "0.6.5"

[[SuiteSparse]]
deps = ["Libdl", "LinearAlgebra", "Serialization", "SparseArrays"]
uuid = "4607b0f0-06f3-5cda-b6b1-a6196a1729e9"

[[TOML]]
deps = ["Dates"]
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"

[[TableTraits]]
deps = ["IteratorInterfaceExtensions"]
git-tree-sha1 = "c06b2f539df1c6efa794486abfb6ed2022561a39"
uuid = "3783bdb8-4a98-5b6b-af9a-565f29a5fe9c"
version = "1.0.1"

[[Tables]]
deps = ["DataAPI", "DataValueInterfaces", "IteratorInterfaceExtensions", "LinearAlgebra", "OrderedCollections", "TableTraits", "Test"]
git-tree-sha1 = "5ce79ce186cc678bbb5c5681ca3379d1ddae11a1"
uuid = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
version = "1.7.0"

[[Tar]]
deps = ["ArgTools", "SHA"]
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"

[[Test]]
deps = ["InteractiveUtils", "Logging", "Random", "Serialization"]
uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[[URIs]]
git-tree-sha1 = "97bbe755a53fe859669cd907f2d96aee8d2c1355"
uuid = "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4"
version = "1.3.0"

[[UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[[Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"

[[UnicodeFun]]
deps = ["REPL"]
git-tree-sha1 = "53915e50200959667e78a92a418594b428dffddf"
uuid = "1cfade01-22cf-5700-b092-accc4b62d6e1"
version = "0.4.1"

[[Unzip]]
git-tree-sha1 = "34db80951901073501137bdbc3d5a8e7bbd06670"
uuid = "41fe7b60-77ed-43a1-b4f0-825fd5a5650d"
version = "0.1.2"

[[Wayland_jll]]
deps = ["Artifacts", "Expat_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Pkg", "XML2_jll"]
git-tree-sha1 = "3e61f0b86f90dacb0bc0e73a0c5a83f6a8636e23"
uuid = "a2964d1f-97da-50d4-b82a-358c7fce9d89"
version = "1.19.0+0"

[[Wayland_protocols_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4528479aa01ee1b3b4cd0e6faef0e04cf16466da"
uuid = "2381bf8a-dfd0-557d-9999-79630e7b1b91"
version = "1.25.0+0"

[[XML2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "1acf5bdf07aa0907e0a37d3718bb88d4b687b74a"
uuid = "02c8fc9c-b97f-50b9-bbe4-9be30ff0a78a"
version = "2.9.12+0"

[[XSLT_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgcrypt_jll", "Libgpg_error_jll", "Libiconv_jll", "Pkg", "XML2_jll", "Zlib_jll"]
git-tree-sha1 = "91844873c4085240b95e795f692c4cec4d805f8a"
uuid = "aed1982a-8fda-507f-9586-7b0439959a61"
version = "1.1.34+0"

[[Xorg_libX11_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll", "Xorg_xtrans_jll"]
git-tree-sha1 = "5be649d550f3f4b95308bf0183b82e2582876527"
uuid = "4f6342f7-b3d2-589e-9d20-edeb45f2b2bc"
version = "1.6.9+4"

[[Xorg_libXau_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4e490d5c960c314f33885790ed410ff3a94ce67e"
uuid = "0c0b7dd1-d40b-584c-a123-a41640f87eec"
version = "1.0.9+4"

[[Xorg_libXcursor_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXfixes_jll", "Xorg_libXrender_jll"]
git-tree-sha1 = "12e0eb3bc634fa2080c1c37fccf56f7c22989afd"
uuid = "935fb764-8cf2-53bf-bb30-45bb1f8bf724"
version = "1.2.0+4"

[[Xorg_libXdmcp_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4fe47bd2247248125c428978740e18a681372dd4"
uuid = "a3789734-cfe1-5b06-b2d0-1dd0d9d62d05"
version = "1.1.3+4"

[[Xorg_libXext_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "b7c0aa8c376b31e4852b360222848637f481f8c3"
uuid = "1082639a-0dae-5f34-9b06-72781eeb8cb3"
version = "1.3.4+4"

[[Xorg_libXfixes_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "0e0dc7431e7a0587559f9294aeec269471c991a4"
uuid = "d091e8ba-531a-589c-9de9-94069b037ed8"
version = "5.0.3+4"

[[Xorg_libXi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll", "Xorg_libXfixes_jll"]
git-tree-sha1 = "89b52bc2160aadc84d707093930ef0bffa641246"
uuid = "a51aa0fd-4e3c-5386-b890-e753decda492"
version = "1.7.10+4"

[[Xorg_libXinerama_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll"]
git-tree-sha1 = "26be8b1c342929259317d8b9f7b53bf2bb73b123"
uuid = "d1454406-59df-5ea1-beac-c340f2130bc3"
version = "1.1.4+4"

[[Xorg_libXrandr_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll"]
git-tree-sha1 = "34cea83cb726fb58f325887bf0612c6b3fb17631"
uuid = "ec84b674-ba8e-5d96-8ba1-2a689ba10484"
version = "1.5.2+4"

[[Xorg_libXrender_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "19560f30fd49f4d4efbe7002a1037f8c43d43b96"
uuid = "ea2f1a96-1ddc-540d-b46f-429655e07cfa"
version = "0.9.10+4"

[[Xorg_libpthread_stubs_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "6783737e45d3c59a4a4c4091f5f88cdcf0908cbb"
uuid = "14d82f49-176c-5ed1-bb49-ad3f5cbd8c74"
version = "0.1.0+3"

[[Xorg_libxcb_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "XSLT_jll", "Xorg_libXau_jll", "Xorg_libXdmcp_jll", "Xorg_libpthread_stubs_jll"]
git-tree-sha1 = "daf17f441228e7a3833846cd048892861cff16d6"
uuid = "c7cfdc94-dc32-55de-ac96-5a1b8d977c5b"
version = "1.13.0+3"

[[Xorg_libxkbfile_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "926af861744212db0eb001d9e40b5d16292080b2"
uuid = "cc61e674-0454-545c-8b26-ed2c68acab7a"
version = "1.1.0+4"

[[Xorg_xcb_util_image_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "0fab0a40349ba1cba2c1da699243396ff8e94b97"
uuid = "12413925-8142-5f55-bb0e-6d7ca50bb09b"
version = "0.4.0+1"

[[Xorg_xcb_util_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll"]
git-tree-sha1 = "e7fd7b2881fa2eaa72717420894d3938177862d1"
uuid = "2def613f-5ad1-5310-b15b-b15d46f528f5"
version = "0.4.0+1"

[[Xorg_xcb_util_keysyms_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "d1151e2c45a544f32441a567d1690e701ec89b00"
uuid = "975044d2-76e6-5fbe-bf08-97ce7c6574c7"
version = "0.4.0+1"

[[Xorg_xcb_util_renderutil_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "dfd7a8f38d4613b6a575253b3174dd991ca6183e"
uuid = "0d47668e-0667-5a69-a72c-f761630bfb7e"
version = "0.3.9+1"

[[Xorg_xcb_util_wm_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "e78d10aab01a4a154142c5006ed44fd9e8e31b67"
uuid = "c22f9ab0-d5fe-5066-847c-f4bb1cd4e361"
version = "0.4.1+1"

[[Xorg_xkbcomp_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxkbfile_jll"]
git-tree-sha1 = "4bcbf660f6c2e714f87e960a171b119d06ee163b"
uuid = "35661453-b289-5fab-8a00-3d9160c6a3a4"
version = "1.4.2+4"

[[Xorg_xkeyboard_config_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xkbcomp_jll"]
git-tree-sha1 = "5c8424f8a67c3f2209646d4425f3d415fee5931d"
uuid = "33bec58e-1273-512f-9401-5d533626f822"
version = "2.27.0+4"

[[Xorg_xtrans_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "79c31e7844f6ecf779705fbc12146eb190b7d845"
uuid = "c5fb5394-a638-5e4d-96e5-b29de1b5cf10"
version = "1.4.0+3"

[[Zlib_jll]]
deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"

[[Zstd_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "e45044cd873ded54b6a5bac0eb5c971392cf1927"
uuid = "3161d3a3-bdf6-5164-811a-617609db77b4"
version = "1.5.2+0"

[[libass_jll]]
deps = ["Artifacts", "Bzip2_jll", "FreeType2_jll", "FriBidi_jll", "HarfBuzz_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "5982a94fcba20f02f42ace44b9894ee2b140fe47"
uuid = "0ac62f75-1d6f-5e53-bd7c-93b484bb37c0"
version = "0.15.1+0"

[[libfdk_aac_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "daacc84a041563f965be61859a36e17c4e4fcd55"
uuid = "f638f0a6-7fb0-5443-88ba-1cc74229b280"
version = "2.0.2+0"

[[libpng_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "94d180a6d2b5e55e447e2d27a29ed04fe79eb30c"
uuid = "b53b4c65-9356-5827-b1ea-8c7a1a84506f"
version = "1.6.38+0"

[[libvorbis_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Ogg_jll", "Pkg"]
git-tree-sha1 = "b910cb81ef3fe6e78bf6acee440bda86fd6ae00c"
uuid = "f27f6e37-5d2b-51aa-960f-b287f2bc3b7a"
version = "1.3.7+1"

[[nghttp2_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"

[[p7zip_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"

[[x264_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4fea590b89e6ec504593146bf8b988b2c00922b2"
uuid = "1270edf5-f2f9-52d2-97e9-ab00b5d0237a"
version = "2021.5.5+0"

[[x265_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "ee567a171cce03570d77ad3a43e90218e38937a9"
uuid = "dfaa095f-4041-5dcd-9319-2fabd8486b76"
version = "3.5.0+0"

[[xkbcommon_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Wayland_jll", "Wayland_protocols_jll", "Xorg_libxcb_jll", "Xorg_xkeyboard_config_jll"]
git-tree-sha1 = "ece2350174195bb31de1a63bea3a41ae1aa593b6"
uuid = "d8fb68d0-12a3-5cfd-a85a-d49703b185fd"
version = "0.9.1+5"
"""

# ╔═╡ Cell order:
# ╠═57c67334-a30d-11ec-3019-012043761a13
# ╟─287d1794-fe2c-49ba-ad5a-71a130ee14be
# ╟─cb547e46-51fb-47ae-97ed-482d89d9c5e5
# ╠═b93fcb5e-64b3-4bda-abd4-ade9cedf56ca
# ╟─61d7c512-c0a0-4341-b97c-ffaf26944b6a
# ╠═6aa0d42e-e778-48eb-a2a7-d923fa80daea
# ╟─ab0da14c-2b52-44d6-a91d-795c669687fb
# ╟─68c4f411-d5a0-48f0-bbba-8e6901b0408f
# ╟─00000000-0000-0000-0000-000000000001
# ╟─00000000-0000-0000-0000-000000000002


"""
        vikor(decisionMat, weights, fns; v = 0.5)

Apply VIKOR (VlseKriterijumska Optimizcija I Kaompromisno Resenje in Serbian) method for a given matrix and weights.

# Arguments:
 - `decisionMat::DataFrame`: n × m matrix of objective values for n candidate (or strategy) and m criteria
 - `weights::Array{Float64, 1}`: m-vector of weights that sum up to 1.0. If the sum of weights is not 1.0, it is automatically normalized.
 - `fns::Array{Function, 1}`: m-vector of function that are either maximum or minimum.
 - `v::Float64`: Optional algorithm parameter. Default is 0.5.

# Description
vikor() applies the VIKOR method to rank n strategies subject to m criteria which are supposed to be either maximized or minimized.

# Output
- `::VikorResult`: VikorResult object that holds multiple outputs including scores and best index.

# Examples
```julia-repl
julia> Amat = [
             100 92 10 2 80 70 95 80 ;
             80  70 8  4 100 80 80 90 ;
             90 85 5 0 75 95 70 70 ;
             70 88 20 18 60 90 95 85
           ];

julia> dmat = makeDecisionMatrix(Amat)
4×8 DataFrame
 Row │ Crt1     Crt2     Crt3     Crt4     Crt5     Crt6     Crt7     Crt8
     │ Float64  Float64  Float64  Float64  Float64  Float64  Float64  Float64
─────┼────────────────────────────────────────────────────────────────────────
   1 │   100.0     92.0     10.0      2.0     80.0     70.0     95.0     80.0
   2 │    80.0     70.0      8.0      4.0    100.0     80.0     80.0     90.0
   3 │    90.0     85.0      5.0      0.0     75.0     95.0     70.0     70.0
   4 │    70.0     88.0     20.0     18.0     60.0     90.0     95.0     85.0

julia> fns = makeminmax([maximum, maximum, maximum, maximum, maximum, maximum, maximum, maximum]);

julia> result = vikor(dmat, w, fns);

julia> result.scores
4-element Array{Float64,1}:
  0.1975012087551764
  0.2194064473270817
  0.3507643203516215
 -0.16727341435277993

julia> result.bestIndex
4

```

# References
Celikbilek Yakup, Cok Kriterli Karar Verme Yontemleri, Aciklamali ve Karsilastirmali
Saglik Bilimleri Uygulamalari ile. Editor: Muhlis Ozdemir, Nobel Kitabevi, Ankara, 2018
"""
function vikor(decisionMat::DataFrame, weights::Array{Float64,1}, fns::Array{Function,1}; v::Float64=0.5)::VikorResult
    w = unitize(weights)

    nalternatives, ncriteria = size(decisionMat)

    # col_max = colmaxs(decisionMat)
    # col_min = colmins(decisionMat)
    col_max = apply_columns(fns, decisionMat)
    col_min = apply_columns(reverseminmax(fns), decisionMat)

    A = similar(decisionMat)

    for i in 1:nalternatives
        for j in 1:ncriteria
            if fns[j] == maximum
                @inbounds A[i, j] = abs((col_max[j] - decisionMat[i, j]) / (col_max[j] - col_min[j]))
            elseif fns[j] == minimum
                @inbounds A[i, j] = abs((decisionMat[i, j] - col_min[j]) / (col_max[j] - col_min[j]))
            else
                @warn fns[j]
                error("Function must be either maximum or minimum.")
            end
        end
    end

    weightedA = w * A

    s = Array{Float64,1}(undef, nalternatives)
    r = similar(s)
    q = similar(s)

    for i in 1:nalternatives
        s[i] = sum(weightedA[i,:])
        r[i] = maximum(weightedA[i,:])
    end

    smin = minimum(s)
    smax = maximum(s)
    rmin = minimum(r)
    rmax = maximum(r)
    q = v .* ((s .- smin ./ (smax .- smin))) + (1 - v) .* ((r .- rmin ./ (rmax .- rmin)))

    scores = q

    # select the one with minimum score
    best_index = sortperm(q) |> first

    result = VikorResult(
        decisionMat,
        w,
        weightedA,
        best_index,
        scores
    )
    return result
end


import Base: full, ==, copy, size, convert
import SparseArrays: sparse

"""
# SkylineMatrix type for symmetric matrices

Type to hold skyline matrices with element types Tv

- sv:         Skyline vector
- kdiag:      Diagonal elements in sv.
- sm:         SkylineMatrix.

### Constructor
```julia
sm = SkylineMatrix{Tv}(kdiag::Vector{Int}, sv::Vector{Tv})

```
### Arguments
```julia
* `kdiag`           : Vector specifying location of diagonal element in sv.
* `sv`              : Non-zero values in upper triangular matrix.
```
### Additional methods
```julia
* `size(sm)`        : Return size of full matrix => (n, n).
* `==(sm1, sm2)`    : Compare 2 SkylineMatrices for equality.
* `copy(sm)`        : Create a copy.
* `sparse(sm)`      : Convert to a SparseMatrixCSC.
* `full(sm)`        : Convert to a full matrix.

* `convert(::Type{NumericalMethodsforEngineers.SkylineMatrix}, kdiag::Vector{Int}, sv::Vector{Tv})`
                    : Default constructor

* `fromskyline(kdiag::Vector{Int}, sv::Vector{Tv})`
                    : Convert kdiag,sv to full matrix.
```
"""
mutable struct SkylineMatrix{Tv}
    kdiag::Vector{Int}        # Diagonal elements
    sv::Vector{Tv}         # Nonzero values
end

size(sm::SkylineMatrix) = (length(sm.kdiag), length(sm.kdiag))

==(sm1::SkylineMatrix, sm2::SkylineMatrix) = (sm1.kdiag==sm2.kdiag && sm1.sv==sm2.sv)

function copy(sm::SkylineMatrix)
  Tv = eltype(sm.sv)
  SkylineMatrix{Tv}(copy(sm.kdiag), copy(sm.sv))
end

function convert(::Type{NumericalMethodsforEngineers.SkylineMatrix}, kdiag::Vector{Int}, sv::Vector)
  Tv = eltype(sv)
  SkylineMatrix{Tv}(kdiag, sv)
end

function full(sm::SkylineMatrix)
  fromskyline(sm.kdiag, sm.sv)
end

function sparse(sm::SkylineMatrix)
	sparse(fromskyline(sm.kdiag, sm.sv))
end

"""
# Convert the pair kdiag, sv to a full matrix

### Function
```julia
m = fromskyline{Tv}(kdiag::Vector{Int}, sv::Vector{Tv})
```
### Arguments
```julia
* `kdiag`           : Vector specifying location of diagonal element in sv.
* `sv`              : Non-zero values in upper triangular matrix.
```
"""
function fromskyline(kdiag::Vector{Int64}, sv::Vector)
	neq = size(kdiag, 1)
	km = zeros(eltype(sv), neq, neq)
	km[1, 1] = sv[kdiag[1]]
	for i in 2:neq
		km[i, i] = sv[kdiag[i]]
		for j in (kdiag[i-1] + 1):kdiag[i]
			km[i, i - (kdiag[i]-j)] = sv[j]
			km[i - (kdiag[i]-j), i] = sv[j]
    end
  end
	km
end


function F_(u::T, v::T, w::T) where T<:Real
	R1 = sqrt(v^2 + w^2)
	R2 = sqrt(u^2 + w^2)
	R3 = sqrt(u^2 + v^2)
	R  = sqrt(u^2 + v^2 + w^2)
	return sign(u*v) *
		(
			abs(u) * log( (abs(v) + R3) * R2 / (abs(u) * (abs(v) + R))) +
			abs(v) * log( (abs(u) + R3) * R1 / (abs(v) * (abs(u) + R))) +
			w * atan(abs(u * v) / w / R)
 		)
end

"""
   gbox(x, y, z, xextent, yextent, zextent)

Compute the vertical component of the gravitational attraction
rectangular box bounded by the axis-parallel planes defined
by `xextent`, `yextent`, and `zextent`.
Observation point is given by the cartesian coordinates `x, y, z`.

Output is given in units of m/s^2.
"""
function gbox(x::T, y::T, z::T, x1::Array{T}, y1::Array{T}, z1::Array{T}) where T<:Real
	Vz = zero(eltype(T))
	for i in 1:2
        for j in 1:2
            for k in 1:2
                f = (-1.0)^(i+j+k)
                Vz += f * F_(x - x1[i], y - y1[j], z - z1[k])
            end
        end
    end
	return Vz * 6.6742e-11
end


# # [Ecosystem Models](@id ecosystem_example)
#
#md # [![](https://img.shields.io/badge/show-nbviewer-579ACA.svg)](@__NBVIEWER_ROOT_URL__/examples/Ecosystem.ipynb)

using AlgebraicDynamics
using AlgebraicDynamics.DWDDynam
using AlgebraicDynamics.UWDDynam

using Catlab.CategoricalAlgebra
using Catlab.WiringDiagrams
using Catlab.Graphics
using Catlab.Programs

using LabelledArrays
using OrdinaryDiffEq
using Plots, Plots.PlotMeasures

# There will be 16 parameters in to the total ecosystem. With four distinct types:
# - parameters $\alpha$ represent the rate at which a species population grows
# - parameters $\beta$ represent the rate at which a population of prey declines in a predation interaction
# - parameters $\gamma$ represent the rate at which a population of predators grows in a predation interaction
# - parameters $\delta$ represent the rate at with a species population declines

params = LVector(αr=0.3, βrf=0.015, γrf=0.015, δf=0.7,
                 βrh=0.01, γrh=0.01, δh=0.5,
                 γfishh=0.001, βfishh=0.003,
                 αfish=0.35, βfishF=0.015, γfishF=0.015,
                 δF=0.7, βFs=0.017, γFs=0.017, δs=0.35)

# ## Land Ecosystem

# ### Rabbits and foxes

# A standard Lotka Volterra predator-prey model is the composition of three primitive resource sharers:

# 1. a model of rabbit growth: this resource sharer has dynamics $\dot r(t) = \alpha r(t)$ and one port which exposes the rabbit population.
# 2. a model of rabbit/fox predation: this resource sharer has dynamics $$\dot r(t) = -\beta r(t) f(t), \dot f(t) = \gamma r(t)f(t)$$ and two ports which expose the rabbit and fox populations respectively.
# 3. a model of fox population decline: this resource sharer has dynamics $\dot f(t) = -\delta f(t)$ and one port which exposes the fox population.

# However, there are not two independent rabbit populations -- one that grows and one that gets eaten by foxes. Likewise, there are not two independent fox populations -- one that declines and one that feasts on rabbits. To capture these interactions between the trio of resource sharers, we compose them by identifying the exposed rabbit populations and identifying the exposed fox populations.
# The syntax for this undirected composition is defined by an undirected wiring diagram.

## Define the primitive systems
dotr(u,p,t) = p.αr*u
dotrf(u,p,t) = [-p.βrf*u[1]*u[2], p.γrf*u[1]*u[2]]
dotf(u,p,t) = -p.δf*u

rabbit_growth = ContinuousResourceSharer{Float64}(1, dotr)
rabbitfox_predation = ContinuousResourceSharer{Float64}(2, dotrf)
fox_decline = ContinuousResourceSharer{Float64}(1, dotf)

## Define the composition pattern
rabbitfox_pattern = @relation (rabbits, foxes) begin
    rabbit_growth(rabbits)
    rabbitfox_predation(rabbits,foxes)
    fox_decline(foxes)
end

## Compose
submodels = Dict(
    :rabbit_growth => rabbit_growth,
    :rabbitfox_predation => rabbitfox_predation,
    :fox_decline => fox_decline
)
rabbitfox_system = oapply(rabbitfox_pattern, submodels)

# Previously, when we derived the Lotka-Volterra model via [undirected composition](https://algebraicjulia.github.io/AlgebraicDynamics.jl/dev/examples/Lotka-Volterra/#Undirected-composition), we by-hand defined the undirected wiring diagram that implements the composition pattern. In contrast, here we implement the same composition pattern as before but this time using the [`@relation` macro](https://algebraicjulia.github.io/Catlab.jl/stable/apis/programs/#Catlab.Programs.RelationalPrograms.@relation-Tuple). This strategy simplifies the definition and explicitly names the boxes and variables. We  visualize the composition pattern below.

to_graphviz(rabbitfox_pattern, box_labels = :name, junction_labels = :variable, edge_attrs=Dict(:len => ".75"))

# We can now construct an `ODEProblem` from the resource sharer `rabbitfox_system` and plot the solution.
u0 = [10.0, 100.0]
tspan = (0.0, 100.0)

prob = ODEProblem(rabbitfox_system, u0, tspan, params)
sol = solve(prob, Tsit5())

plot(sol, rabbitfox_system,
    lw=2,
    title = "Lotka-Volterra Predator-Prey Model",
    xlabel = "time", ylabel = "population size")

# ### Rabbits, foxes, and hawks
# Suppose we now have a three species ecosystem containing rabbits, foxes, and hawks. Foxes and hawks both prey upon rabbits but do not interact with each other. This ecosystem consists of five primitive systems which share variables.
# 1. rabbit growth:  $\dot r(t) = \alpha r(t)$
# 2. rabbit/fox predation:  $\dot r(t) = -\beta r(t) f(t), \dot f(t) = \delta r(t)f(t)$
# 3. fox decline:  $\dot f(t) = -\gamma f(t)$
# 4. rabbit/hawk predation: $\dot r(t) = -\beta' r(t)h(t), \dot h(t) = \delta' r(t)h(t)$
# 5. hawk decline:  $\dot h(t) = -\gamma' h(t)$

# This means the desired composition pattern has five boxes and many ports and wires to keep track of. Instead of implementing this composition pattern by hand, we construct it as a pushout.

## Define the composition pattern for rabbit growth
rabbit_pattern = @relation (rabbits,) -> rabbit_growth(rabbits)

## Define the composition pattern for the rabbit/hawk Lotka Volterra model
rabbithawk_pattern = @relation (rabbits, hawks) begin
    rabbit_growth(rabbits)
    rabbithawk_predation(rabbits,hawks)
    hawk_decline(hawks)
end

## Define transformations between the composition patterns
rabbitfox_transform  = ACSetTransformation((Box=[1], Junction=[1], Port=[1], OuterPort=[1]), rabbit_pattern, rabbitfox_pattern)
rabbithawk_transform = ACSetTransformation((Box=[1], Junction=[1], Port=[1], OuterPort=[1]), rabbit_pattern, rabbithawk_pattern)

## Take the pushout to define the composition pattern for the rabbit, fox, hawk system
rabbitfoxhawk_pattern = ob(pushout(rabbitfox_transform, rabbithawk_transform))

## Visualize the compsition pattern
to_graphviz(rabbitfoxhawk_pattern, box_labels = :name, junction_labels = :variable, edge_attrs=Dict(:len => ".9"))

#-
## Define the additional primitive systems
dotrh(u, p, t) = [-p.βrh*u[1]*u[2], p.γrh*u[1]*u[2]]
doth(u, p, t)  = -p.δh*u

rabbithawk_predation = ContinuousResourceSharer{Float64}(2, dotrh)
hawk_decline         = ContinuousResourceSharer{Float64}(1, doth)

## Compose
merge!(submodels, Dict(
    :rabbithawk_predation => rabbithawk_predation,
    :hawk_decline => hawk_decline
))

land_system = oapply(rabbitfoxhawk_pattern, submodels)

## Solve and plot
u0 = [10.0, 100.0, 50.0]
tspan = (0.0, 100.0)

prob = ODEProblem(land_system, u0, tspan, params)
sol = solve(prob, Tsit5())

plot(sol, land_system,
    lw=2,
    title = "Land Ecosystem",
    xlabel = "time", ylabel = "population size")

# Unfortunately, the hawks are going extinct in this model. We'll have to give hawks something else to eat!
#-
# ## Ocean Ecosystem

# Consider a ocean ecosystem containing three species —- little fish, big fish, and sharks -— with two predation interactions —- sharks eat big fish and big fish eat little fish.

# This ecosystem can be modeled as the composition of 3 machines:
# 1. Evolution of the little fish population:  this machine has one exogenous variable which represents a population of predators $h(t)$ that hunt little fish. This machine has one output which emits the little fish population. The dynamics of this machine is the driven ODE $$\dot f(t) = \alpha f(t) - \beta f(t)h(t)$$
# 2. Evolution of the big fish population:  this machine has two exogenous variables which represent a population of prey $e(t)$ that are eaten by big fish and a population of predators $h(t)$ which hunt big fish. This machine has one output which emits the big fish population. The dynamics of this machine is the drive ODE $$\dot F(t) = \gamma F(t)e(t) - \delta F(t) - \beta'F(t)h(t)$$
# 3. Evolution of the shark population:  this machine has one exogenous variable which represents a population of prey $e(t)$ that are eaten by sharks. This machine has one output which emits the shark population. The dynamics of this machine is the driven ODE $$\dot s(t) = \gamma's(t)e(t) - \delta's(t)$$

## Define the primitive systems
dotfish(f, x, p, t) = [p.αfish*f[1] - p.βfishF*x[1]*f[1]]
dotFISH(F, x, p, t) = [p.γfishF*x[1]*F[1] - p.δF*F[1] - p.βFs*x[2]*F[1]]
dotsharks(s, x, p, t) = [p.γFs*s[1]*x[1]-p.δs*s[1]]

fish   = ContinuousMachine{Float64}(1,1,1, dotfish,   (f,p,t)->f)
FISH   = ContinuousMachine{Float64}(2,1,1, dotFISH,   (F,p,t)->F)
sharks = ContinuousMachine{Float64}(1,1,1, dotsharks, (s,p,t)->s)

# We compose these machines by (1) sending the output of the big fish machine as the input to both the little fish and shark machines and (2) sending the output of the little fish and shark machines as the inputs to the big fish machine.
# The syntax for this directed composition is given by a directed wiring diagram.

## Define the composition pattern
ocean_pattern = WiringDiagram([], [:fish, :Fish, :shark])
fish_box = add_box!(ocean_pattern, Box(:fish, [:pop], [:pop]))
Fish_box = add_box!(ocean_pattern, Box(:Fish, [:pop, :pop], [:pop]))
shark_box = add_box!(ocean_pattern, Box(:shark, [:pop], [:pop]))

output_box = output_id(ocean_pattern)
add_wires!(ocean_pattern, Pair[
    (fish_box, 1)  => (Fish_box, 1),
    (shark_box, 1) => (Fish_box, 2),
    (Fish_box, 1)  => (fish_box, 1),
    (Fish_box, 1)  => (shark_box, 1),
    (fish_box, 1)  => (output_box, 1),
    (Fish_box, 1)  => (output_box, 2),
    (shark_box, 1) => (output_box, 3)
])

## Visualize the composition pattern
to_graphviz(ocean_pattern, orientation=TopToBottom)

#-

## Compose
ocean_system = oapply(ocean_pattern, [fish, FISH, sharks])

## Solve and plot
u0 = [100.0, 10, 2.0]
tspan = (0.0, 100.0)

prob = ODEProblem(ocean_system, u0, tspan, params)
sol = solve(prob, Tsit5())

plot(sol, ocean_system,
    lw=2, title = "Ocean Ecosystem",
    xlabel = "time", ylabel = "population size"
)

# ## Total ecosystem
# ### Another layer of composition

# We will introduce a final predation interaction -- hawks eat little fish --  which will combine the land and ocean ecosystems.
# The composition will be as resource shareres so the first thing we will do is use the dynamics of the machine `ocean_system` to define the dynamics of a resource sharer. We will also define a resource sharer that models hawk/little fish predation.

## Define the additional primitive systems
ocean_system_rs = ContinuousResourceSharer{Float64}(3, (u,p,t) -> eval_dynamics(ocean_system, u, [], p))

dothf(u,p,t) = [p.γfishh*u[1]*u[2], -p.βfishh*u[1]*u[2]]
fishhawk_predation = ContinuousResourceSharer{Float64}(2, dothf)

## Define the composition pattern
eco_pattern = @relation (rabbits, foxes, hawks, littlefish, BigFish, sharks)  begin
    turf(rabbits,foxes,hawks)
    air(hawks, littlefish)
    surf(littlefish, BigFish, sharks)
end

## Visualize the composition pattern
to_graphviz(eco_pattern, box_labels = :name, junction_labels = :variable, edge_attrs=Dict(:len => ".75"))

#-
## Compose

ecosystem = oapply(eco_pattern, Dict(
    :turf => land_system,
    :air => fishhawk_predation,
    :surf => ocean_system_rs)
)

# We can now plot the evolution of the total ecosystem.

## Solve and plot
u0 = [100.0, 50.0, 20.0, 100, 10, 2.0]
tspan = (0.0, 100.0)

prob = ODEProblem(ecosystem, u0, tspan, params)
sol = solve(prob, Tsit5())
plot(sol, ecosystem,
    lw=2, title = "Total Ecosystem",
    xlabel = "time", ylabel = "population size"
)

# Let's zoom in on a narrower time-window.
tspan = (0.0, 30.0)

prob = ODEProblem(ecosystem, u0, tspan, params)
sol = solve(prob, Tsit5())
plot(sol, ecosystem,
    lw=2, title = "Total Ecosystem",
    xlabel = "time", ylabel = "population size"
)
# As a sanity check we can define the rates for the hawk/little fish predation to be 0. This decouples the land and ocean ecosystems. As expected, the plot shows the original evolution of the land ecosystem overlayed with the original evolution of the ocean ecosystem. This shows that they two ecosystems now evolve independently.

tspan = (0.0, 100.0)
params.βfishh = 0; params.γfishh = 0

prob = ODEProblem(ecosystem, u0, tspan, params)
sol = solve(prob, Tsit5())
plot(sol, ecosystem,
    lw=2, title = "Decoupled Ecosystem",
    xlabel = "time", ylabel = "population size"
)


## Julia program for Gamma Metropolis-Hastings
## author: weiya <szcfweiya@gmail.com>
## date: 2018-08-21

## import function gamma_int
include("../../GenRV/gamma.jl")

function mh_gamma(T = 100, alpha = 1.5)
    a = Int(floor(alpha))
    b = a/alpha
    x = ones(T+1) # initial value: 1
    for t = 1:T
        yt = rgamma_int(a, b)
        rt = (yt / x[t] * exp((x[t] - yt) / alpha))^(alpha-a)
        if rt >= 1
            x[t+1] = yt
        else
            u = rand()
            if u < rt
                x[t+1] = yt
            else
                x[t+1] = x[t]
            end
        end
    end
    return(x)
end

# example
mh_gamma()

# comparison with accept-reject
res = mh_gamma(5000, 2.43)[2:end]
est = cumsum(res.^2) ./ collect(1:5000)

res2 = ones(5000)
for i = 1:5000
    res2[i] = rgamma(2.43, 1)
end
est2 = cumsum(res2.^2) ./ collect(1:5000)

using Plots
plot(est, label="Independent MH")
plot!(est2, label="Accept-Reject")
hline!([8.33], label="True value")


# From HyperDualNumbers
const DIFF_RULES = [
    (  :sqrt,    :(1/2/sqrt(x)),                       :(-1/4/(sqrt(x) * x)))
    (  :cbrt,    :(1/3/x^(2/3)),                       :(-2/9/x^(5/3)))
    (  :abs2,    :(2*x),                               :(2))
    (  :abs,     :(signbit(x) ? -one(x) : one(x)),     :(0))
    (  :inv,     :(-1/x^2),                            :(2/x^3))
    (  :log,     :(1/x),                               :(-1/x^2))
    (  :log10,   :(1/(log(10)*x)),                     :(-1/(log(10)*x^2)))
    (  :log2,    :(1/(log(2)*x)),                      :(-1/(log(2)*x^2)))
    (  :log1p,   :(1/(x + 1)),                         :(-1/(x + 1)^2))
    (  :exp,     :(exp(x)),                            :(exp(x)))
    (  :exp2,    :(2^x*log(2)),                        :(2^x*log(2)^2))
    (  :exp10,   :(10^x*log(10)),                      :(10^x*log(10)^2))
    (  :expm1,   :(exp(x)),                            :(exp(x)))
   #( :sin,     :(cos(x)),                             :(-sin(x)))
   #( :cos,     :(-sin(x)),                            :(-cos(x)))
    (  :tan,     :(tan(x)^2 + 1),                      :(2*(tan(x)^2 + 1)*tan(x)))
    (  :sec,     :(sec(x)*tan(x)),                     :(sec(x)*tan(x)^2 + (tan(x)^2 + 1)*sec(x)))
    (  :csc,     :(-cot(x)*csc(x)),                    :(cot(x)^2*csc(x) + (cot(x)^2 + 1)*csc(x)))
    (  :cot,     :(-cot(x)^2 - 1),                     :(2*(cot(x)^2 + 1)*cot(x)))
    (  :sind,    :(π*cos(π*x/180)/180),                :(-π^2*sin(π*x/180)/180^2))
    (  :cosd,    :(-π*sin(π*x/180)/180),               :(-π^2*cos(π*x/180)/180^2))
    (  :tand,    :(π*(tan(π*x/180)^2 + 1)/180),        :(2*π^2*(tan(π*x/180)^2 + 1)*tan(π*x/180)/180^2))
    (  :secd,    :(π*sec(π*x/180)*tan(π*x/180)/180),   :(π^2*sec(π*x/180)*tan(π*x/180)^2/180^2 + π^2*(tan(π*x/180)^2 + 1)*sec(π*x/180)/180^2))
    (  :cscd,    :(-π*cot(π*x/180)*csc(π*x/180)/180),  :(π^2*cot(π*x/180)^2*csc(π*x/180)/180^2 + π^2*(cot(π*x/180)^2 + 1)*csc(π*x/180)/180^2))
    (  :cotd,    :(-π*(cot(π*x/180)^2 + 1)/180),       :(2*π^2*(cot(π*x/180)^2 + 1)*cot(π*x/180)/180^2))
    (  :asin,    :(1/sqrt(-x^2 + 1)),                  :(x/(-x^2 + 1)^(3/2)))
    (  :acos,    :(-1/sqrt(-x^2 + 1)),                 :(-x/(-x^2 + 1)^(3/2)))
    (  :atan,    :(1/(x^2 + 1)),                       :(-2*x/(x^2 + 1)^2))
    (  :asec,    :(1/(sqrt(x^2 - 1)*x)),               :(-1/(x^2 - 1)^(3/2) - 1/(sqrt(x^2 - 1)*x^2)))
    (  :acsc,    :(-1/(sqrt(x^2 - 1)*x)),              :(1/(x^2 - 1)^(3/2) + 1/(sqrt(x^2 - 1)*x^2)))
    (  :acot,    :(-1/(x^2 + 1)),                      :(2*x/(x^2 + 1)^2))
    (  :asind,   :(π/(sqrt(-π^2*x^2/180^2 + 1)*180)),  :(π^3*x/((-π^2*x^2/180^2 + 1)^(3/2)*180^3)))
    (  :acosd,   :(-π/(sqrt(-π^2*x^2/180^2 + 1)*180)), :(-π^3*x/((-π^2*x^2/180^2 + 1)^(3/2)*180^3)))
    (  :atand,   :(π/((π^2*x^2/180^2 + 1)*180)),       :(-2*π^3*x/((π^2*x^2/180^2 + 1)^2*180^3)))
    (  :asecd,   :(1/(sqrt(π^2*x^2/180^2 - 1)*x)),     :(-π^2/((π^2*x^2/180^2 - 1)^(3/2)*180^2) - 1/(sqrt(π^2*x^2/180^2 - 1)*x^2)))
    (  :acscd,   :(-1/(sqrt(π^2*x^2/180^2 - 1)*x)),    :(π^2/((π^2*x^2/180^2 - 1)^(3/2)*180^2) + 1/(sqrt(π^2*x^2/180^2 - 1)*x^2)))
    (  :acotd,   :(-π/((π^2*x^2/180^2 + 1)*180)),      :(2*π^3*x/((π^2*x^2/180^2 + 1)^2*180^3)))
    (  :sinh,    :(cosh(x)),                           :(sinh(x)))
    (  :cosh,    :(sinh(x)),                           :(cosh(x)))
    (  :tanh,    :(-tanh(x)^2 + 1),                    :(2*(tanh(x)^2 - 1)*tanh(x)))
    (  :sech,    :(-sech(x)*tanh(x)),                  :(sech(x)*tanh(x)^2 + (tanh(x)^2 - 1)*sech(x)))
    (  :csch,    :(-coth(x)*csch(x)),                  :(coth(x)^2*csch(x) + csch(x)/sinh(x)^2))
    (  :coth,    :(-1/sinh(x)^2),                      :(2*cosh(x)/sinh(x)^3))
    (  :asinh,   :(1/sqrt(x^2 + 1)),                   :(-x/(x^2 + 1)^(3/2)))
    (  :acosh,   :(1/(sqrt(x + 1)*sqrt(x - 1))),       :(-1/2/(sqrt(x + 1)*(x - 1)^(3/2)) - 1/2/((x + 1)^(3/2)*sqrt(x - 1))))
    (  :atanh,   :(-1/(x^2 - 1)),                      :(2*x/(x^2 - 1)^2))
    (  :asech,   :(-1/(sqrt(-x^2 + 1)*x)),             :(-1/(-x^2 + 1)^(3/2) + 1/(sqrt(-x^2 + 1)*x^2)))
    (  :acsch,   :(-1/(sqrt(x^2 + 1)*x)),              :(1/(x^2 + 1)^(3/2) + 1/(sqrt(x^2 + 1)*x^2)))
    (  :acoth,   :(-1/(x^2 - 1)),                      :(2*x/(x^2 - 1)^2))
    (  :deg2rad, :(π/180),                             :(0))
    (  :rad2deg, :(180/π),                             :(0))
  # ( :erf,     :(2*exp(-x^2)/sqrt(π)),               :(-4*x*exp(-x^2)/sqrt(π)))
  # ( :erfinv,  :(1/2*sqrt(π)*exp(erfinv(x)^2)),      :(1/2*π*erfinv(x)*exp(2*erfinv(x)^2)))
  # ( :erfc,    :(-2*exp(-x^2)/sqrt(π)),              :(4*x*exp(-x^2)/sqrt(π)))
  # ( :erfi,    :(2*exp(x^2)/sqrt(π)),                :(4*x*exp(x^2)/sqrt(π)))
    (  :cospi,   :(-π*sinpi(x)),                       :(-π^2*cospi(x)))
    (  :sinpi,   :(π*cospi(x)),                        :(-π^2*sinpi(x)))
]

changeprecision(x) = x
changeprecision(x::Integer) = :(T($x))
function changeprecision(x::Symbol)
    x == :π  ?  :(T($x)) : x
end
function changeprecision(ex::Expr)
    if Meta.isexpr(ex, :call, 3) && ex.args[1] == :^ && ex.args[3] isa Int
        return Expr(:call, :^, changeprecision(ex.args[2]), ex.args[3])
    else
        return Expr(ex.head, changeprecision.(ex.args)...)
    end
end

for (i, rule) in enumerate(DIFF_RULES)
    prec_f′ = changeprecision(rule[2])
    prec_f′′ = changeprecision(rule[3])
    DIFF_RULES[i] = (rule[1], prec_f′, prec_f′′)
end


using Interpolations
using Plots
plotly()

# Example
A_x = 1.:2.:40.
A = [log(x) for x in A_x]
itp = interpolate(A, BSpline(Cubic(Line(OnGrid()))))

sitp = scale(itp, A_x)

sitp(3.0)
sitp(3.5)

x = convert(Vector{Float64}, range(1.0, 39.0, step=0.25))
ỹ = sitp.(x)
y = log.(x)
plot(x, ỹ)
plot!(x, y)

# parametric-splines
t = 0:0.1:1
x = sin.(2pi*t)
y = cos.(2pi*t)
A = hcat(x,y)

itp = Interpolations.scale(interpolate(A, (BSpline(Cubic(Natural(OnGrid()))), NoInterp())), t, 1:2)
tfine = 0:0.01:1
xs, ys = [itp(t, 1) for t in tfine], [itp(t,2) for t in tfine]

scatter(x, y, label="knots")
plot!(xs, ys, label="spline")

itp1 = Interpolations.scale(interpolate(A, (BSpline(Cubic(Natural(OnCell()))), NoInterp())), t, 1:2)
xs1, ys1 = [itp1(t, 1) for t in tfine], [itp1(t,2) for t in tfine]
plot!(xs1, ys1, label="spline2")

itp2 = Interpolations.scale(interpolate(A, (BSpline(Cubic(Periodic(OnCell()))), NoInterp())), t, 1:2)
xs2, ys2 = [itp2(t, 1) for t in tfine], [itp2(t,2) for t in tfine]
plot!(xs2, ys2, label="spline3")

# itp3 = Interpolations.scale(interpolate(A, (BSpline(Cubic(Free(OnCell()))), NoInterp())), t, 1:2)
# xs3, ys3 = [itp3(t, 1) for t in tfine], [itp3(t,2) for t in tfine]
# plot!(xs3, ys3, label="spline3")
#
# itp4 = Interpolations.scale(interpolate(A, (BSpline(Cubic(Free(OnGrid()))), NoInterp())), t, 1:2)
# xs4, ys4 = [itp4(t, 1) for t in tfine], [itp4(t,2) for t in tfine]
# plot!(xs4, ys4, label="spline4")
#
# itp5 = Interpolations.scale(interpolate(A, (BSpline(Cubic(Reflect(OnGrid()))), NoInterp())), t, 1:2)
# xs5, ys5 = [itp5(t, 1) for t in tfine], [itp5(t,2) for t in tfine]
# plot!(xs5, ys5, label="spline5")

# parametric-splines 2
t = [0.0, 1/6, 1/6, 2/6, 3/6, 3/6, 4/6, 5/6, 5/6, 1.0]
x = [0.0, -2.0, -2.0, -3.0, -4.0, -4.0, 0.25, 4.5, 4.5, 0.0]
y = [0.0, -2.0, -2.0, 0.0, -2.0, -2.0, 0.0, -2.0, -2.0, 0.0]
A = hcat(x,y)

itp = Interpolations.scale(interpolate(A, (BSpline(Cubic(Natural(OnGrid()))), NoInterp())), t, 1:2)
tfine = 0:0.01:1
xs, ys = [itp(t, 1) for t in tfine], [itp(t,2) for t in tfine]

scatter(x, y, label="knots")
plot!(xs, ys, label="spline")

itp1 = Interpolations.scale(interpolate(A, (BSpline(Cubic(Natural(OnCell()))), NoInterp())), t, 1:2)
xs1, ys1 = [itp1(t, 1) for t in tfine], [itp1(t,2) for t in tfine]
plot!(xs1, ys1, label="spline2")

itp2 = Interpolations.scale(interpolate(A, (BSpline(Cubic(Periodic(OnCell()))), NoInterp())), t, 1:2)
xs2, ys2 = [itp2(t, 1) for t in tfine], [itp2(t,2) for t in tfine]
plot!(xs2, ys2, label="spline3")

using BSplines

basis = BSplines.BSplineBasis(4, 0:5)
plot(basis)
spl = BSplines.approximate(sin, basis)
plot(1:5, spl)
plot(spl, 1,3)
plot(spl)

f1 = Function(spl)
f2 = Function(spl, Derivative(1))
f3 = Function(spl, Derivative(2))
plot([f1, f2, f3])

basis2 = BSplineBasis(3, [1, 2, 3, 4, 5, 5, 6, 7, 8])
plot(basis2)
BSplines.knots(basis2)
b = bsplines(basis2, 3.5)

# splines consist of b-spline basis and a coefficient vector of the same length
t = [0, 1, 2, 3, 4, 5]
points = [0 0; 1 0; 1 1; 0 1; 0 2; 1 2];
x = points[:,1]
y = points[:,2]
plot(t,x); scatter!(t,x)
plot(t,y); scatter!(t,y)
plot(x,y); scatter!(x,y)

basis = averagebasis(3, t)
xspline = BSplines.interpolate(basis, t, x)
plot(t, xspline); scatter!(t, x)
yspline = BSplines.interpolate(basis, t, y)
plot(t, yspline); scatter!(t, y)
xapprox = xspline.(range(0, 5.0; length=100))
yapprox = yspline.(range(0.0, 5.0; length=100))
plot(xapprox, yapprox); scatter!(x,y)

# 3-d splines
t = [0, 1, 2, 3, 4, 5]
points = [0 0 0; 1 0 0; 1 1 0; 0 1 0; 0 2 0; 1 2 0];
x = points[:,1]
y = points[:,2]
z = points[:,3]
plot(t,x); scatter!(t,x)
plot(t,y); scatter!(t,y)
plot(t,z); scatter!(t,z)
plot3d(x,y,z); scatter3d!(x,y,z)

basis = averagebasis(3,t)
xspline = BSplines.interpolate(basis, t, x)
plot(xspline); scatter!(t,x)
yspline = BSplines.interpolate(basis, t, y)
plot(yspline); scatter!(t,y)
zspline = BSplines.interpolate(basis, t, z)
plot(zspline); scatter!(t,z)
xapprox = xspline.(range(0, 5.0; length=100))
yapprox = yspline.(range(0.0, 5.0; length=100))
zapprox = zspline.(range(0.0, 5.0; length=100))
plot3d(xapprox, yapprox, zapprox)
scatter3d!(x,y,z)

# get the derivatives
xdis = Function(xspline)
xvel = Function(xspline, Derivative(1))
xacc = Function(xspline, Derivative(2))
plot([xdis, xvel, xacc], labels=["disp","vel", "acc"])

ydis = Function(yspline)
yvel = Function(yspline, Derivative(1))
yacc = Function(yspline, Derivative(2))
# yjerk = Function(yspline, Derivative(3))
plot([ydis, yvel, yacc], labels=["disp","vel", "acc"])

zdis = Function(zspline)
zvel = Function(zspline, Derivative(1))
zacc = Function(zspline, Derivative(2))
plot([zdis, zvel, zacc], labels=["disp","vel", "acc"])

# try a 3-d path
t = [0, 1, 2, 3, 4]
points = [0 0 0; 0 0.5 -2; 1 1 0; 2 1.5 -2; 0 0 0];
x = points[:,1]
y = points[:,2]
z = points[:,3]
plot(t,x); scatter!(t,x)
plot(t,y); scatter!(t,y)
plot(t,z); scatter!(t,z)
plot3d(x,y,z); scatter3d!(x,y,z)

basis = averagebasis(3,t)
xspline = BSplines.interpolate(basis, t, x)
plot(xspline); scatter!(t,x)
yspline = BSplines.interpolate(basis, t, y)
plot(yspline); scatter!(t,y)
zspline = BSplines.interpolate(basis, t, z)
plot(zspline); scatter!(t,z)
xapprox = xspline.(range(0, 5.0; length=100))
yapprox = yspline.(range(0.0, 5.0; length=100))
zapprox = zspline.(range(0.0, 5.0; length=100))
plot3d(xapprox, yapprox, zapprox)
scatter3d!(x,y,z)

# try another basis with stops at -2 points
t = [0, 1, 2, 3, 4]
points = [0 0 0; 0 0.5 -2; 1 1 0; 2 1.5 -2; 0 0 0];
x = points[:,1]
y = points[:,2]
z = points[:,3]
plotly()
p1 = plot(t,x, label="x"); scatter!(t,x, label=:none)
p2 = plot(t,y, label="y", linecolor=:green); scatter!(t,y, label=:none)
p3 = plot(t,z, label="z", linecolor=:purple); scatter!(t,z, label="Via Points")
plt = plot(p1, p2, p3, layout=(3,1), legend=:outertopright)
p4 = plot3d(x,y,z, xlabel="x", ylabel="y", zlabel="z", label=:none); scatter3d!(x,y,z, markersize=2, label=:none)
plt2 = plot(plt, p4, layout=(1,2))

plotly()
basis = averagebasis(3, t)
basis = BSplines.BSplineBasis(4, [0.0, 1.0, 1.0, 2.0, 3.0, 3.0, 4.0]) # BEAUTIFUL!
# basis = averagebasis(4, t)
# basis = BSplines.BSplineBasis(3, [0.0, 0.0, 1.0, 1.0, 1.5, 2.5, 4.0, 4.0])
# basis = BSplines.BSplineBasis(4, [0.0, 0.0, 1.0, 1.0, 3.0, 3.0, 4.0, 4.0]) # BEAUTIFUL!
# basis = BSplines.BSplineBasis(4, [0.0, 0.0, 1.0, 3.0, 4.0, 4.0]) # BEAUTIFUL!
xspline = BSplines.interpolate(basis, t, x)
p1 = plot(xspline, label="x"); scatter!(t,x, label=:none)
yspline = BSplines.interpolate(basis, t, y)
p2 = plot(yspline, label="y", linecolor=:green); scatter!(t,y, label=:none)
zspline = BSplines.interpolate(basis, t, z)
p3 = plot(zspline, label="z", linecolor=:purple); scatter!(t,z, label="Via Points")
plt = plot(p1, p2, p3, layout=(3,1), legend=:outertopright)
xapprox = xspline.(range(0.0, 4.0; length=401))
yapprox = yspline.(range(0.0, 4.0; length=401))
zapprox = zspline.(range(0.0, 4.0; length=401))
p4 = plot3d(xapprox, yapprox, zapprox, xlabel="x", ylabel="y", zlabel="z", label=:none); scatter3d!(x,y,z, markersize=2, label=:none)
plt2 = plot(plt, p4, layout=(1,2))

plot(basis, legend=:none, title="Trajectory Basis Functions")

# get the derivatives
xdis = Function(xspline)
xvel = Function(xspline, Derivative(1))
xacc = Function(xspline, Derivative(2))

ydis = Function(yspline)
yvel = Function(yspline, Derivative(1))
yacc = Function(yspline, Derivative(2))

zdis = Function(zspline)
zvel = Function(zspline, Derivative(1))
zacc = Function(zspline, Derivative(2))


export SinusoidalFit
import LsqFit

"""
    SinusoidalFit(s, fs) <: Decomposition
Decompose a timeseries `s` into a **sum** `x + r` (with `x` the periodic component),
by fitting sinuisoidals with given
frequencies `fs` to the signal `s` using the package `LsqFit`. Specifically, fit
```math
s - \\bar{s} \\approx \\sum_i A_i \\sin(2\\pi f_i t + \\phi_i)
```
with ``\\bar{s}`` the mean. The fit happens on the amplitudes
and phases ``A_i, \\phi_i``. After the `decomposition` you can find these in the struct's
fields `amps, phases`. The fit is done on `s` versus `t`, so be sure that you have
transformed `t` appropriately (e.g. if `t` is "days" but your frequencies are multiples of
years, then you should give `t/365.26`).

    SinusoidalFit(fs, φ0s, A0s [, ub, lb])

The quality of the fit depends dramatically on the initial guesses for the phases
and amplitudes, `φ0s, A0s`. This second constructor
gives full control over initial phases and amplitudes, as well as
upper and lower bounds on the amplitudes `ub, lb` (also vectors of `length(fs)`).
In the first constructor `φ0 = 0, A0 = abs(-(extrema(s)...))/2, ub = Inf, lb = -Inf` for
all frequencies. (the bounds for the phases are always ± π)

**Notice**: `LsqFit` performs poorly for fitting sinusoidals and `Fourier`
should be preferred over this method if the signal given is in
multiples of the expected periods (and of course `t` is equally spaced).
"""
mutable struct SinusoidalFit{T<:AbstractFloat} <: Decomposition
    fs::Vector{T}
    φ0s::Vector{T}
    A0s::Vector{T}
    ub::Vector{T}
    lb::Vector{T}
    amps::Vector{T}
    phases::Vector{T}
end

function SinusoidalFit(s::AbstractVector{T}, fs) where {T}
    φ0s = zero(fs)
    A0s = fill(abs(-(extrema(s)...))/2, length(fs))
    ub = fill(T(Inf), length(fs))
    lb = fill(T(-Inf), length(fs))
    SinusoidalFit{T}(fs, φ0s, A0s, ub, lb, zero(fs), zero(fs))
end

SinusoidalFit(fs, φ0s, A0s, ub=fill(Inf, length(fs)), lb=fill(-Inf, length(fs))) =
SinusoidalFit{eltype(fs)}(fs, φ0s, A0s, ub, lb, zero(fs), zero(fs))

function decompose(t, s, method::SinusoidalFit{T}) where {T}
    fs = method.fs
    L = length(fs)
    function sinuisoidal(t, p)
        r = zeros(T, length(t))
        for i in 1:L
            @. r += p[i]*sin(2π * (fs[i]*t + p[i+L]))
        end
        return r
    end

    p0 = vcat(method.A0s, method.φ0s)
    ub = vcat(method.ub, fill( T(π), L))
    lb = vcat(method.lb, fill(-T(π), L))

    fit = LsqFit.curve_fit(sinuisoidal, t, s .- mean(s), p0; lower=lb, upper=ub)
    method.amps .= LsqFit.coef(fit)[1:L]
    method.phases .= LsqFit.coef(fit)[L+1:2L]
    periodic = sinuisoidal(t, LsqFit.coef(fit)) .+ mean(s)
    return periodic, s .- periodic
end


# Hessian Eigenmaps (HLLE)
# ---------------------------
# Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data,
# D. Donoho and C. Grimes, Proc Natl Acad Sci U S A. 2003 May 13; 100(10): 5591–5596

import Combinatorics: combinations

#### HLLE type
struct HLLE{T <: Real} <: AbstractDimensionalityReduction
    k::Int
    λ::AbstractVector{T}
    proj::Projection{T}

    HLLE{T}(k::Int, λ::AbstractVector{T}, proj::Projection{T}) where T = new(k, λ, proj)
end

## properties
outdim(R::HLLE) = size(R.proj, 1)
eigvals(R::HLLE) = R.λ
neighbors(R::HLLE) = R.k

## show
summary(io::IO, R::HLLE) = print(io, "Hessian Eigenmaps(outdim = $(outdim(R)), neighbors = $(neighbors(R)))")

## interface functions
function fit(::Type{HLLE}, X::AbstractMatrix{T}; maxoutdim::Int=2, k::Int=12) where {T<:Real}
    n = size(X, 2)

    # Identify neighbors
    D, E = find_nn(X, k)

    # Obtain tangent coordinates and develop Hessian estimator
    hs = (maxoutdim*(maxoutdim+1)) >> 1
    W = spzeros(T, hs*n, n)
    for i=1:n
        II = @view E[:,i]
        # re-center points in neighborhood
        μ = mean(X[:, II], dims=2)
        N = X[:, II] .- μ
        # calculate tangent coordinates
        tc = svd(N).V[:,1:maxoutdim]

        # Develop Hessian estimator
        Yi = [ones(T, k) tc zeros(T, k, hs)]
        for ii=1:maxoutdim
            Yi[:,maxoutdim+ii+1] = tc[:,ii].^2
        end
        yi = 2*(1+maxoutdim)
        for (ii,jj) in combinations(1:maxoutdim, 2)
            Yi[:, yi] = tc[:, ii] .* tc[:, jj]
            yi += 1
        end
        F = qr(Yi)
        H = transpose(F.Q[:,(end-(hs-1)):end])
        W[(1:hs).+(i-1)*hs, II] = H
    end

    # decomposition
    λ, V = decompose(transpose(W)*W, maxoutdim)
    return HLLE{T}(k, λ, transpose(V) .* convert(T, sqrt(n)))
end

transform(R::HLLE) = R.proj


"Example of a more complicated periodic contractor"
function sawtooth(X::IntervalBox)

    x, y = X

    x = x ∩ (-1..1)
    y = y ∩ (-2..2)

    y = y ∩ (2x)
    x = x ∩ (y/2)

    return IntervalBox(x, y)
end

function constant_contractor(X, y_val)
    x, y = X
    y = y ∩ Interval(y_val)
    return IntervalBox(x, y)
end

function add_one(X)  # y = x + 1
    x, y = X

    y = y ∩ (x + 1)
    x = x ∩ (y - 1)

    return IntervalBox(x, y)
end

function cube0(X::IntervalBox)  # contractor for y=x^3, x>=0

    x, y = X

    x = x ∩ (0..∞)

    y = y ∩ (x ^ 3)
    x = x ∩ Interval(y.lo ^ (1/3), y.hi^(1/3))   # not rigorous!

    return x × y
end

"""
Make the "odd" version of a box
odd(X::IntervalBox) = ( (x,y) = X; IntervalBox(-x, -y) )
"""

cube_neg = symmetrise(cube0, odd)
cube = cube0 ∪ cube_neg

ff(x) = x^2 - x^3

function ff(X::IntervalBox)
    x, y = X

    a = x^2
    b = x^3

    y = y ∩ (a - b)  # y = a - b

    a = a ∩ (y + b)
    b = b ∩ (a - y)

    x, a = square(IntervalBox(x, a))
    x, b = cube(IntervalBox(x, b))

    return IntervalBox(x, y)

end


n = 256
w = exp(-2im*pi/n)
DFT = zeros(Complex, n, n)

for i = 1:n
    for j = 1:n
        DFT[i,j] = w^((i-1)*(j-1))
    end
end


"""
Find root of h(x) in the interval [a, b] using binary search
"""
function bisection(h, a, b, tol=1e-5, nmax=10)
    #find root in [a, b]
    n=1
    c=0
    while n<nmax
        c = (a+b)/2
        if abs(h(c))<tol
            return c
        elseif (sign(h(c))==sign(h(a)))
            a=c
        else
            b=c
        end
        n=n+1
    end
    return c
end

"""
Computes the symmetric 95% confidence interval about the median of a
probability distribution given by its PDF using bisection
"""
function confidence(pdf, median, alpha=0.95)
    f = h -> int1D(pdf, median-h, median+h, "2") - alpha
    c = bisection(f, 0, median, 1e-3, 10)
    if abs(c)>median*.9 #this should mean that integration scheme failed to detect spike, indicating that interval is very thin to begin with
        c=.1
    end
    return c
end


###########################################
# From last lecture
###########################################

abstract type Step end

struct GD <: Step
    α::Real
end

optim_step(s::GD, f, g, x) = -s.α*g(x)

function optim(f, g, x, s::Step; max_iter=100)
    for i in 1:max_iter
        x += optim_step(s, f, g, x)
    end
    return x
end

###########################################
# Linear regression
###########################################

# EXTRA

σ(z) = 1/(1+exp(-z))
plot(-10:0.01:10, σ)

# Load data

using Plots
using StatsPlots
using RDatasets
using LinearAlgebra
using Query
using Statistics

iris = dataset("datasets", "iris")

iris[1:5,:]

# Exercise

y = iris.PetalWidth
X = hcat(iris.PetalLength, ones(length(y)))

@df iris scatter(
    :PetalLength,
    :PetalWidth;
    label="",
    xlabel = "Petal length",
    ylabel = "Petal width"
)

# Exercise

w = (X'*X)^(-1)*X'*y
w = (X'*X)^(-1)*(X'*y)
w = X'*X \ X'*y

g(w) = X'*(X*w-y)
w2 = optim([], g, zeros(size(X,2)), GD(1e-4); max_iter=10000)

norm(w-w2)

# EXTRA

w = (X'*X)^(-1)*X'*y
w = (X'*X)^(-1)*(X'*y)
w = X'*X \ X'*y

A = randn(1000,1000)
b = randn(1000)

@time (A*A)*b;
@time A*(A*b);

# Exercise

f_pred(x::Real, w) = ([x 1]*w)[1]
f_pred(x::Real, w) = w[1]*x + w[2]*1

x_lims = extrema(iris.PetalLength) .+ [-0.1, 0.1]

@df iris scatter(
    :PetalLength,
    :PetalWidth;
    xlabel = "Petal length",
    ylabel = "Petal width",
    label = "",
    legend = :topleft,
)

plot!(x_lims, x -> f_pred(x,w); label = "Prediction", line = (:black,3))

###########################################
# Logistic regression
###########################################

# Exercise

iris_reduced = @from i in iris begin
    @where i.Species != "setosa"
    @select {
        i.PetalLength,
        i.PetalWidth,
        intercept = 1,
        i.Species,
        label = i.Species == "virginica",
    }
    @collect DataFrame
end

# EXTRA

iris_reduced2 = iris[iris.Species .!= "setosa", :]
iris_reduced2 = iris_reduced2[:,[3;4;5]]

insertcols!(iris_reduced2, 3, :intercept => 1)
insertcols!(iris_reduced2, 5, :label => iris_reduced2.Species .== "virginica")

isequal(iris_reduced, iris_reduced2)

# Extract data

X = Matrix(iris_reduced[:, 1:3])
y = iris_reduced.label

# Exercise

@df iris_reduced scatter(
    :PetalLength,
    :PetalWidth;
    group = :Species,
    xlabel = "Petal length",
    ylabel = "Petal width",
    legend = :topleft,
)

# EXTRA

ii = iris_reduced.Species .== "virginica"

scatter(iris_reduced.PetalLength[ii,:], iris_reduced.PetalWidth[ii,:];
    xlabel = "Petal length",
    ylabel = "Petal width",
    legend = :topleft,
    label = "virginica"
)

scatter!(iris_reduced.PetalLength[.!ii,:], iris_reduced.PetalWidth[.!ii,:];
    label = "versicolor"
)

# Exercise

function log_reg(X, y, w; max_iter=100, tol=1e-6)
    X_mult = [row*row' for row in eachrow(X)]
    for i in 1:max_iter
        y_hat = σ.(X*w)
        grad = X'*(y_hat.-y) / size(X,1)
        hess = y_hat.*(1 .-y_hat).*X_mult |> mean
        w -= hess \ grad
    end
    return w
end

w = log_reg(X, y, zeros(size(X,2)))

# EXTRA

function g(w, X, y)
    n = size(X,1)
    y_hat = σ.(X*w)
    return X'*(y_hat - y) / n
end

optim([], w -> g(w, X, y), [0;0;0], GD(5e-1); max_iter=1000000)

# Plot the separating hyperplane

separ(x, w) = (-w[3]-w[1]*x)/w[2]

xlims = extrema(iris_reduced.PetalLength) .+ [-0.1, 0.1]
ylims = extrema(iris_reduced.PetalWidth) .+ [-0.1, 0.1]

@df iris_reduced scatter(
    :PetalLength,
    :PetalWidth;
    group = :Species,
    xlabel = "Petal length",
    ylabel = "Petal width",
    legend = :topleft,
    xlims,
    ylims,
)

plot!(xlims, x -> separ(x,w); label = "Separation", line = (:black,3))

# Check optimality

y_hat = σ.(X*w)
grad = X'*(y_hat.-y) / size(X,1)
norm(grad)

# Exercise

pred = y_hat .>= 0.5
"Correct number of predictions: " * string(sum(pred .== y))
"Wrong   number of predictions: " * string(sum(pred .!= y))


using Random
using Makie
using BasicBSpline
using ExportNURBS
Random.seed!(2)

## 1-dim B-spline manifold
p = 2 # degree of polynomial
k = Knots(1:12) # knot vector
P = FastBSplineSpace(p, k) # B-spline space
a = [[i-5, 3*sin(i^2)] for i in 1:dim(P)] # control points
M = BSplineCurve([P], a) # Define B-spline manifold
save_png("docs/src/img/1dim.png", M, unitlength = 50)

## 2-dim B-spline manifold
p = 2 # degree of polynomial
k = Knots(1:8) # knot vector
P = BSplineSpace(p, k) # B-spline space
rand_a = [rand(2) for i in 1:dim(P), j in 1:dim(P)]
a = [[2 * i - 6.5, 2 * j - 6.5] for i in 1:dim(P), j in 1:dim(P)] + rand_a # random generated control points
M = BSplineManifold([P, P], a) # Define B-spline manifold
save_png("docs/src/img/2dim.png", M, unitlength = 50)

## h-Refinement
k₊ = [Knots(3.3, 4.2), Knots(3.8, 3.2, 5.3)]
M′ = refinement(M, k₊ = k₊)
save_png("docs/src/img/2dim_h-refinement.png", M′, unitlength = 50)

## p-Refinement
p₊ = [1,2]
M′ = refinement(M, p₊ = p₊)
save_png("docs/src/img/2dim_p-refinement.png", M′, unitlength = 50)

## Makie
points = [M([u,v]) for u in range(3.0,6.0,length=50), v in range(3.0,6.0,length=50)]
X = [point[1] for point in points]
Y = [point[2] for point in points]
scene = Scene(resolution=(1000,1000))
Makie.surface!(X,Y)
Makie.xlims!(-5,5)
Makie.ylims!(-5,5)
save("docs/src/img/2dim_makie.png", scene)

## Fitting
p1 = 2
p2 = 2
k1 = Knots(-10:10) + p1 * Knots(-10, 10)
k2 = Knots(-10:10) + p2 * Knots(-10, 10)
P1 = FastBSplineSpace(p1, k1)
P2 = FastBSplineSpace(p2, k2)

f(u1,u2) = [2u1 + sin(u1) + cos(u2) + u2 / 2, 3u2 + sin(u2) + sin(u1) / 2 + u1^2 / 6] / 5

a = fittingcontrolpoints(f, P1, P2)
M = BSplineManifold([P1, P2], a)
save_png("docs/src/img/fitting.png", M, unitlength = 50, up = 10, down = -10, left = -10, right = 10)

## Coarse fitting
p1 = 2
p2 = 2
k1 = Knots(-10:5:10) + p1 * Knots(-10, 10)
k2 = Knots(-10:5:10) + p2 * Knots(-10, 10)
P1 = FastBSplineSpace(p1, k1)
P2 = FastBSplineSpace(p2, k2)

f(u) = [2u[1] + sin(u[1]) + cos(u[2]) + u[2] / 2, 3u[2] + sin(u[2]) + sin(u[1]) / 2 + u[1]^2 / 6] / 5

a = fittingcontrolpoints(f, [P1, P2])
M = BSplineManifold([P1, P2], a)
save_png("docs/src/img/fitting_coarse.png", M, unitlength = 50, up = 10, down = -10, left = -10, right = 10)

## Sine curve
p = 3
k = Knots(range(-2π, 2π, length = 8)) + p * Knots(-2π, 2π)
P = FastBSplineSpace(p, k)

f(u) = [u, sin(u)]
a = fittingcontrolpoints(f, P)
M = BSplineManifold([P], a)
save_svg("docs/src/img/sine_curve.svg", M, unitlength = 50, up = 2, down = -2, left = -8, right = 8)


#saha function
function fHI(T,ne)
     χ   = 13.60
    temp = 15.6826 - log10(ne) + log10(2) + 1.5 * log10(T) - χ * (5040 / T)

    return temp
end

function fHeI(T,ne)
     χ   = 24.58
    temp = 15.6826 - log10(ne) + log10(2) + 1.5 * log10(T) - χ * (5040 / T)

    return temp
end

function fHeII(T,ne)
     χ   = 54.41
    temp = 15.6826 - log10(ne) + log10(2) + 1.5 * log10(T) - χ * (5040 / T)

    return temp
end

function saha(T,N;ionization = false)
    Ne  = N * 0.1
    X   = 0.9
    Y   = 0.1
    NH  = X * N
    NHe = Y * N
    δ   = 1e-5

    X1  = fHI(  T, Ne)
    X2  = fHeI( T, Ne)
    X3  = fHeII(T, Ne)

    Y1  = 10^(X1) #H0
    Y2  = 10^(X2) #He0
    Y3  = 10^(X3) #He+

    X4  = X * N  / (1 + Y1)
    X5  = Y * N  / (1 + Y2 * (1 + Y3))
    Ne_last = 0

    AHI    = 0
    AHII   = 0
    AHeI   = 0
    AHeII  = 0
    AHeIII = 0
    ANe    = 0

    while true
        HI    = X4
        HII   = X * N - HI
        HeI   = X5
        HeII  = Y2 * HeI
        HeIII = Y3 * HeII
        Ne    = HII + HeII + 2 * HeIII

        if (1 - Ne / Ne_last) < δ

            AHI    = HI
            AHII   = HII
            AHeI   = HeI
            AHeII  = HeIII
            AHeIII = HeIII
            ANe    = Ne

            break
        else
            Ne_last = (Ne_last + Ne) /2
        end
    end

    if ionization == true
        return [AHI,AHII,AHeI,AHeII,AHeIII,ANe]
    else
        return ANe, AHI, AHII, AHeI, AHeII, AHeIII
    end
end

export fHI, fHeI, fHeII, saha


import SpecialFunctions: erfc, erfcinv

"""
inverse of cdf.
"""
@inline function ppf(x)
    return -  sqrt(2) * erfcinv(2 * x)
end

@inline function pdf(x)
    return (1. / (sqrt(2 * pi) ) *
        exp(-(((x)) ^ 2 / 2.0)))
end

@inline function cdf(x)
    return 0.5 * erfc(-(x) / (sqrt(2)))
end


export
    Plane3,
    get_signed_distance

"""
Plane3

A hyperplane is an affine subspace of dimension n-1 in a space of dimension n.
For example, a hyperplane in a plane is a line; a hyperplane in 3-space is a plane.

This class represents an hyperplane in 3D space. It is the zero set of
n⋅x + d = 0 where n is a unit normal vector of the plane (linear part)
and d is the distance (offset) to the origin.
"""
struct Plane3
    normal::VecE3 # should always be normalized
    offset::Float64

    Plane3() = new(VecE3(1.0,0.0,0.0), 0.0)

    """
    Constructs a plane from its normal n and distance to the origin d
    such that the algebraic equation of the plane is n⋅x + d = 0.
    """
    Plane3(normal::VecE3, offset::Real) = new(normalize(normal), convert(Float64, offset))

    """
    Construct a plane from its normal and a point on the plane.
    """
    function Plane3(normal::VecE3, P::VecE3)
        n = normalize(normal)
        offset = -n⋅P
        return new(n, offset)
    end
end

"""
Constructs a plane passing through the three points.
"""
function Plane3(p0::VecE3, p1::VecE3, p2::VecE3)
    v0 = p2 - p0
    v1 = p1 - p0

    normal = v0×v1
    nnorm = norm(normal)
    if nnorm <= norm(v0)*norm(v1)*eps()
        M = hcat(convert(Vector{Float64}, v0),
                 convert(Vector{Float64}, v1))'
        s = svdfact(M, thin=false)
        normal = convert(VecE3, s[:V][:,2])
    else
        normal = normalize(normal)
    end

    offset = -p0⋅normal

    return Plane3(normal, offset)
end

"""
Returns the signed distance between the plane and a point
"""
get_signed_distance(plane::Plane3, A::VecE3) = plane.normal⋅A + plane.offset

"""
Returns the absolute distance between the plane and a point
"""
get_distance(plane::Plane3, A::VecE3) = abs(get_signed_distance(plane, A))

"""
Returns the projection of a point onto the plane
"""
proj(A::VecE3, plane::Plane3) = A - get_signed_distance(plane, A) * plane.normal

"""
What side of the plane you are on
Is -1 if on the negative side, 1 if on the positive side, and 0 if on the plane
"""
function get_side(plane::Plane3, A::VecE3, ε::Float64=DUMMY_PRECISION)
    signed_dist = get_signed_distance(plane, A)
    if abs(signed_dist) < ε
        return 0
    else
        return sign(signed_dist)
    end
end


# coding: utf-8

# In[1]:

function pascal_triangle(n)

    row=Any[]

    #base case
    if n==1

        return Any[1]

    elseif n==2

        return Any[1,1]

    else

        #calculate the elements in each row
        for i in 2:n-1

            #rolling sum all the values within 2 windows from the previous row
            #but we cannot include two boundary numbers 1 in this row
            push!(row,pascal_triangle(n-1)[i-1]+pascal_triangle(n-1)[i])

        end

        #append 1 for both front and rear of the row
        pushfirst!(row,1)
        push!(row,1)

    end

    return row

end

# In[2]:

function styled_row(nested)

    #the first loop is to concatenate all rows
    for i in 1:length(nested)

        temp=nested[i]

        #this loop is to reshape the row
        #insert '' between each element in the row
        for j in 2:(2*i-1)

            #if index k is an even number,insert ''
            if j%2==0

                insert!(temp,j,"")

            end

        end

        #need to add '' to both sides of rows
        #length(nested)-i=((length(nested)+length(nested)-1)-(i+i-1))/2
        #we set the n th row plus n-1 space as the total elements in a row
        #we minus the reshaped row (i+i-1)
        #we get the space for both sides
        #finally we divide it by 2 and add to both sides of the row
        #we append the styled row into rows
        nested[i]=cat(fill("",length(nested)-i),temp,fill("",length(nested)-i),dims=1)

    end

    return nested

end

# In[3]:

#print out each row
function printit(n)

    nested=Any[]

    for i in 1:n

        push!(nested,pascal_triangle(i))

    end

    for i in styled_row(nested)

        println(i)

    end

end

# In[4]:

#using memoization

# In[5]:

function mmz(n)

    row=Any[]

    if !(n in keys(memoization))

        #rolling sum all the values within 2 windows from the previous row
        #but we cannot include two boundary numbers 1 in this row
        for i in 2:n-1

            push!(row,mmz(n-1)[i-1]+mmz(n-1)[i])

        end

        #append 1 for both front and rear of the row
        pushfirst!(row,1)
        push!(row,1)

        global memoization[n]=row

    end

    return memoization[n]

end

# In[6]:

function printit_mmz(n)

    global memoization=Dict(1=>Any[1],2=>Any[1,1])

    nested=Any[]

    for i in 1:n

        push!(nested,mmz(i))

    end

    for i in styled_row(nested)

        println(i)

    end

end

# In[7]:

#0.021448 seconds (1.95 k allocations: 72.875 KiB)
@time begin

    printit(5)

end

# In[8]:

#0.355459 seconds (344.14 k allocations: 17.351 MiB)
@time begin

    printit_mmz(5)

end

# In[9]:

#3.822380 seconds (36.60 M allocations: 2.678 GiB, 8.57% gc time)
@time begin

    printit(10)

end

# In[10]:

#0.232830 seconds (7.27 k allocations: 254.938 KiB)
@time begin

    printit_mmz(10)

end

#at the first glance,memoization isnt faster
#as the number grows larger,memoization reveals its true nature


#Haario et al. 2001 An  adaptive  Metropolis  algorithm

#proposal function
function mcmc_proposal_func_ad(par,sig)
    return par.+rand(MvNormal(sig))
end

function mcmc_update_Σ_n_μ(x,Σ,μ,ϵ,s_d,T)

    μ_new = T*μ/(T+1)+x/(T+1)
    #Σ_new = Hermitian((T-1)/T*Σ+s_d/T*(T*μ*transpose(μ)-(T+1)*μ_new*transpose(μ_new)+x*transpose(x)+UniformScaling(ϵ)))
    Σ_new = (T-1)/T*Σ+s_d/T*(T*(μ*transpose(μ))-(T+1)*(μ_new*transpose(μ_new))+(x*transpose(x))+UniformScaling(ϵ))

    return Σ_new,μ_new
end

#n runs of the Metropolis algorithm
function mcmc_run_ad!(x_current,P_current,p,sig,n::Integer,n_chains::Integer)
    n_accept = zeros(n_chains)

    Threads.@threads for j = 1:n_chains
    #for j = 1:n_chains
        for i = 1:n
            x_new = mcmc_proposal_func_ad(x_current[j],sig[j]) #ceate new candidate
            p_new = p(x_new) #Probability of candidate
            A = min(1,p_new/P_current[j]) #calcualte acceptance ratio
            u = rand() #Probability of accepting candidate
            if u<=A
                x_current[j],P_current[j]= x_new,p_new
                n_accept[j] += 1
            end
        end
    end
    return n_accept
end

#n runs of the Metropolis algorithm
function mcmc_run_ad!(x_current,P_current,p,Σ,μ,ϵ,s_d,T,n::Integer,n_chains::Integer)
    n_accept = zeros(n_chains)

    Threads.@threads for j = 1:n_chains
    #for j = 1:n_chains
        T_local = T
        for i = 1:n
            x_new = mcmc_proposal_func_ad(x_current[j],Σ[j]) #ceate new candidate
            p_new = p(x_new) #Probability of candidate
            A = min(1,p_new/P_current[j]) #calcualte acceptance ratio
            u = rand() #Probability of accepting candidate
            if u<=A
                x_current[j],P_current[j]= x_new,p_new
                n_accept[j] += 1
            end
            Σ[j],μ[j] = mcmc_update_Σ_n_μ(reshape(x_current[j],(length(x_current[j]),1)),Σ[j],μ[j],ϵ,s_d,T_local)
            T_local += 1
        end
    end
    T += n
    return n_accept
end

function mcmc_create_Σ_n_μ!(x_current,P_current,p,sig,n::Integer,n_chains::Integer)

    μ = Array{Array{Float64,2},1}(undef,n_chains) #Sample mean
    Σ = Array{Array{Float64,2},1}(undef,n_chains) #Sample Covariance matrix

    Threads.@threads for j = 1:n_chains
    #for j = 1:n_chains

        samples = Array{Float64,2}(undef,n,length(x_current[j]))

        for i = 1:n
            x_new = mcmc_proposal_func_ad(x_current[j],sig[j]) #ceate new candidate
            p_new = p(x_new) #Probability of candidate
            A = min(1,p_new/P_current[j]) #calcualte acceptance ratio
            u = rand() #Probability of accepting candidate
            if u<=A
                x_current[j],P_current[j]= x_new,p_new
            end
            samples[i,:] = x_current[j]
        end
        μ[j] = transpose(mean(samples, dims=1))
        Σ[j] = cov(samples)
    end
    return Σ,μ
end

function metropolis_mcmc_ad!(x_current,P_current,sig,p,s_d;
    n_samples::Integer=100,burn_in_init::Integer=100,burn_in::Integer=100,
    sample_freq::Integer=5,init_sample::Integer=300,ϵ::Float64=1.0e-9,every_n_sample_accept_check::Integer=10)

    n_chains = length(x_current)

    #Run initial burn-in
    println("Start: Init Burn-in")
    mcmc_run_ad!(x_current,P_current,p,sig,burn_in,n_chains)
    println("Done: Init Burn-in")

    #Create sample Covariance matrix matrix and mean
    println("Start: Create sample Σ and μ")
    Σ,μ = mcmc_create_Σ_n_μ!(x_current,P_current,p,sig,init_sample,n_chains)
    T = init_sample #Total number of iteration after initial burn-in
    println("Done: Create sample Σ and μ")

    #Run burn-in
    println("Start: Burn-in")
    mcmc_run_ad!(x_current,P_current,p,Σ,μ,ϵ,s_d,T,burn_in,n_chains)
    println("Done: Burn-in")

    #Create container for samples
    samples = Array{typeof(x_current[1]),2}(undef,n_samples,n_chains)
    accept_rate = zeros(n_chains)
    accept_rate_monitor = zeros(n_chains)

    println("Start: Sampling")
    #Run samples
    for i = 1:n_samples
        println("---Sample: ",i,"/",n_samples)
        #Run mcmc
        n_accept = mcmc_run_ad!(x_current,P_current,p,Σ,μ,ϵ,s_d,T,sample_freq,n_chains)
        accept_rate = accept_rate.+n_accept
        accept_rate_monitor = accept_rate_monitor.+n_accept
        #Save sample
        for j = 1:n_chains
            samples[i,j] = x_current[j]
        end

        if mod(i,every_n_sample_accept_check)==0
            println("Acceptance rate (short term): ",accept_rate_monitor/(every_n_sample_accept_check*sample_freq))
            println("Acceptance rate (long term): ",accept_rate/(i*sample_freq))
            accept_rate_monitor = zeros(n_chains)
        end
    end
    println("Done: Sampling")
    accept_rate /= n_samples*sample_freq

    return samples,accept_rate
end


"""
A `Compunding` type gives information on how to map
an interest rate `r` and year fraction `y`
to a discount factor or effective rate factor.
"""
abstract type Compounding end

"""
The Effective Rate Factor under `Continuous` compounding
is defined by:

```math
exp(rt)
```

given an annual rate `r` and year fraction `t`.
"""
struct Continuous <: Compounding end

"""
The Effective Rate Factor under `Simple` compounding
is defined by:

```math
(1 + rt)
```

given an annual rate `r` and year fraction `t`.
"""
struct Simple <: Compounding end

"""
The Effective Rate Factor under `Exponential` compounding
is defined by:

```math
(1 + r)^t
```

given an annual rate `r` and year fraction `t`.
"""
struct Exponential <: Compounding end


# ============================================================================ #
# General
# ============================================================================ #

orthogonal_vector(a::AbstractArray) = [-a[2], a[1]]

function point_projection(point::AbstractPoint, line::Line)
    e12 = normalize!( vector(line) )
    p = line.p1 + Point(e12 .* dot(point - line.p1, e12))
end

function cross2(a::AbstractArray, b::AbstractArray)
    if !(length(a) == length(b) == 2)
        throw(DimensionMismatch("cross2 is only defined for vectors of length 2"))
    end
    unsafe_cross2(a, b)
end

unsafe_cross2(a::AbstractArray, b::AbstractArray) = a[1] * b[2] - a[2] * b[1]

function cross2(a_x::T, a_y::T, b_x::T, b_y::T)::T where {T}
    return a_x * b_y - a_y * b_x
end

function dot2(a_x::T, a_y::T, b_x::T, b_y::T)::T where {T}
    return a_x * b_x + a_y * b_y
end

function is_left(line::Line, point::AbstractPoint)::Bool
    v12 = vector(line)
    v1p = point - line.p1
    c = cross2(v12, v1p)
    return c > 0
end

function is_left_on(line::Line, point::AbstractPoint)::Bool
    v12 = vector(line)
    v1p = point - line.p1
    c = cross2(v12, v1p)
    return c >= 0
end

function is_left_on(line_1::AbstractPoint, line_2::AbstractPoint, point::AbstractPoint)::Bool
    # v12 = vector(line)
    v12 = line_2 - line_1
    v1p = point - line_1
    c = cross2(v12, v1p)
    return c >= 0
end

function is_left(line_1::AbstractPoint, line_2::AbstractPoint, point::AbstractPoint)::Bool
    # v12 = vector(line)
    # v12 = line_2 - line_1
    # v1p = point - line_1
    # c = cross2(v12, v1p)

    v12_x = line_2.x - line_1.x
    v12_y = line_2.y - line_1.y
    v1p_x = point.x - line_1.x
    v1p_y = point.y - line_1.y

    c = cross2(v12_x, v12_y, v1p_x, v1p_y)

    return c > 0
end

function is_left_on_inlined(line::Line, point::AbstractPoint)::Bool
    v12_x = line.p2.x - line.p1.x
    v12_y = line.p2.y - line.p1.y
    v1p_x = point.x - line.p1.x
    v1p_y = point.y - line.p1.y

    c = cross2(v12_x, v12_y, v1p_x, v1p_y)
    return c >= 0
end

function is_left_on_inlined(p1::AbstractPoint, p2::AbstractPoint, point::AbstractPoint)::Bool
    v12_x = p2.x - p1.x
    v12_y = p2.y - p1.y
    v1p_x = point.x - p1.x
    v1p_y = point.y - p1.y

    c = cross2(v12_x, v12_y, v1p_x, v1p_y)
    return c >= 0
end

function is_on_line(line::Line, point::AbstractPoint, tol = 1e-10)::Bool
    d = distance(line, point)
    return d <= tol
end

function is_on_line(line_p1::AbstractPoint, line_p2::AbstractPoint, point::AbstractPoint, tol = 1e-10)::Bool
    d = distance(line_p1, line_p2, point)
    return d <= tol
end


using FractionalCalculus, Plots

tspan=collect(0:0.01:15)

examplederivative = []

for i in range(0.1, 0.9, step=0.1)
    push!(examplederivative, fracdiff(sin, i, tspan, 0.001, CaputoDiethelm()))
end

plot(tspan, examplederivative, title="Different order of derivative", linewidth=3, label=["α=0.1" "α=0.2" "α=0.3" "α=0.4" "α=0.5" "α=0.6" "α=0.7" "α=0.8" "α=0.9" "α=1"], legend=:bottomright)