### A Pluto.jl notebook ###
# v0.17.0

using Markdown
using InteractiveUtils

# ╔═╡ 2d6d0588-f4e4-4cad-8353-9c0ba020fec6
begin
	using FileIO
	using ImageShow
	using ImageIO
	using PlutoUI
	using Cbc
	using DataFrames
	using Distances
	using JuMP
	using LinearAlgebra
end

# ╔═╡ d44f3f70-705b-43a2-85c0-a441e08fbbe6
load("Pydata_Jeddah_22_new-06.png")

# ╔═╡ a12d1135-ce89-414c-866e-14b4415e1990
md"""
# Geographical Clustering of Saudi Cities With Additional Constraint
**Originally Contributed by**: Matthew Helm ([with help from Mathieu Tanneau on Julia Discourse](https://discourse.julialang.org/t/which-jump-jl-solver-for-this-problem/43350/17?u=mthelm85))
**Customized by**: Hassan Alsawadi
"""

# ╔═╡ d35c109a-e5b6-4c01-a9c2-333b883783f2
md"""
#### Intializing packages

_When running this notebook for the first time, this could take up to 5 minutes. Hang in there!_
"""

# ╔═╡ 2877f488-1ab3-46da-8a49-7db65c3d8a45
md"""
## Objective
The goal of this exercise is to cluster 𝑛 cities into 𝑘 groups, minimizing the total pairwise distance between cities and ensuring that the variance in the total populations of each group is relatively small.

For this example, we'll use the 20 most populous cities in Saudi Arabia.
"""

# ╔═╡ cf6d3f77-781d-4cd0-ad45-9c5c58380354
cities = DataFrame(
    city=["Riyadh", "Jeddah", "Mecca", "Medina", "Dammam", "Abha", "Ha'il", "Hofuf", "Al-Mubarraz", "Sakakah", "Taif", "Khamis Mushait", "Jubail", "Buraydah", "Tabuk", "Najran", "Qatif", "Khobar", "Al Bahah", "Hafr Al-Batin"],
    population=[7676654, 4697000, 2042000, 1488782, 1252523, 1093705, 936465, 858395, 837000, 779480, 695000, 667000, 684531, 669000, 667000, 505652, 474573, 455541, 366000, 359000],
    lat=[24.633333, 21.543333, 21.4225, 24.466667, 26.433333, 18.216944, 27.516667, 25.383333, 25.383333, 29.969722, 21.275094, 18.3, 27, 26.333333, 28.397222, 17.491667, 26.556, 26.283333, 20.0125, 28.434167],
    lon=[46.716667, 39.172778, 39.823333, 39.6, 50.1, 42.505278, 41.683333, 49.583333, 49.6, 40.2, 40.406156, 42.733333, 49.666667, 43.966667, 36.578889, 44.132222, 49.996, 50.2, 41.465278, 45.963611])

# ╔═╡ 11c8323a-9cd5-4580-93d9-ebcd085b4551
md"""
## Model Specifics
We will cluster these 20 cities into 3 different groups and we will assume that the ideal or target population  𝑃  for a group is simply the total population of the 20 cities divided by 3:
"""

# ╔═╡ 0a9212cd-4c79-4752-930b-45dce6a1f3e8
begin
	n = size(cities,1)
	k = 3
	P = sum(cities.population) / k
end

# ╔═╡ 7f7439d3-3d3b-4098-8be5-ca9aa59373ce
md"""### Obtaining the distances between each city
Let's leverage the *Distances.jl* package to compute the pairwise Haversine distance between each of the cities in our data
set and store the result in a variable we'll call `dm`:
"""

# ╔═╡ 3e708ffa-05d9-4cf7-ba4c-11ca5a3e43e7
dm = Distances.pairwise(Haversine(6372.8), Matrix(cities[:, [3,4]])', dims=2)

# ╔═╡ 54082ac4-c31d-4d27-9cc3-5248018c6838
md"""Our distance matrix is symmetric so we'll convert it to a `LowerTriangular` matrix so that we can better interpret the
objective value of our model (if we don't do this the total distance will be doubled):"""

# ╔═╡ 6bc42cc7-e3d4-4470-bece-6af950280372
dm2 = LowerTriangular(dm)

# ╔═╡ a5f3abe7-a843-4ea5-8e77-dfd075b243a5
md"""
## Build the model
Now that we have the basics taken  care of, we can set up our model, create decision variables, add constraints, and then
solve.

First, we'll set up a model that leverages the [Cbc](https://github.com/coin-or/Cbc) solver. Next, we'll set up a binary
variable $x_{i,k}$ that takes the value $1$ if city $i$ is in group $k$ and $0$ otherwise. Each city must be in a group, so
we'll add the constraint $\sum_kx_{i,k} = 1$ for every $i$.
"""

# ╔═╡ ee41a6b3-fb13-4269-ab22-ad2a85bf26e5
begin
	model = Model(Cbc.Optimizer)

	@variable(model, x[1:n, 1:k], Bin)

	for i in 1:n
	    @constraint(model, sum(x[i,:]) == 1)
	end
end

# ╔═╡ dd262e8b-4697-4aa7-9bc7-a32ac25c96dc
md"""
The total population of a group $k$ is $Q_k = \sum_ix_{i,k}q_i$ where $q_i$ is simply the $i$th value from the `population`
column in our `cities` DataFrame. Let's add constraints so that $\alpha \leq (Q_k - P) \leq \beta$. We'll set $\alpha$
equal to -2,500,000 and $\beta$ equal to 2,500,000. By adjusting these thresholds you'll find that there is a tradeoff
between having relatively even populations between groups and having geographically close cities within each group. In
other words, the larger the absolute values of $\alpha$ and $\beta$, the closer together the cities in a group will be but
the variance between the group populations will be higher.
"""

# ╔═╡ d5fc7ca7-33cd-4f8a-aecf-8897bcd40e15
begin
	α = -2_500_000
	β = 2_500_000
	for i in 1:k
	    @constraint(model, (x' * cities.population)[i] - P <= β)
	    @constraint(model, (x' * cities.population)[i] - P >= α)
	end
end

# ╔═╡ 4ab84877-f5d8-4c1b-b0d2-346f0048bb20
md"""
Now we need to add one last binary variable $z_{i,j}$ to our model that we'll use to compute the total distance between the
cities in our groups, defined as  $\sum_{i,j}d_{i,j}z_{i,j}$. Variable $z_{i,j}$ will equal $1$ if cities $i$ and $j$ are
in the same group, and $0$ if they are not in the same group.

To ensure that $z_{i,j} = 1$ if and only if cities $i$ and $j$ are in the same group, we add the constraints $z_{i,j} \geq
x_{i,k} + x_{j,k} - 1$ for every pair $i,j$ and every $k$:
"""

# ╔═╡ 1054b6a4-1c28-46b6-88a4-6f7cae84edee
begin
	@variable(model, z[1:n,1:n], Bin)

	for k in 1:k, i in 1:n, j in 1:n
	    @constraint(model, z[i,j] >= x[i,k] + x[j,k] - 1)
	end
end

# ╔═╡ 07f58656-4826-4ce5-8c97-bf5570f0fc60
md"""
We can now add an objective to our model which will simply be to minimize the dot product of $z$ and our distance matrix,
`dm`. We can then call `optimize!` and review the results.
"""

# ╔═╡ 17088a0b-a568-431c-8d01-13f957e2a4c0
begin
	@objective(model, Min, dot(z,dm));

	optimize!(model)
end

# ╔═╡ 4ded521e-c41f-4927-854b-86f8912269db
md"""## Reviewing the Results
Now that we have results, we can add a column to our `cities` DataFrame for the group and then loop through our $x$
variable to assign each city to its group. Once we have that, we can look at the total population for each group and also
plot the cities and their groups to verify visually that they are grouped by geographic proximity.
"""

# ╔═╡ dab4e559-9dc6-4a72-8d38-9a0487b6251a
PlutoUI.with_terminal() do
	cities.group = zeros(n)

	for i in 1:n, j in 1:k
	    if round(value.(x)[i,j]) == 1.0
	        cities.group[i] = j
	    end
	end

	for group in groupby(cities, :group)
	    @show group
	    println("")
	    @show sum(group.population)
	    println("")
	end
end

# ╔═╡ aa340dc7-8833-4e40-b63e-c602096e40e3
md"""
The populations of each group are fairly even and we can see from the plot below that the groupings look good in terms of
geographic proximity:
"""

# ╔═╡ a2fad4bc-66fa-4466-ae29-b495590f035f
load("output.png")

# ╔═╡ 00000000-0000-0000-0000-000000000001
PLUTO_PROJECT_TOML_CONTENTS = """
[deps]
Cbc = "9961bab8-2fa3-5c5a-9d89-47fab24efd76"
DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
Distances = "b4f34e82-e78d-54a5-968a-f98e89d6e8f7"
FileIO = "5789e2e9-d7fb-5bc7-8068-2c6fae9b9549"
ImageIO = "82e4d734-157c-48bb-816b-45c225c6df19"
ImageShow = "4e3cecfd-b093-5904-9786-8bbb286a6a31"
JuMP = "4076af6c-e467-56ae-b986-b466b2749572"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
PlutoUI = "7f904dfe-b85e-4ff6-b463-dae2292396a8"

[compat]
Cbc = "~0.9.1"
DataFrames = "~1.3.2"
Distances = "~0.10.7"
FileIO = "~1.13.0"
ImageIO = "~0.6.1"
ImageShow = "~0.3.3"
JuMP = "~0.22.3"
PlutoUI = "~0.7.34"
"""

# ╔═╡ 00000000-0000-0000-0000-000000000002
PLUTO_MANIFEST_TOML_CONTENTS = """
# This file is machine-generated - editing it directly is not advised

[[ASL_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "6252039f98492252f9e47c312c8ffda0e3b9e78d"
uuid = "ae81ac8f-d209-56e5-92de-9978fef736f9"
version = "0.1.3+0"

[[AbstractFFTs]]
deps = ["ChainRulesCore", "LinearAlgebra"]
git-tree-sha1 = "6f1d9bc1c08f9f4a8fa92e3ea3cb50153a1b40d4"
uuid = "621f4979-c628-5d54-868e-fcf4e3e8185c"
version = "1.1.0"

[[AbstractPlutoDingetjes]]
deps = ["Pkg"]
git-tree-sha1 = "8eaf9f1b4921132a4cff3f36a1d9ba923b14a481"
uuid = "6e696c72-6542-2067-7265-42206c756150"
version = "1.1.4"

[[Adapt]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "af92965fb30777147966f58acb05da51c5616b5f"
uuid = "79e6a3ab-5dfb-504d-930d-738a2a938a0e"
version = "3.3.3"

[[ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"

[[Artifacts]]
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"

[[Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[BenchmarkTools]]
deps = ["JSON", "Logging", "Printf", "Profile", "Statistics", "UUIDs"]
git-tree-sha1 = "4c10eee4af024676200bc7752e536f858c6b8f93"
uuid = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
version = "1.3.1"

[[BinaryProvider]]
deps = ["Libdl", "Logging", "SHA"]
git-tree-sha1 = "ecdec412a9abc8db54c0efc5548c64dfce072058"
uuid = "b99e7846-7c00-51b0-8f62-c81ae34c0232"
version = "0.5.10"

[[Bzip2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "19a35467a82e236ff51bc17a3a44b69ef35185a2"
uuid = "6e34b625-4abd-537c-b88f-471c36dfa7a0"
version = "1.0.8+0"

[[CEnum]]
git-tree-sha1 = "215a9aa4a1f23fbd05b92769fdd62559488d70e9"
uuid = "fa961155-64e5-5f13-b03f-caf6b980ea82"
version = "0.4.1"

[[Calculus]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "f641eb0a4f00c343bbc32346e1217b86f3ce9dad"
uuid = "49dc2e85-a5d0-5ad3-a950-438e2897f1b9"
version = "0.5.1"

[[Cbc]]
deps = ["BinaryProvider", "CEnum", "Cbc_jll", "Libdl", "MathOptInterface", "SparseArrays"]
git-tree-sha1 = "6656166f484075dd146c9f452b1428116eaf76d4"
uuid = "9961bab8-2fa3-5c5a-9d89-47fab24efd76"
version = "0.9.1"

[[Cbc_jll]]
deps = ["ASL_jll", "Artifacts", "Cgl_jll", "Clp_jll", "CoinUtils_jll", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "OpenBLAS32_jll", "Osi_jll", "Pkg"]
git-tree-sha1 = "a3c5986d7713bce4260d9826deead060a17c8e2d"
uuid = "38041ee0-ae04-5750-a4d2-bb4d0d83d27d"
version = "200.1000.501+0"

[[Cgl_jll]]
deps = ["Artifacts", "Clp_jll", "CoinUtils_jll", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Osi_jll", "Pkg"]
git-tree-sha1 = "11eb7b7688925e9751b5d7a187aaa4291eae2664"
uuid = "3830e938-1dd0-5f3e-8b8e-b3ee43226782"
version = "0.6000.300+0"

[[ChainRulesCore]]
deps = ["Compat", "LinearAlgebra", "SparseArrays"]
git-tree-sha1 = "7dd38532a1115a215de51775f9891f0f3e1bac6a"
uuid = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
version = "1.12.1"

[[ChangesOfVariables]]
deps = ["ChainRulesCore", "LinearAlgebra", "Test"]
git-tree-sha1 = "bf98fa45a0a4cee295de98d4c1462be26345b9a1"
uuid = "9e997f8a-9a97-42d5-a9f1-ce6bfc15e2c0"
version = "0.1.2"

[[Clp_jll]]
deps = ["Artifacts", "CoinUtils_jll", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "METIS_jll", "MUMPS_seq_jll", "OpenBLAS32_jll", "Osi_jll", "Pkg"]
git-tree-sha1 = "b1031dcfbb44553194c9e650feb5ab65e372504f"
uuid = "06985876-5285-5a41-9fcb-8948a742cc53"
version = "100.1700.601+0"

[[CodecBzip2]]
deps = ["Bzip2_jll", "Libdl", "TranscodingStreams"]
git-tree-sha1 = "2e62a725210ce3c3c2e1a3080190e7ca491f18d7"
uuid = "523fee87-0ab8-5b00-afb7-3ecf72e48cfd"
version = "0.7.2"

[[CodecZlib]]
deps = ["TranscodingStreams", "Zlib_jll"]
git-tree-sha1 = "ded953804d019afa9a3f98981d99b33e3db7b6da"
uuid = "944b1d66-785c-5afd-91f1-9de20f533193"
version = "0.7.0"

[[CoinUtils_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "OpenBLAS32_jll", "Pkg"]
git-tree-sha1 = "44173e61256f32918c6c132fc41f772bab1fb6d1"
uuid = "be027038-0da8-5614-b30d-e42594cb92df"
version = "200.1100.400+0"

[[ColorTypes]]
deps = ["FixedPointNumbers", "Random"]
git-tree-sha1 = "024fe24d83e4a5bf5fc80501a314ce0d1aa35597"
uuid = "3da002f7-5984-5a60-b8a6-cbb66c0b333f"
version = "0.11.0"

[[ColorVectorSpace]]
deps = ["ColorTypes", "FixedPointNumbers", "LinearAlgebra", "SpecialFunctions", "Statistics", "TensorCore"]
git-tree-sha1 = "3f1f500312161f1ae067abe07d13b40f78f32e07"
uuid = "c3611d14-8923-5661-9e6a-0046d554d3a4"
version = "0.9.8"

[[Colors]]
deps = ["ColorTypes", "FixedPointNumbers", "Reexport"]
git-tree-sha1 = "417b0ed7b8b838aa6ca0a87aadf1bb9eb111ce40"
uuid = "5ae59095-9a9b-59fe-a467-6f913c188581"
version = "0.12.8"

[[CommonSubexpressions]]
deps = ["MacroTools", "Test"]
git-tree-sha1 = "7b8a93dba8af7e3b42fecabf646260105ac373f7"
uuid = "bbf7d656-a473-5ed7-a52c-81e309532950"
version = "0.3.0"

[[Compat]]
deps = ["Base64", "Dates", "DelimitedFiles", "Distributed", "InteractiveUtils", "LibGit2", "Libdl", "LinearAlgebra", "Markdown", "Mmap", "Pkg", "Printf", "REPL", "Random", "SHA", "Serialization", "SharedArrays", "Sockets", "SparseArrays", "Statistics", "Test", "UUIDs", "Unicode"]
git-tree-sha1 = "44c37b4636bc54afac5c574d2d02b625349d6582"
uuid = "34da2185-b29b-5c13-b0c7-acf172513d20"
version = "3.41.0"

[[CompilerSupportLibraries_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"

[[Crayons]]
git-tree-sha1 = "249fe38abf76d48563e2f4556bebd215aa317e15"
uuid = "a8cc5b0e-0ffa-5ad4-8c14-923d3ee1735f"
version = "4.1.1"

[[DataAPI]]
git-tree-sha1 = "cc70b17275652eb47bc9e5f81635981f13cea5c8"
uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a"
version = "1.9.0"

[[DataFrames]]
deps = ["Compat", "DataAPI", "Future", "InvertedIndices", "IteratorInterfaceExtensions", "LinearAlgebra", "Markdown", "Missings", "PooledArrays", "PrettyTables", "Printf", "REPL", "Reexport", "SortingAlgorithms", "Statistics", "TableTraits", "Tables", "Unicode"]
git-tree-sha1 = "ae02104e835f219b8930c7664b8012c93475c340"
uuid = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
version = "1.3.2"

[[DataStructures]]
deps = ["Compat", "InteractiveUtils", "OrderedCollections"]
git-tree-sha1 = "3daef5523dd2e769dad2365274f760ff5f282c7d"
uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
version = "0.18.11"

[[DataValueInterfaces]]
git-tree-sha1 = "bfc1187b79289637fa0ef6d4436ebdfe6905cbd6"
uuid = "e2d170a0-9d28-54be-80f0-106bbe20a464"
version = "1.0.0"

[[Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[DelimitedFiles]]
deps = ["Mmap"]
uuid = "8bb1440f-4735-579b-a4ab-409b98df4dab"

[[DiffResults]]
deps = ["StaticArrays"]
git-tree-sha1 = "c18e98cba888c6c25d1c3b048e4b3380ca956805"
uuid = "163ba53b-c6d8-5494-b064-1a9d43ac40c5"
version = "1.0.3"

[[DiffRules]]
deps = ["IrrationalConstants", "LogExpFunctions", "NaNMath", "Random", "SpecialFunctions"]
git-tree-sha1 = "dd933c4ef7b4c270aacd4eb88fa64c147492acf0"
uuid = "b552c78f-8df3-52c6-915a-8e097449b14b"
version = "1.10.0"

[[Distances]]
deps = ["LinearAlgebra", "SparseArrays", "Statistics", "StatsAPI"]
git-tree-sha1 = "3258d0659f812acde79e8a74b11f17ac06d0ca04"
uuid = "b4f34e82-e78d-54a5-968a-f98e89d6e8f7"
version = "0.10.7"

[[Distributed]]
deps = ["Random", "Serialization", "Sockets"]
uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b"

[[DocStringExtensions]]
deps = ["LibGit2"]
git-tree-sha1 = "b19534d1895d702889b219c382a6e18010797f0b"
uuid = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae"
version = "0.8.6"

[[Downloads]]
deps = ["ArgTools", "LibCURL", "NetworkOptions"]
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"

[[FileIO]]
deps = ["Pkg", "Requires", "UUIDs"]
git-tree-sha1 = "80ced645013a5dbdc52cf70329399c35ce007fae"
uuid = "5789e2e9-d7fb-5bc7-8068-2c6fae9b9549"
version = "1.13.0"

[[FixedPointNumbers]]
deps = ["Statistics"]
git-tree-sha1 = "335bfdceacc84c5cdf16aadc768aa5ddfc5383cc"
uuid = "53c48c17-4a7d-5ca2-90c5-79b7896eea93"
version = "0.8.4"

[[Formatting]]
deps = ["Printf"]
git-tree-sha1 = "8339d61043228fdd3eb658d86c926cb282ae72a8"
uuid = "59287772-0a20-5a39-b81b-1366585eb4c0"
version = "0.4.2"

[[ForwardDiff]]
deps = ["CommonSubexpressions", "DiffResults", "DiffRules", "LinearAlgebra", "LogExpFunctions", "NaNMath", "Preferences", "Printf", "Random", "SpecialFunctions", "StaticArrays"]
git-tree-sha1 = "1bd6fc0c344fc0cbee1f42f8d2e7ec8253dda2d2"
uuid = "f6369f11-7733-5829-9624-2563aa707210"
version = "0.10.25"

[[Future]]
deps = ["Random"]
uuid = "9fa8497b-333b-5362-9e8d-4d0656e87820"

[[Graphics]]
deps = ["Colors", "LinearAlgebra", "NaNMath"]
git-tree-sha1 = "1c5a84319923bea76fa145d49e93aa4394c73fc2"
uuid = "a2bd30eb-e257-5431-a919-1863eab51364"
version = "1.1.1"

[[Hyperscript]]
deps = ["Test"]
git-tree-sha1 = "8d511d5b81240fc8e6802386302675bdf47737b9"
uuid = "47d2ed2b-36de-50cf-bf87-49c2cf4b8b91"
version = "0.0.4"

[[HypertextLiteral]]
git-tree-sha1 = "2b078b5a615c6c0396c77810d92ee8c6f470d238"
uuid = "ac1192a8-f4b3-4bfe-ba22-af5b92cd3ab2"
version = "0.9.3"

[[IOCapture]]
deps = ["Logging", "Random"]
git-tree-sha1 = "f7be53659ab06ddc986428d3a9dcc95f6fa6705a"
uuid = "b5f81e59-6552-4d32-b1f0-c071b021bf89"
version = "0.2.2"

[[ImageBase]]
deps = ["ImageCore", "Reexport"]
git-tree-sha1 = "b51bb8cae22c66d0f6357e3bcb6363145ef20835"
uuid = "c817782e-172a-44cc-b673-b171935fbb9e"
version = "0.1.5"

[[ImageCore]]
deps = ["AbstractFFTs", "ColorVectorSpace", "Colors", "FixedPointNumbers", "Graphics", "MappedArrays", "MosaicViews", "OffsetArrays", "PaddedViews", "Reexport"]
git-tree-sha1 = "9a5c62f231e5bba35695a20988fc7cd6de7eeb5a"
uuid = "a09fc81d-aa75-5fe9-8630-4744c3626534"
version = "0.9.3"

[[ImageIO]]
deps = ["FileIO", "JpegTurbo", "Netpbm", "OpenEXR", "PNGFiles", "QOI", "Sixel", "TiffImages", "UUIDs"]
git-tree-sha1 = "464bdef044df52e6436f8c018bea2d48c40bb27b"
uuid = "82e4d734-157c-48bb-816b-45c225c6df19"
version = "0.6.1"

[[ImageShow]]
deps = ["Base64", "FileIO", "ImageBase", "ImageCore", "OffsetArrays", "StackViews"]
git-tree-sha1 = "d0ac64c9bee0aed6fdbb2bc0e5dfa9a3a78e3acc"
uuid = "4e3cecfd-b093-5904-9786-8bbb286a6a31"
version = "0.3.3"

[[Imath_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "87f7662e03a649cffa2e05bf19c303e168732d3e"
uuid = "905a6f67-0a94-5f89-b386-d35d92009cd1"
version = "3.1.2+0"

[[IndirectArrays]]
git-tree-sha1 = "012e604e1c7458645cb8b436f8fba789a51b257f"
uuid = "9b13fd28-a010-5f03-acff-a1bbcff69959"
version = "1.0.0"

[[Inflate]]
git-tree-sha1 = "f5fc07d4e706b84f72d54eedcc1c13d92fb0871c"
uuid = "d25df0c9-e2be-5dd7-82c8-3ad0b3e990b9"
version = "0.1.2"

[[InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[InverseFunctions]]
deps = ["Test"]
git-tree-sha1 = "a7254c0acd8e62f1ac75ad24d5db43f5f19f3c65"
uuid = "3587e190-3f89-42d0-90ee-14403ec27112"
version = "0.1.2"

[[InvertedIndices]]
git-tree-sha1 = "bee5f1ef5bf65df56bdd2e40447590b272a5471f"
uuid = "41ab1584-1d38-5bbf-9106-f11c6c58b48f"
version = "1.1.0"

[[IrrationalConstants]]
git-tree-sha1 = "7fd44fd4ff43fc60815f8e764c0f352b83c49151"
uuid = "92d709cd-6900-40b7-9082-c6be49f344b6"
version = "0.1.1"

[[IteratorInterfaceExtensions]]
git-tree-sha1 = "a3f24677c21f5bbe9d2a714f95dcd58337fb2856"
uuid = "82899510-4779-5014-852e-03e436cf321d"
version = "1.0.0"

[[JLLWrappers]]
deps = ["Preferences"]
git-tree-sha1 = "abc9885a7ca2052a736a600f7fa66209f96506e1"
uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210"
version = "1.4.1"

[[JSON]]
deps = ["Dates", "Mmap", "Parsers", "Unicode"]
git-tree-sha1 = "3c837543ddb02250ef42f4738347454f95079d4e"
uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
version = "0.21.3"

[[JpegTurbo]]
deps = ["CEnum", "FileIO", "ImageCore", "JpegTurbo_jll", "TOML"]
git-tree-sha1 = "a77b273f1ddec645d1b7c4fd5fb98c8f90ad10a5"
uuid = "b835a17e-a41a-41e7-81f0-2f016b05efe0"
version = "0.1.1"

[[JpegTurbo_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b53380851c6e6664204efb2e62cd24fa5c47e4ba"
uuid = "aacddb02-875f-59d6-b918-886e6ef4fbf8"
version = "2.1.2+0"

[[JuMP]]
deps = ["Calculus", "DataStructures", "ForwardDiff", "JSON", "LinearAlgebra", "MathOptInterface", "MutableArithmetics", "NaNMath", "OrderedCollections", "Printf", "Random", "SparseArrays", "SpecialFunctions", "Statistics"]
git-tree-sha1 = "fe0f87cc077fc6a23c21e469318993caf2947d10"
uuid = "4076af6c-e467-56ae-b986-b466b2749572"
version = "0.22.3"

[[LibCURL]]
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"

[[LibCURL_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"

[[LibGit2]]
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"

[[LibSSH2_jll]]
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"

[[Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[LinearAlgebra]]
deps = ["Libdl"]
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[[LogExpFunctions]]
deps = ["ChainRulesCore", "ChangesOfVariables", "DocStringExtensions", "InverseFunctions", "IrrationalConstants", "LinearAlgebra"]
git-tree-sha1 = "e5718a00af0ab9756305a0392832c8952c7426c1"
uuid = "2ab3a3ac-af41-5b50-aa03-7779005ae688"
version = "0.3.6"

[[Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[METIS_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "1d31872bb9c5e7ec1f618e8c4a56c8b0d9bddc7e"
uuid = "d00139f3-1899-568f-a2f0-47f597d42d70"
version = "5.1.1+0"

[[MUMPS_seq_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "METIS_jll", "OpenBLAS32_jll", "Pkg"]
git-tree-sha1 = "29de2841fa5aefe615dea179fcde48bb87b58f57"
uuid = "d7ed1dd3-d0ae-5e8e-bfb4-87a502085b8d"
version = "5.4.1+0"

[[MacroTools]]
deps = ["Markdown", "Random"]
git-tree-sha1 = "3d3e902b31198a27340d0bf00d6ac452866021cf"
uuid = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"
version = "0.5.9"

[[MappedArrays]]
git-tree-sha1 = "e8b359ef06ec72e8c030463fe02efe5527ee5142"
uuid = "dbb5928d-eab1-5f90-85c2-b9b0edb7c900"
version = "0.4.1"

[[Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[MathOptInterface]]
deps = ["BenchmarkTools", "CodecBzip2", "CodecZlib", "JSON", "LinearAlgebra", "MutableArithmetics", "OrderedCollections", "Printf", "SparseArrays", "Test", "Unicode"]
git-tree-sha1 = "e8c9653877adcf8f3e7382985e535bb37b083598"
uuid = "b8f27783-ece8-5eb3-8dc8-9495eed66fee"
version = "0.10.9"

[[MbedTLS_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"

[[Missings]]
deps = ["DataAPI"]
git-tree-sha1 = "bf210ce90b6c9eed32d25dbcae1ebc565df2687f"
uuid = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28"
version = "1.0.2"

[[Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"

[[MosaicViews]]
deps = ["MappedArrays", "OffsetArrays", "PaddedViews", "StackViews"]
git-tree-sha1 = "b34e3bc3ca7c94914418637cb10cc4d1d80d877d"
uuid = "e94cdb99-869f-56ef-bcf0-1ae2bcbe0389"
version = "0.3.3"

[[MozillaCACerts_jll]]
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"

[[MutableArithmetics]]
deps = ["LinearAlgebra", "SparseArrays", "Test"]
git-tree-sha1 = "842b5ccd156e432f369b204bb704fd4020e383ac"
uuid = "d8a4904e-b15c-11e9-3269-09a3773c0cb0"
version = "0.3.3"

[[NaNMath]]
git-tree-sha1 = "b086b7ea07f8e38cf122f5016af580881ac914fe"
uuid = "77ba4419-2d1f-58cd-9bb1-8ffee604a2e3"
version = "0.3.7"

[[Netpbm]]
deps = ["FileIO", "ImageCore"]
git-tree-sha1 = "18efc06f6ec36a8b801b23f076e3c6ac7c3bf153"
uuid = "f09324ee-3d7c-5217-9330-fc30815ba969"
version = "1.0.2"

[[NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"

[[OffsetArrays]]
deps = ["Adapt"]
git-tree-sha1 = "043017e0bdeff61cfbb7afeb558ab29536bbb5ed"
uuid = "6fe1bfb0-de20-5000-8ca7-80f57d26f881"
version = "1.10.8"

[[OpenBLAS32_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "ba4a8f683303c9082e84afba96f25af3c7fb2436"
uuid = "656ef2d0-ae68-5445-9ca0-591084a874a2"
version = "0.3.12+1"

[[OpenEXR]]
deps = ["Colors", "FileIO", "OpenEXR_jll"]
git-tree-sha1 = "327f53360fdb54df7ecd01e96ef1983536d1e633"
uuid = "52e1d378-f018-4a11-a4be-720524705ac7"
version = "0.3.2"

[[OpenEXR_jll]]
deps = ["Artifacts", "Imath_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "923319661e9a22712f24596ce81c54fc0366f304"
uuid = "18a262bb-aa17-5467-a713-aee519bc75cb"
version = "3.1.1+0"

[[OpenLibm_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "05823500-19ac-5b8b-9628-191a04bc5112"

[[OpenSpecFun_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "13652491f6856acfd2db29360e1bbcd4565d04f1"
uuid = "efe28fd5-8261-553b-a9e1-b2916fc3738e"
version = "0.5.5+0"

[[OrderedCollections]]
git-tree-sha1 = "85f8e6578bf1f9ee0d11e7bb1b1456435479d47c"
uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
version = "1.4.1"

[[Osi_jll]]
deps = ["Artifacts", "CoinUtils_jll", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "OpenBLAS32_jll", "Pkg"]
git-tree-sha1 = "28e0ddebd069f605ab1988ab396f239a3ac9b561"
uuid = "7da25872-d9ce-5375-a4d3-7a845f58efdd"
version = "0.10800.600+0"

[[PNGFiles]]
deps = ["Base64", "CEnum", "ImageCore", "IndirectArrays", "OffsetArrays", "libpng_jll"]
git-tree-sha1 = "eb4dbb8139f6125471aa3da98fb70f02dc58e49c"
uuid = "f57f5aa1-a3ce-4bc8-8ab9-96f992907883"
version = "0.3.14"

[[PaddedViews]]
deps = ["OffsetArrays"]
git-tree-sha1 = "03a7a85b76381a3d04c7a1656039197e70eda03d"
uuid = "5432bcbf-9aad-5242-b902-cca2824c8663"
version = "0.5.11"

[[Parsers]]
deps = ["Dates"]
git-tree-sha1 = "13468f237353112a01b2d6b32f3d0f80219944aa"
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
version = "2.2.2"

[[Pkg]]
deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"

[[PkgVersion]]
deps = ["Pkg"]
git-tree-sha1 = "a7a7e1a88853564e551e4eba8650f8c38df79b37"
uuid = "eebad327-c553-4316-9ea0-9fa01ccd7688"
version = "0.1.1"

[[PlutoUI]]
deps = ["AbstractPlutoDingetjes", "Base64", "ColorTypes", "Dates", "Hyperscript", "HypertextLiteral", "IOCapture", "InteractiveUtils", "JSON", "Logging", "Markdown", "Random", "Reexport", "UUIDs"]
git-tree-sha1 = "8979e9802b4ac3d58c503a20f2824ad67f9074dd"
uuid = "7f904dfe-b85e-4ff6-b463-dae2292396a8"
version = "0.7.34"

[[PooledArrays]]
deps = ["DataAPI", "Future"]
git-tree-sha1 = "db3a23166af8aebf4db5ef87ac5b00d36eb771e2"
uuid = "2dfb63ee-cc39-5dd5-95bd-886bf059d720"
version = "1.4.0"

[[Preferences]]
deps = ["TOML"]
git-tree-sha1 = "2cf929d64681236a2e074ffafb8d568733d2e6af"
uuid = "21216c6a-2e73-6563-6e65-726566657250"
version = "1.2.3"

[[PrettyTables]]
deps = ["Crayons", "Formatting", "Markdown", "Reexport", "Tables"]
git-tree-sha1 = "dfb54c4e414caa595a1f2ed759b160f5a3ddcba5"
uuid = "08abe8d2-0d0c-5749-adfa-8a2ac140af0d"
version = "1.3.1"

[[Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[Profile]]
deps = ["Printf"]
uuid = "9abbd945-dff8-562f-b5e8-e1ebf5ef1b79"

[[ProgressMeter]]
deps = ["Distributed", "Printf"]
git-tree-sha1 = "afadeba63d90ff223a6a48d2009434ecee2ec9e8"
uuid = "92933f4c-e287-5a05-a399-4b506db050ca"
version = "1.7.1"

[[QOI]]
deps = ["ColorTypes", "FileIO", "FixedPointNumbers"]
git-tree-sha1 = "18e8f4d1426e965c7b532ddd260599e1510d26ce"
uuid = "4b34888f-f399-49d4-9bb3-47ed5cae4e65"
version = "1.0.0"

[[REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[[Random]]
deps = ["Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[Reexport]]
git-tree-sha1 = "45e428421666073eab6f2da5c9d310d99bb12f9b"
uuid = "189a3867-3050-52da-a836-e630ba90ab69"
version = "1.2.2"

[[Requires]]
deps = ["UUIDs"]
git-tree-sha1 = "838a3a4188e2ded87a4f9f184b4b0d78a1e91cb7"
uuid = "ae029012-a4dd-5104-9daa-d747884805df"
version = "1.3.0"

[[SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"

[[Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[SharedArrays]]
deps = ["Distributed", "Mmap", "Random", "Serialization"]
uuid = "1a1011a3-84de-559e-8e89-a11a2f7dc383"

[[Sixel]]
deps = ["Dates", "FileIO", "ImageCore", "IndirectArrays", "OffsetArrays", "REPL", "libsixel_jll"]
git-tree-sha1 = "8fb59825be681d451c246a795117f317ecbcaa28"
uuid = "45858cf5-a6b0-47a3-bbea-62219f50df47"
version = "0.1.2"

[[Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"

[[SortingAlgorithms]]
deps = ["DataStructures"]
git-tree-sha1 = "b3363d7460f7d098ca0912c69b082f75625d7508"
uuid = "a2af1166-a08f-5f64-846c-94a0d3cef48c"
version = "1.0.1"

[[SparseArrays]]
deps = ["LinearAlgebra", "Random"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"

[[SpecialFunctions]]
deps = ["ChainRulesCore", "IrrationalConstants", "LogExpFunctions", "OpenLibm_jll", "OpenSpecFun_jll"]
git-tree-sha1 = "8d0c8e3d0ff211d9ff4a0c2307d876c99d10bdf1"
uuid = "276daf66-3868-5448-9aa4-cd146d93841b"
version = "2.1.2"

[[StackViews]]
deps = ["OffsetArrays"]
git-tree-sha1 = "46e589465204cd0c08b4bd97385e4fa79a0c770c"
uuid = "cae243ae-269e-4f55-b966-ac2d0dc13c15"
version = "0.1.1"

[[StaticArrays]]
deps = ["LinearAlgebra", "Random", "Statistics"]
git-tree-sha1 = "95c6a5d0e8c69555842fc4a927fc485040ccc31c"
uuid = "90137ffa-7385-5640-81b9-e52037218182"
version = "1.3.5"

[[Statistics]]
deps = ["LinearAlgebra", "SparseArrays"]
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[[StatsAPI]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "c3d8ba7f3fa0625b062b82853a7d5229cb728b6b"
uuid = "82ae8749-77ed-4fe6-ae5f-f523153014b0"
version = "1.2.1"

[[TOML]]
deps = ["Dates"]
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"

[[TableTraits]]
deps = ["IteratorInterfaceExtensions"]
git-tree-sha1 = "c06b2f539df1c6efa794486abfb6ed2022561a39"
uuid = "3783bdb8-4a98-5b6b-af9a-565f29a5fe9c"
version = "1.0.1"

[[Tables]]
deps = ["DataAPI", "DataValueInterfaces", "IteratorInterfaceExtensions", "LinearAlgebra", "TableTraits", "Test"]
git-tree-sha1 = "bb1064c9a84c52e277f1096cf41434b675cd368b"
uuid = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
version = "1.6.1"

[[Tar]]
deps = ["ArgTools", "SHA"]
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"

[[TensorCore]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "1feb45f88d133a655e001435632f019a9a1bcdb6"
uuid = "62fd8b95-f654-4bbd-a8a5-9c27f68ccd50"
version = "0.1.1"

[[Test]]
deps = ["InteractiveUtils", "Logging", "Random", "Serialization"]
uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[[TiffImages]]
deps = ["ColorTypes", "DataStructures", "DocStringExtensions", "FileIO", "FixedPointNumbers", "IndirectArrays", "Inflate", "OffsetArrays", "PkgVersion", "ProgressMeter", "UUIDs"]
git-tree-sha1 = "991d34bbff0d9125d93ba15887d6594e8e84b305"
uuid = "731e570b-9d59-4bfa-96dc-6df516fadf69"
version = "0.5.3"

[[TranscodingStreams]]
deps = ["Random", "Test"]
git-tree-sha1 = "216b95ea110b5972db65aa90f88d8d89dcb8851c"
uuid = "3bb67fe8-82b1-5028-8e26-92a6c54297fa"
version = "0.9.6"

[[UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[[Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"

[[Zlib_jll]]
deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"

[[libpng_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "94d180a6d2b5e55e447e2d27a29ed04fe79eb30c"
uuid = "b53b4c65-9356-5827-b1ea-8c7a1a84506f"
version = "1.6.38+0"

[[libsixel_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "78736dab31ae7a53540a6b752efc61f77b304c5b"
uuid = "075b6546-f08a-558a-be8f-8157d0f608a5"
version = "1.8.6+1"

[[nghttp2_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"

[[p7zip_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"
"""

# ╔═╡ Cell order:
# ╟─d44f3f70-705b-43a2-85c0-a441e08fbbe6
# ╟─a12d1135-ce89-414c-866e-14b4415e1990
# ╟─d35c109a-e5b6-4c01-a9c2-333b883783f2
# ╠═2d6d0588-f4e4-4cad-8353-9c0ba020fec6
# ╟─2877f488-1ab3-46da-8a49-7db65c3d8a45
# ╠═cf6d3f77-781d-4cd0-ad45-9c5c58380354
# ╟─11c8323a-9cd5-4580-93d9-ebcd085b4551
# ╠═0a9212cd-4c79-4752-930b-45dce6a1f3e8
# ╟─7f7439d3-3d3b-4098-8be5-ca9aa59373ce
# ╠═3e708ffa-05d9-4cf7-ba4c-11ca5a3e43e7
# ╟─54082ac4-c31d-4d27-9cc3-5248018c6838
# ╠═6bc42cc7-e3d4-4470-bece-6af950280372
# ╟─a5f3abe7-a843-4ea5-8e77-dfd075b243a5
# ╠═ee41a6b3-fb13-4269-ab22-ad2a85bf26e5
# ╟─dd262e8b-4697-4aa7-9bc7-a32ac25c96dc
# ╠═d5fc7ca7-33cd-4f8a-aecf-8897bcd40e15
# ╟─4ab84877-f5d8-4c1b-b0d2-346f0048bb20
# ╠═1054b6a4-1c28-46b6-88a4-6f7cae84edee
# ╟─07f58656-4826-4ce5-8c97-bf5570f0fc60
# ╠═17088a0b-a568-431c-8d01-13f957e2a4c0
# ╟─4ded521e-c41f-4927-854b-86f8912269db
# ╠═dab4e559-9dc6-4a72-8d38-9a0487b6251a
# ╟─aa340dc7-8833-4e40-b63e-c602096e40e3
# ╟─a2fad4bc-66fa-4466-ae29-b495590f035f
# ╟─00000000-0000-0000-0000-000000000001
# ╟─00000000-0000-0000-0000-000000000002


@doc raw"""
    galerkin_projection(
        f, B::AbstractBSplineBasis,
        [deriv = Derivative(0)], [VectorType = Vector{Float64}],
    )

Perform Galerkin projection of a function `f` onto the given basis.

By default, returns a vector with values

```math
φ_i = ⟨ b_i, f ⟩
= ∫_a^b b_i(x) \, f(x) \, \mathrm{d}x,
```

where ``a`` and ``b`` are the boundaries of the B-spline basis
``\{ b_i \}_{i = 1}^N``.

The integrations are performed using Gauss--Legendre quadrature.
The number of quadrature nodes is chosen so that the result is exact when ``f``
is a polynomial of degree ``k - 1`` (or, more generally, a spline belonging to
the space spanned by the basis `B`).
Here ``k`` is the order of the B-spline basis.
In the more general case, this function returns a quadrature approximation of
the projection.

See also [`galerkin_projection!`](@ref) for the in-place operation, and
[`galerkin_matrix`](@ref) for more details.
"""
function galerkin_projection(
        f, B::AbstractBSplineBasis,
        deriv = Derivative(0),
        ::Type{V} = Vector{Float64},
    ) where {V <: AbstractVector}
    N = length(B)
    φ = V(undef, N)
    galerkin_projection!(f, φ, B, deriv)
end

"""
    galerkin_projection!(
        f, φ::AbstractVector, B::AbstractBSplineBasis, [deriv = Derivative(0)],
    )

Compute Galerkin projection ``φ_i = ⟨ b_i, f ⟩``.

See [`galerkin_projection`](@ref) for details.
"""
function galerkin_projection!(
        f, φ::AbstractVector, B::AbstractBSplineBasis, deriv = Derivative(0),
    )
    N = length(B)
    if length(φ) != N
        throw(DimensionMismatch("incorrect length of output vector φ"))
    end
    Base.require_one_based_indexing(φ)

    k = order(B)
    ts = knots(B)

    # Quadrature information (nodes, weights).
    quadx, quadw = _quadrature_prod(Val(2k - 2))
    @assert length(quadx) == k  # we need k quadrature points per knot segment

    fill!(φ, 0)

    nlast = last(eachindex(ts))

    # We loop over all knot segments Ω[n] = (ts[n], ts[n + 1]).
    # For all B-splines with support in this segment, we integrate the product
    # B[i] * f over this segment, adding the result to φ[i].
    @inbounds for n in eachindex(ts)
        n == nlast && break
        tn, tn1 = ts[n], ts[n + 1]
        tn1 == tn && continue  # interval of length = 0

        metric = QuadratureMetric(tn, tn1)

        # Unnormalise quadrature nodes, such that xs ∈ [tn, tn1]
        xs = metric .* quadx
        # @assert all(x -> tn ≤ x ≤ tn1, xs)

        is = nonzero_in_segment(B, n)

        # This is a property of B-spline bases, which should be preserved by
        # derived (recombined) bases.
        @assert 0 < length(is) ≤ k

        # Evaluate all required basis functions on quadrature nodes.
        bis = eval_basis_functions(B, is, xs, deriv)

        fs = ntuple(i -> f(xs[i]), Val(length(xs)))  # this works fine!

        # For some reason, the alternatives below allocate and give bad
        # performance... (on Julia 1.7-beta2)
        # fs = map(f, xs)
        # fs = f.(xs)

        for (ni, i) in enumerate(is)
            bs = bis[ni]
            φ[i] += metric.α * ((bs .* fs) ⋅ quadw)
        end
    end

    φ
end


#=
  From https://probmods.org/chapters/conditioning.html
  (WebPPL):
  """
  This classic Bayesian inference task is a special case of conditioning. Kahneman and Tversky,
  and Gigerenzer and colleagues, have studied how people make simple judgments like the following:

  The probability of breast cancer is 1% for a woman at 40 who participates in a routine screening.
  If a woman has breast cancer, the probability is 80% that she will have a positive mammography.
  If a woman does not have breast cancer, the probability is 9.6% that she will also have a
  positive mammography. A woman in this age group had a positive mammography in a routine
  screening. What is the probability that she actually has breast cancer?

  What is your intuition? Many people without training in statistical inference judge the
  probability to be rather high, typically between 0.7 and 0.9. The correct answer is much lower,
  less than 0.1, as we can see by running this WebPPL inference:
  """

  p(breastCancer): 0.0737

  According to the WebPPL model:
    breastCancer:
    false : 0.922360248447205
    true : 0.07763975155279507

=#

using Turing
include("jl_utils.jl")

@model function medical_diagnosis2()
    breastCancer ~ flip(0.01)
    positiveMammogram ~ breastCancer ? flip(0.8) : flip(0.096)

    true ~ Dirac(positiveMammogram)

end

model = medical_diagnosis2()

chns = sample(model, MH(), 10_000)
# chns = sample(model, PG(15), 10_000)
# chns = sample(model, IS(), 10_000)
# chns = sample(model, SMC(), 10_000)

display(chns)


using Pkg; Pkg.add("Plots")
using Plots
# IJulia.clear_output(true)

# Define functions and parameters
H=4000.;L=H/10.;A=20.;C=50.;D=2.5;B=8.;E=5.e-4;
ft(x,L)=exp(-x/L)
th1(z)=A*ft.(-z .+ L, L);
th2(z)=C*ft(D*(-z .+ L), L);

# Plot for equally spaced discrete points
npt=1000;
z=-collect(range(0, 1, length=npt)) .* H
phi1=th1.(z)
phi2=th2.(z)
theta=phi1 .- phi2 .+ B .+ E .* z;
plot( theta ,z,label=false, lw=3, color = :black )
plot!(phi1 .+ B ,z,label=false , line=(:dash) )
plot!(phi2 .+ B ,z,label=false , line=(:dash) )

# Plot for ECCO 50-level discrete points
dzECCO50l=[10.00, 10.00, 10.00, 10.00, 10.00, 10.00, 10.00, 10.01,
    10.03, 10.11, 10.32, 10.80, 11.76, 13.42, 16.04 , 19.82,
    24.85,31.10, 38.42, 46.50, 55.00, 63.50, 71.58, 78.90,
     85.15, 90.18, 93.96, 96.58, 98.25, 99.25,100.01,101.33,
    104.56,111.33,122.83,139.09,158.94,180.83,203.55,226.50,
     249.50,272.50,295.50,318.50,341.50,364.50,387.50,410.50,
    433.50,456.50];
dz=dzECCO50l;
z=-(cumsum(dz).-dz./2);
phi1=th1.(z)
phi2=th2.(z)
theta=phi1 .- phi2 .+ B .+ E .* z;
plot( theta ,z,label=false, lw=3, color = :black )
plot!(phi1 .+ B ,z,label=false , line=(:dash) )
plot!(phi2 .+ B ,z,label=false , line=(:dash) )


#!/usr/bin/julia

# Date: 21 August 2016
# https://github.com/trizen

# An efficient algorithm for computing large Fibonacci numbers, modulus some n.
# Algorithm from: http://codeforces.com/blog/entry/14516

function fibmod(n::Int64, mod::Int64)

    n <= 1 && return n
    cache = Dict{Int64, Int64}()

    function f(n::Int64)

        n <= 1           && return 1
        haskey(cache, n) && return cache[n]

        k = div(n, 2)

        cache[n] =
            if n % 2 == 0
                (f(k) * f(k  ) + f(k-1) * f(k-1)) % mod
            else
                (f(k) * f(k+1) + f(k-1) * f(k  )) % mod
            end
    end

    f(n-1)
end

println(fibmod(1000, 10^4))


using Test
using Statistics
using DelimitedFiles
using Dates

@testset "nse" begin
        for i in 1: 1000
                Observed_Discharge = rand(5:0.5:80, 1000)
                Modelled_Discharge = rand(5:0.5:80, 1000)
                Mean_Observed_Discharge = ones(1000) * mean(Observed_Discharge)
                Nominator = sum((Modelled_Discharge - Observed_Discharge).^2)
                Denominator = sum((Observed_Discharge - Mean_Observed_Discharge).^2)
                NSE = 1 - Nominator / Denominator
                @test NSE == nse(Observed_Discharge, Modelled_Discharge)
                # observed is the same as modelled
                Modelled_Discharge = Observed_Discharge
                @test nse(Observed_Discharge, Modelled_Discharge) == 1
        end
end

@testset "nselog" begin
        for i in 1: 1000
                Observed_Discharge = rand(5:0.5:80, 1000)
                Modelled_Discharge = rand(5:0.5:80, 1000)
                Mean_Observed_Discharge = ones(1000) * mean(Observed_Discharge)
                Nominator = sum((log.(Modelled_Discharge) - log.(Observed_Discharge)).^2)
                Denominator = sum((log.(Observed_Discharge) - log.(Mean_Observed_Discharge)).^2)
                NSE = 1 - Nominator / Denominator
                @test NSE == lognse(Observed_Discharge, Modelled_Discharge)
                # observed is the same as modelled
                Modelled_Discharge = Observed_Discharge
                @test lognse(Observed_Discharge, Modelled_Discharge) == 1
        end
end

@testset "VE" begin
        for i in 1: 10000
                Observed_Discharge = rand(5:0.5:80, 1000)
                Modelled_Discharge = rand(5:0.5:80, 1000)
                Nominator = sum(abs.(Modelled_Discharge - Observed_Discharge))
                Denominator = sum(Observed_Discharge)
                VE = 1 - Nominator / Denominator
                @test VE == volumetricefficiency(Observed_Discharge, Modelled_Discharge)
                # observed is the same as modelled
                Modelled_Discharge = Observed_Discharge
                @test volumetricefficiency(Observed_Discharge, Modelled_Discharge) == 1
        end
end

@testset "FDC" begin
        for i in 1:1000
                Discharge = rand(5:0.5:80, 1000)
                SortedQ = sort(Discharge, rev = true)
                Rank = collect(1 : length(Discharge))
                @test minimum(Discharge) == SortedQ[end]
                @test maximum(Discharge) == SortedQ[1]
                Exceedance = Rank / (length(Discharge) + 1)
                SortedQ_Function, Exceedance_Function = flowdurationcurve(Discharge)
                @test Exceedance == Exceedance_Function
                @test SortedQ == SortedQ_Function
        end
end

@testset "autocorrelation" begin
        for i in 1:1000
                Discharge = rand(5:0.5:80, 1000)
                Timelag = 1
                Discharge_1 = Discharge[1 + Timelag: end]
                Discharge_0 = Discharge[1: end - Timelag]
                Mean_Discharge = ones(length(Discharge_0)) * mean(Discharge_0)
                Nominator = sum((Discharge_0 - Mean_Discharge) .* (Discharge_1 - Mean_Discharge))
                Denominator = sum((Discharge_0 - Mean_Discharge).^2)
                Correlation = Nominator / Denominator
                @test round(Correlation, digits=12) == round(autocorrelation(Discharge, Timelag), digits=12)
                #@test round(Correlation, digits=12) == round(autocorrelation2(Discharge, Timelag), digits = 12)
                # if timelag is zero correlation should be one
                Timelag = 0
                @test 1 == autocorrelation(Discharge, Timelag)
                #@test 1 == autocorrelation2(Discharge, Timelag)
        end
end

@testset "autocorrelationcurve" begin
        for i in 1: 1000
                Discharge = rand(5:0.5:80, 1000)
                Timelag = 30
                Discharge_0 = Discharge[1 : end - Timelag]
                Discharge_0_Shift = Discharge[1 + Timelag : end]
                Mean_Discharge = ones(length(Discharge_0)) * mean(Discharge_0)
                AC = zeros(Timelag + 1)
                AC_cor = zeros(Timelag + 1)
                count = 0
                count_large = 0
                for i in 0 : Timelag
                        Discharge_1 = Discharge[1 + i : end - Timelag + i]
                        Discharge_1_shift = Discharge[1 + Timelag - i : end - i]
                        Nominator = sum((Discharge_0 - Mean_Discharge) .* (Discharge_1 - Mean_Discharge))
                        Denominator = sum((Discharge_0 - Mean_Discharge).^2)
                        AC[i + 1] = Nominator / Denominator
                        @test AC[1] >= AC[i + 1]
                        if i > 0 && AC[i + 1] <= AC[i]
                                count += 1
                        elseif i > 0 && AC[i + 1] > AC[i]
                                count_large += 1
                        end
                        # for other function
                        AC_cor[i + 1] = cor(Discharge_0_Shift, Discharge_1_shift)
                end
                Lags = collect(0:Timelag)
                #@test count >= count_large
                AC_Function, Lags_Function = autocorrelationcurve(Discharge, Timelag)
                @test AC == AC_Function
                @test Lags == Lags_Function
                #test other function
                # AC_Function, Lags_Function = autocorrelationcurve(Discharge, Timelag)
                # @test AC_cor == AC_Function
                # @test Lags == Lags_Function
        end
end

@testset "runoff" begin
        for i in 1 :100
                Area = rand(50.:5.:500.)[1]
                Timespan = 1096
                Precipitation = rand(5:0.5:80, Timespan)
                Discharge = rand(5:0.5:80, Timespan)
                Timeseries = readdlm("Defreggental/tas_model_timeseries.txt")[1 : Timespan]
                Timeseries_DateFormat = Date.(Timeseries, dateformat"y,m,d")
                Runoff_1 = sum(Discharge[1:31]) / Area * (1000 * 3600 * 24) / sum(Precipitation[1:31])
                Runoff_2 = sum(Discharge[32:59]) / Area * (1000 * 3600 * 24) / sum(Precipitation[32:59])
                Runoff_Function, Month = monthlyrunoff(Area, Precipitation, Discharge,Timeseries_DateFormat)
                @test round(Runoff_Function[1], digits= 6) == round(Runoff_1, digits = 6)
                @test round(Runoff_Function[2], digits= 6) == round(Runoff_2, digits = 6)

                Runoff_mean_yearly = averagemonthlyrunoff(Area, Precipitation, Discharge,Timeseries_DateFormat)
                Runoff_51 = sum(Discharge[366: 366+30]) / Area * (1000 * 3600 * 24) / sum(Precipitation[366 : 30 + 366])
                Runoff_52 = sum(Discharge[731: 731+30]) / Area * (1000 * 3600 * 24) / sum(Precipitation[731: 30 + 731])
                meanRunoff = mean([Runoff_1 Runoff_51 Runoff_52])
                @test round(meanRunoff, digits=5) == round(Runoff_mean_yearly[1], digits=5)

                # if Precipitation is equal to Discharge than Runoff = 1
                Timespan = 31
                Precipitation = rand(5:0.5:80, Timespan)
                Discharge = rand(5:0.5:80, Timespan)
                Timeseries = readdlm("Defreggental/tas_model_timeseries.txt")[1 : Timespan]
                Timeseries_DateFormat = Date.(Timeseries, dateformat"y,m,d")
                Discharge = Precipitation * Area / (1000 * 3600 * 24)
                Runoff_Function, Month = monthlyrunoff(Area, Precipitation, Discharge,Timeseries_DateFormat)
                @test Runoff_Function[1] == 1.0

                # calculate yearly mean runoff
        end

end


using Test
using OrdinaryDiffEq
using SparseArrays
using LinearAlgebra

#https://github.com/JuliaDiffEq/SparseDiffTools.jl/blob/master/test/test_integration.jl
function f(dx,x,p,t)
    for i in 2:length(x)-1
      dx[i] = x[i-1] - 2x[i] + x[i+1]
    end
    dx[1] = -2x[1] + x[2]
    dx[end] = x[end-1] - 2x[end]
    nothing
  end

function second_derivative_stencil(N)
  A = zeros(N,N)
  for i in 1:N, j in 1:N
    (j-i==-1 || j-i==1) && (A[i,j]=1)
    j-i==0 && (A[i,j]=-2)
  end
  A
end

function generate_sparsity_pattern(N::Integer)
  dl = repeat([1.0],N-1)
  du = repeat([1.0],N-1)
  d = repeat([-2.0],N)
  return Tridiagonal(dl,d,du)
end

jac_sp = sparse(generate_sparsity_pattern(10))
#jac = second_derivative_stencil(10)
colors = repeat(1:3,10)[1:10]
u0=[1.,2.,3,4,5,5,4,3,2,1]
tspan=(0.,10.)
odefun_sp= ODEFunction(f,colorvec=colors,jac_prototype=jac_sp)
prob_sp = ODEProblem(odefun_sp,u0,tspan)
prob_std = ODEProblem(f,u0,tspan)

sol_sp=solve(prob_sp,Rodas5(autodiff=false),abstol=1e-10,reltol=1e-10)
@test sol_sp.retcode==:Success#test sparse finitediff
sol=solve(prob_std,Rodas5(autodiff=false),abstol=1e-10,reltol=1e-10)
@test sol_sp.u[end]≈sol.u[end] atol=1e-10
@test length(sol_sp.t)==length(sol.t)

sol_sp=solve(prob_sp,Rodas5(autodiff=false))
sol=solve(prob_std,Rodas5(autodiff=false))
@test sol_sp.u[end]≈sol.u[end]
@test length(sol_sp.t)==length(sol.t)

sol_sp=solve(prob_sp,Rodas5())
sol=solve(prob_std,Rodas5())
@test sol_sp.u[end]≈sol.u[end]
@test length(sol_sp.t)==length(sol.t)


data = [
    8.9 14.0 4.3 19.9 2.1 28.0 3.6 1.3 4.3;
    13.5 9.3 4.1 17.5 4.5 26.6 5.7 2.1 4.0;
    18.0 9.9 3.3 19.5 5.7 28.1 4.8 2.4 6.5;
    13.9 10.0 4.7 25.8 2.2 24.0 6.2 1.6 2.9;
    9.5 13.6 3.6 23.4 2.5 22.4 4.2 1.8 3.7;
    13.1 10.1 3.1 23.8 2.3 25.6 2.8 2.4 4.9;
    17.4 5.7 4.7 20.6 4.3 24.3 4.7 3.4 3.3;
    11.4 12.5 4.1 18.8 3.4 18.6 5.2 1.5 3.8
]

N = size(data)[1]
prototypes = copy(data)
clusters = [[i] for i = 1:N]
C = length(clusters)

d²(x⃗, y⃗) = sum((x⃗ - y⃗).^2)
d′(x⃗ᵢ, x⃗ⱼ, nᵢ, nⱼ) = ((nᵢ * nⱼ) / (nᵢ + nⱼ)) * sum((x⃗ᵢ - x⃗ⱼ).^2)

T = 2 # C

for t = 1:T
    D = [i<j ? d′(prototypes[i, :], prototypes[j, :], length(clusters[i]), length(clusters[j])) : (i == j ? Inf : Inf) for i = 1:C, j = 1:C]
    _, idx = findmin(D)
    i, j = idx[1], idx[2]

    append!(clusters[i], clusters[j])
    deleteat!(clusters, j)
    C = length(clusters)

    prototypes = Matrix{Float64}(undef, C, 9)
    for c = 1:C
        vectors = [data[k, :] for k in clusters[c]]
        mean = sum(vectors) / length(vectors)
        prototypes[c, :] = mean
    end

    variances = zeros(C)
    for r = 1:C
        for k in clusters[r]
            variances[r] += d²(data[k, :], prototypes[r, :])
        end
    end
    total_variance = sum(variances)
    println("total variance = $total_variance")
end


@testset "linear complementarity problem" begin
    M = [0  0 -1 -1 ;
         0  0  1 -2 ;
         1 -1  2 -2 ;
         1  2 -2  4 ]

    q = [2; 2; -2; -6]

    myfunc(x) = M*x + q

    n = 4
    lb = zeros(n)
    ub = 100*ones(n)

    status, z, f = solveLCP(myfunc, M, lb, ub)
    @show status
    @show z
    @show f
    @test isapprox(z, [2.8, 0.0, 0.8, 1.2])
    @test status == :Solved

    status, z, f = solveLCP(myfunc, lb, ub)
    @show status
    @show z
    @show f
    @test isapprox(z, [2.8, 0.0, 0.8, 1.2])
    @test status == :Solved

    println("-------------------------------------------------------")

    M = [0  0 -1 -1 ;
         0  0  1 -2 ;
         1 -1  2 -2 ;
         1  2 -2  4 ]

    q = [2; 2; -2; -6]

    myfunc(x) = M*x + q

    n = 4
    lb = zeros(n)
    ub = 100*ones(n)

    options(convergence_tolerance=1e-2, output=:no, time_limit=3600)

    status, z, f = solveLCP(myfunc, M, lb, ub)
    @show status
    @show z
    @show f
    @test isapprox(z, [2.8, 0.0, 0.8, 1.2])
    @test status == :Solved

    status, z, f = solveLCP(myfunc, lb, ub)
    @show status
    @show z
    @show f
    @test isapprox(z, [2.8, 0.0, 0.8, 1.2])
    @test status == :Solved

    println("-------------------------------------------------------")

    function elemfunc(x)
        val = similar(x)
        val[1] = -x[3]-x[4] + q[1]
        val[2] = x[3] -2x[4] + q[2]
        val[3] = x[1]-x[2]+2x[3]-2x[4] + q[3]
        val[4] = x[1]+2x[2]-2x[3]+4x[4] + q[4]
        return val
    end

    n = 4
    lb = zeros(n)
    ub = 100*ones(n)

    var_name = ["x1", "x2", "x3", "x4"]
    con_name = ["F1", "F2", "F3", "F4"]

    options(convergence_tolerance=1e-2, output=:yes, time_limit=3600)

    status, z, f = solveLCP(elemfunc, M, lb, ub)
    status, z, f = solveLCP(elemfunc, M, lb, ub, var_name)
    status, z, f = solveLCP(elemfunc, M, lb, ub, var_name, con_name)
    status, z, f = solveLCP(elemfunc, lb, ub)
    status, z, f = solveLCP(elemfunc, lb, ub, var_name)
    status, z, f = solveLCP(elemfunc, lb, ub, var_name, con_name)

    @show status
    @show z
    @show f

    @test isapprox(z, [2.8, 0.0, 0.8, 1.2])
    @test status == :Solved
end


using Derivatives, InplaceRealFFT
using LESFilter: rfftfreq, fftfreq
using Base.Test

#=
Test function: f = sin(3x)*cos(3y)*sin(4z)
=#
for (lx,ly,lz) in ((1.,1.,1.),(2.,2.,2.),(3.,3.,3.))

# lx = 1.0
# ly = 1.0
# lz = 1.0

nx = 32
ny = 32
nz = 32

x = reshape(linspace(0,lx*2π*(1-1/nx),nx),(nx,1,1))
y = reshape(linspace(0,ly*2π*(1-1/ny),ny),(1,ny,1))
z = reshape(linspace(0,lz*2π*(1-1/nz),nz),(1,1,nz))

field = @. sin(3*x) * cos(3*y) * sin(4*z)

correct(f::typeof(Derivatives.dx)) = @. 3*cos(3*x) * cos(3*y) * sin(4*z)
correct(f::typeof(Derivatives.dy)) = @. sin(3*x) * -3*sin(3*y) * sin(4*z)
correct(f::typeof(Derivatives.dz)) = @. sin(3*x) * cos(3*y) * 4*cos(4*z)

correct2(f::typeof(Derivatives.dx)) = @. -9*sin(3*x) * cos(3*y) * sin(4*z)
correct2(f::typeof(Derivatives.dy)) = @. sin(3*x) * -9*cos(3*y) * sin(4*z)
correct2(f::typeof(Derivatives.dz)) = @. sin(3*x) * cos(3*y) * -16*sin(4*z)

correct3(f::typeof(Derivatives.dx)) = @. -27*cos(3*x) * cos(3*y) * sin(4*z)
correct3(f::typeof(Derivatives.dy)) = @. sin(3*x) * 27*sin(3*y) * sin(4*z)
correct3(f::typeof(Derivatives.dz)) = @. sin(3*x) * cos(3*y) * -64*cos(4*z)

for (c,n) in zip((correct,correct2,correct3),(1,2,3))
  for (f,l) in zip((Derivatives.dx,Derivatives.dy,Derivatives.dz),(lx,ly,lz))
    @test f(field,l,n) ≈ c(f)
  end
end

field2 = PaddedArray(field)

for (c,n) in zip((correct,correct2,correct3),(1,2,3))
  for (f,l) in zip((Derivatives.dx,Derivatives.dy,Derivatives.dz),(lx,ly,lz))
    @test real(f(field,l,n)) ≈ c(f)
  end
end

rfft!(field2)

@inferred Derivatives.loopdx!(complex(field2),rfftfreq(nx,lx))
@inferred Derivatives.loopdy!(complex(field2),fftfreq(ny,ly))
@inferred Derivatives.loopdz!(complex(field2),fftfreq(nz,lz))

end


"""
```
function interp(x::Vector{T}, x_knot::Vector{T}) where T<:AbstractFloat
```
Compute linear interpolation over 1D grid. It extrapolates outside of
knot points. (SeHyoun Ahn, March 2017)

# Arguments:
- `x_fine`: x grid to interpolate to
- `x_knot`: x knot points to interpolate from

# Output:
`V`: matrix giving interpolation values

# Example:
```
x_fine = linspace(-1,1,300)'
x_knot = linspace(0,2,20)'
V = interp(x_fine, x_knot)
plot(V * (x_knot.^5 + x_knot * 0.5))
```
"""
function interp(x::Vector{T}, x_knot::Vector{T}) where {T<:AbstractFloat}
    loc = sum(broadcast(-, x, x_knot') .>= 0, dims=2)
    loc = max.(min.(loc, length(x_knot) - 1), 1)

    t = (x - x_knot[loc]) ./ (x_knot[loc+1] - x_knot[loc])
    ind_x = 1:length(x)

    i = vec(repeat(ind_x,2,1))
    j = [loc; loc + 1]
    v = [(1 - t); t]

    return SparseArrays.sparse(i, j, v, length(x), length(x_knot))
end

"""
```
function interp(x_fine::Vector{T}, y_fine::Vector{T},
                x_knot::Vector{T}, y_knot::Vector{T}) where {T<:Float64}
```
Compute linear interpolation over 2D grid. It extrapolates outside of
knot points. (SeHyoun Ahn, March 2017)

# Arguments
 - `x_fine::Vector{T}`: x grid to interpolate to
 - `y_fine::Vector{T}`: y grid to interpolate to
 - `x_knot::Vector{T}`: x knot points to interpolate from
 - `y_knot::Vector{T}`: y knot points to interpolate from

# Output
 - `V`: sparse matrix giving interpolation values

# Example
```
x_fine = linspace(0,2,80)'
y_fine = linspace(-1,1,100)'
x_knot = linspace(0,1,10)'
y_knot = linspace(0,1,10)'

V = interp(x_fine, y_fine, x_knot, y_knot)

z = broadcast(+, x_knot.^3 + exp(-x_knot), (y_knot' - 0.5).^2)
surf(x_fine,y_fine,reshape(V * vec(z), 80, 100)')
```
"""
function interp(x_fine::Vector{T}, y_fine::Vector{T},
                x_knot::Vector{T}, y_knot::Vector{T}) where {T<:Float64}

    n_x      = length(x_knot)
    ind_fine = 1:length(x_fine) * length(y_fine)

    loc_x = sum(broadcast(-, x_fine, x_knot') .>= 0, dims=2)
    loc_x = vec(max.(min.(loc_x, n_x - 1), 1))

    loc_y = sum(broadcast(-, y_fine, y_knot') .>= 0, dims=2)
    loc_y = max.(min.(loc_y, length(y_knot) - 1), 1)'

    t_x = (x_fine .- x_knot[loc_x]) ./ (x_knot[loc_x .+ 1] .- x_knot[loc_x])
    t_y = ((y_fine .- vec(y_knot[loc_y])) ./
           (vec(y_knot[loc_y .+ 1]) .- vec(y_knot[loc_y])))'

    j_SW = broadcast(+, loc_x,      n_x.*(loc_y .- 1))
    j_SE = broadcast(+, loc_x .+ 1, n_x.*(loc_y .- 1))
    j_NW = broadcast(+, loc_x,      n_x.*(loc_y))
    j_NE = broadcast(+, loc_x .+ 1, n_x.*(loc_y))
    v_SW = broadcast(*, (1 .- t_x), (1 .- t_y))
    v_SE = broadcast(*, (t_x),      (1 .- t_y))
    v_NW = broadcast(*, (1 .- t_x), (t_y))
    v_NE = broadcast(*, (t_x),      (t_y))

    i = vec(repeat(ind_fine, 4, 1))
    j = [vec(j_SW); vec(j_SE); vec(j_NW); vec(j_NE)]
    v = [vec(v_SW); vec(v_SE); vec(v_NW); vec(v_NE)]

    return SparseArrays.sparse(i, j, v, length(x_fine) * length(y_fine), n_x * length(y_knot))
end


# ---
# jupyter:
#   jupytext:
#     formats: ipynb,jl:light
#     text_representation:
#       extension: .jl
#       format_name: light
#       format_version: '1.4'
#       jupytext_version: 1.2.1
#   kernelspec:
#     display_name: Julia 1.1.1
#     language: julia
#     name: julia-1.1
# ---

# # DifferentialEquations.jl Workshop Exercise 1
#
# - Exercise 1 takes the user through solving a stiff ordinary differential equation
#   and using the ModelingToolkit.jl to automatically convert the function to a
#   symbolic form to derive the analytical Jacobian to speed up the solver. The
#   same biological system is then solved with stochasticity, utilizing
#   EnsembleProblems to understand 95% bounds on the solution. Finally,
#   probabilistic programming is employed to perform Bayesian parameter estimation
#   of the parameters against data.

# # Problem 1: Investigating Sources of Randomness and Uncertainty in a Stiff Biological System (B)
#
# In this problem we will walk through the basics of simulating models with
# DifferentialEquations.jl. Let's take the
# [Oregonator model of the Belousov-Zhabotinskii chemical reaction system](https://www.radford.edu/~thompson/vodef90web/problems/demosnodislin/Demos_Pitagora/DemoOrego/demoorego.pdf).
# This system describes a classical example in non-equilibrium thermodynmics
# and is a well-known natural chemical oscillator.
#
# ## Part 1: Simulating the Oregonator ODE model
#
# When modeling, usually one starts off by investigating the deterministic model.
# The deterministic ODE formulation of the Oregonator is
# given by the equations
#
# $$\begin{align}
# \frac{dx}{dt} &= s(y-xy + x - qx^2)\\
# \frac{dy}{dt} &= (-y - xy + z)/s\\
# \frac{dz}{dt} &= w(x - z)\end{align}$$

# +
using ModelingToolkit, DifferentialEquations, Plots

@parameters t s w q
@variables x(t) y(t) z(t)
@derivatives D'~t
# -

eqs = [D(x) ~ s*(y-x*y+x-q*x^2),
       D(y) ~ (-y - x*y + z)/s,
       D(z) ~ w*(x - z)]

de = ODESystem(eqs)

generate_function(de, [x,y,z], [s, w, q])

f = ODEFunction(de, [x,y,z], [s, w, q])

# with parameter values $s=77.27$, $w=0.161$, and $q=8.375 \times 10^{-6}$, and
# initial conditions $x(0)=1$, $y(0)=2$, and $z(0)=3$. Use
# [the tutorial on solving ODEs](http://docs.juliadiffeq.org/latest/tutorials/ode_example.html)
# to solve this differential equation on the
# timespan of $t\in[0,360]$ with the default ODE solver. To investigate the result,
# plot the solution of all components over time, and plot the phase space plot of
# the solution (hint: use `vars=(1,2,3)`). What shape is being drawn in phase space?

u0 = [1.0,2.0,3.0]
tspan = (0.0,360.0)
p = [77.27,0.161,8.375e-6]
prob = ODEProblem(f,u0,tspan,p)

sol = solve(prob);

plot(sol,vars=(1,2,3))

# ## Part 2: Investigating Stiffness
#
# Because the reaction rates of `q` vs `s` is very large, this model has a "fast"
# system and a "slow" system. This is typical of ODEs which exhibit a property
# known as stiffness. Stiffness changes the ODE solvers which can handle the
# equation well. [Take a look at the ODE solver page](http://docs.juliadiffeq.org/latest/solvers/ode_solve.html)
# and investigate solving the equation using methods for non-stiff equations
# (ex: `Tsit5`) and stiff equations (ex: `Rodas5`).
#
# Benchmark using $t\in[0,50]$ using `@btime` from BenchmarkTools.jl. What
# happens when you increase the timespan?

# ## (Optional) Part 3: Specifying Analytical Jacobians (I)
#
# Stiff ODE solvers internally utilize the Jacobian of the ODE system in order
# to improve the stepsizes in the solution. However, computing and factorizing
# the Jacobian is costly, and thus it can be beneficial to provide the analytical
# solution.
#
# Use the
# [ODEFunction definition page](http://docs.juliadiffeq.org/latest/features/performance_overloads.html)
# to define an `ODEFunction` which holds both the OREGO ODE and its Jacobian, and solve using `Rodas5`.
#
# ## (Optional) Part 4: Automatic Symbolicification and Analytical Jacobian Calculations
#
# Deriving Jacobians by hand is tedious. Thankfully symbolic mathematical systems
# can do the work for you. And thankfully, DifferentialEquations.jl has tools
# to automatically convert numerical problems into symbolic problems to perform
# the analysis on!
#
# follow the [ModelingToolkit.jl README](https://github.com/JuliaDiffEq/ModelingToolkit.jl)
# to automatically convert your ODE definition
# to its symbolic form using `modelingtoolkitize` and calculate the analytical
# Jacobian. Use the compilation functions to build the `ODEFunction` with the
# embedded analytical solution.
#
# ## Part 5: Adding stochasticity with stochastic differential equations
#
# How does this system react in the presense of stochasticity? We can investigate
# this question by using stochastic differential equations. A stochastic
# differential equation formulation of this model is known as the multiplicative
# noise model, is created with:
#
# $$\begin{align}
# dx &= s(y-xy + x - qx^2)dt + \sigma_1 x dW_1\\
# dy &= \frac{-y - xy + z}{s}dt + \sigma_2 y dW_2\\
# dz &= w(x - z)dt + \sigma_3 z dW_3\end{align}$$
#
# with $\sigma_i = 0.1$ where the `dW` terms describe a Brownian motion, a
# continuous random process with normally distributed increments. Use the
# [tutorial on solving SDEs](http://docs.juliadiffeq.org/latest/tutorials/sde_example.html)
# to solve simulate this model. Then,
# [use the `EnsembleProblem`](http://docs.juliadiffeq.org/latest/features/ensemble.html)
# to generate and plot 100 trajectories of the stochastic model, and use
# `EnsembleSummary` to plot the mean and 5%-95% region over time.
#
# Try solving with the `ImplicitRKMil` and `SOSRI` methods. Notice that it isn't
# stiff every single time!
#
# (For fun, see if you can make the Euler-Maruyama `EM()` method solve this equation.
# This requires a choice of `dt` small enough to be stable. This is the "standard"
# method!)
#
# ## Part 6: Gillespie jump models of discrete stochasticity
#
# When biological models have very few particles, continuous models no longer
# make sense, and instead using the full discrete formulation can be required
# to accuracy describe the dynamics. A discrete differential equation, or
# Gillespie model, is a continuous-time Markov chain with Poisson-distributed
# jumps. A discrete description of the Oregonator model is given by a chemical
# reaction systems:

A+Y -> X+P
X+Y -> 2P
A+X -> 2X + 2Z
2X  -> A + P (note: this has rate kX^2!)
B + Z -> Y

# where reactions take place at a rate which is propoertional to its components,
# i.e. the first reaction has a rate `k*A*Y` for some `k`.
# Use the [tutorial on Gillespie SSA models](http://docs.juliadiffeq.org/latest/tutorials/discrete_stochastic_example.html)
# to implement the `JumpProblem` for this model, and use the `EnsembleProblem`
# and `EnsembleSummary` to characterize the stochastic trajectories.
#
# For what rate constants does the model give the oscillatory dynamics for the
# ODE approximation? For information on the true reaction rates, consult
# [the original paper](https://pubs.acs.org/doi/abs/10.1021/ja00780a001).
#
# ## Part 7: Probabilistic Programming / Bayesian Parameter Estimation with DiffEqBayes.jl + Turing.jl (I)
#
# In many casees, one comes to understand the proper values for their model's
# parameters by utilizing data fitting techniques. In this case, we will use
# the DiffEqBayes.jl library to perform a Bayesian estimation of the parameters.
# For our data we will the following potential output:

t = 0.0:1.0:30.0
data = [1.0 2.05224 2.11422 2.1857 2.26827 2.3641 2.47618 2.60869 2.7677 2.96232 3.20711 3.52709 3.97005 4.64319 5.86202 9.29322 536.068 82388.9 57868.4 1.00399 1.00169 1.00117 1.00094 1.00082 1.00075 1.0007 1.00068 1.00066 1.00065 1.00065 1.00065
        2.0 1.9494 1.89645 1.84227 1.78727 1.73178 1.67601 1.62008 1.56402 1.50772 1.45094 1.39322 1.33366 1.2705 1.19958 1.10651 0.57194 0.180316 0.431409 251.774 591.754 857.464 1062.78 1219.05 1335.56 1419.88 1478.22 1515.63 1536.25 1543.45 1539.98
        3.0 2.82065 2.68703 2.58974 2.52405 2.48644 2.47449 2.48686 2.52337 2.58526 2.67563 2.80053 2.9713 3.21051 3.5712 4.23706 12.0266 14868.8 24987.8 23453.4 19202.2 15721.6 12872.0 10538.8 8628.66 7064.73 5784.29 4735.96 3877.66 3174.94 2599.6]

# [Follow the exmaples on the parameter estimation page](http://docs.juliadiffeq.org/latest/analysis/parameter_estimation.html#Bayesian-Methods-1)
# to perform a Bayesian parameter estimation. What are the most likely parameters
# for the model given the posterior parameter distributions?
#
# Use the `ODEProblem` to perform the fit. If you have time, use the `EnsembleProblem`
# of `SDEProblem`s to perform a fit over averages of the SDE solutions. Note that
# the SDE fit will take significantly more computational resources! See the GPU
# parallelism section for details on how to accelerate this.
#
# ## (Optional) Part 8: Using DiffEqBiological's Reaction Network DSL
#
# DiffEqBiological.jl is a helper library for the DifferentialEquations.jl
# ecosystem for defining chemical reaction systems at a high leevel for easy
# simulation in these various forms. Use the descrption
# [from the Chemical Reaction Networks documentation page](http://docs.juliadiffeq.org/latest/models/biological.html)
# to build a reaction network and generate the ODE/SDE/jump equations, and
# compare the result to your handcoded versions.
#
#


@doc raw"""
```
Z,p,q = retrieve_surface(algorithm::DiscreteShape, img::AbstractArray, iterations::Int=2000)
```
Attempts to produce a heightmap from a grayscale image by minimization of a set
of Euler-Lagrange equations. This is done discretely at each point in the image
utilizing the second derivates of the surface normals and their Fourier Transforms.
# Output
Returns an M by N array of Float `Z` that represents the reconstructed height at
the point and the gradients in M by N arrays of Float 'p' and 'q'.
# Details
The algorithm attempts to minimise the brightness deviation ``ϵ`` through the
brightness constraint ``\epsilon_1`` and smoothness constraint ``\epsilon_2``
as defined bellow:
```math
\epsilon=\iint((\epsilon_1+\lambda\epsilon_2))dxdy=\iint((E(x,y)-R(p,q))^2+
\lambda(p_x^2+q_x^2+p_y^2+q_y^2))
```
This is minimised using the Euler-Lagrange equations defined as;
```math
\dfrac{\delta\epsilon}{\delta p}-\dfrac{\delta}{\delta x}\dfrac{\delta\epsilon}{
\delta p_x}-\dfrac{\delta}{\delta y}\dfrac{\delta\epsilon}{\delta p_y}=0
```
and
```math
\dfrac{\delta
\epsilon}{\delta q}-\dfrac{\delta}{\delta x}\dfrac{\delta\epsilon}{\delta q_x}-
\dfrac{\delta}{\delta y}\dfrac{\delta\epsilon}{\delta q_y}=0
```
which become:
```math
\begin{gathered}
-2(E-R)\dfrac{\delta R}{\delta p}-2\lambda p_{xx}-2\lambda p_{yy}=0\\
-2(E-R)\dfrac{\delta R}{\delta p}-2\lambda q_{xx}-2\lambda q_{yy}=0
\end{gathered}
```
which can be further simplified to give:
```math
\begin{gathered}
\nabla^2p=\dfrac{1}{\lambda}(R-E)\dfrac{\delta R}{\delta p}\\
\nabla^2q=\dfrac{1}{\lambda}(R-E)\dfrac{\delta R}{\delta q}
\end{gathered}
```
where ``\nabla^2p=p_{xx}+p_{yy}`` and ``\nabla^2q=q_{xx}+q_{yy}`` are Laplacians
of p and q.

However, for computation we are dealing with a discreate case of these equations
which can be defined as below:
```math
\begin{gathered}
p_{i,j}=\bar{p}_{i,j}+\dfrac{1}{4\lambda}(E-R)\dfrac{\delta R}{\delta p}\\q_{i,j}=
\bar{q}_{i,j}+\dfrac{1}{4\lambda}(E-R)\dfrac{\delta R}{\delta q}\\
\end{gathered}
```
where
```math
\bar{p}_{i,j}=\dfrac{p_{i+1,j}+p_{i-1,j}+p_{i,j+1}+p_{i,j-1}}{4}
```
and
```math
\bar{q}_{i,j}=\dfrac{q_{i+1,j}+q_{i-1,j}+q_{i,j+1}+q_{i,j-1}}{4}
```
Finally, the algorithm needs to enforce integrability on p and q and retrieve
the surface Z. This can be done by taking the Fast Fourier Transform of p and q
()``c_p(\omega_x,\omega_y)`` and ``c_q(\omega_x,\omega_y)``) and then using the
Inverse Fast Fourier Transform to recover Z and update p and q as per bellow:
```math
\begin{gathered}
p=\sum c_p(\omega_x,\omega_y)e^{j(\omega_xx+\omega_yy)}\\
q=\sum c_q(\omega_x,\omega_y)e^{j(\omega_xx+\omega_yy)}\\Z=\sum c(\omega_x,\omega_y)
e^{j(\omega_xx+\omega_yy)}\\
\end{gathered}
```
where
```math
c(\omega_x,\omega_y)=\dfrac{-j(\omega_xc_p(\omega_x,\omega_y)+\omega_yc_q(\omega_x
,\omega_y))}{\omega_x^2+\omega_y^2}
```

The `albedo` and `illumination_direction` can be manually defined using the
function signature:
```
Z,p,q = retrieve_surface(algorithm::DiscreteShape, img::AbstractArray, albedo::Real, illumination_direction::Vector{T} where T <: Real, iterations::Int=2000)
```
Note: if `albedo` and `illumination_direction` are not supplied they will be
calculated at runtime using `estimate_img_properties`.
# Arguments
The function arguments are described in more detail below.
##  `img`
An `AbstractArray` storing the grayscale value of each pixel within
the range [0,1].
## `albedo`
A `Real` that specifies the albedo (amount of light reflected) of the image. If
`albedo` is specified to must the `illumination_direction`.
## `illumination_direction`
A `Vector{T} where T <: Real` that specifies the tilt value to be used by the
algorithm. The `illumination_direction` should be a vector of the form [x,y,z]
where x,y,z are int he range [0,1]. If `illumination_direction` is specified
to must the `albedo`.
## `iterations`
An `Int` that specifies the number of iterations the algorithm is to perform. If
left unspecified a default value of 2000 is used.
## `smoothness`
An `Int` that specifies the strength of the smoothness constraint in the minimised
function.
# Example
Compute the heightmap for a synthetic image generated by `generate_surface`.
```julia
using Images, Makie, ShapeFromShading

#generate synthetic image
img = generate_surface(SynthSphere(), 1, [0.2,0,0.9], radius = 5)

#calculate the heightmap (using 500 iterations)
Z,p,q = retrieve_surface(DiscreteShape(), img, 500)

#normalize to maximum of 1 (not necessary but makes displaying easier)
Z = Z./maximum(Z)

#display using Makie (Note: Makie can often take several minutes first time)
r = 0.0:0.1:2
surface(r, r, Z)
```
# Reference
1. S. Elhabian, "Hands on Shape from Shading", Computer Vision and Image Processing, 2008.
"""
function retrieve_surface(algorithm::DiscreteShape, img::AbstractArray, iterations::Int=2000; smoothness::Int=1000)
    ρ,I,σ,τ = estimate_img_properties(img)
    λ=smoothness
    return retrieve_surface(DiscreteShape(), img, ρ, I, iterations, smoothness=λ)
end

function retrieve_surface(algorithm::DiscreteShape, img::AbstractArray, albedo::Real, illumination_direction::Vector{T} where T <: Real, iterations::Int=2000; smoothness::Int=1000)
    ρ = albedo
    I = illumination_direction
    E = Array{Float64}(img)

    #initialize variables
    p = zeros(Complex{Float64},axes(E))
    q = zeros(Complex{Float64},axes(E))
    R = zeros(Complex{Float64},axes(E))
    Z = zeros(axes(E))
    return solve_EulerLagrange(ρ, I, iterations, p, q, R, smoothness, E, Z)
end


# algorithm on page 504 of https://people.maths.ox.ac.uk/trefethen/barycentric.pdf
function barycentric_lagrange_weights(x)
    n = length(x) - 1
    w = similar(x)
    w[1] = 1.0
    for j in 1:n
        for k in 0:j-1
            w[k+1] = (x[k+1] - x[j+1]) * w[k+1]
        end
        w[j+1] = prod(0:j-1) do k
            x[j+1] - x[k+1]
        end
    end
    for j in 0:n
        w[j+1] = 1/w[j+1]
    end
    return w
end

# equation 4.2 of https://people.maths.ox.ac.uk/trefethen/barycentric.pdf
function barycentric_lagrange_matrix(xs,xt,w=barycentric_lagrange_weights(xs))
    ns = length(xs)
    nt = length(xt)
    d = similar(xt)
    for i in 1:nt
        d[i] = sum(zip(xs,w)) do (x,w)
            w /(xt[i]-x)
        end
    end
    A= [w[j]/(xt[i]-xs[j]) / d[i] for i in 1:nt, j in 1:ns]
    return A
end


@testset "Numerical integral" begin
    gl1 = BasicBSpline.GaussLegendre(1)
    gl2 = BasicBSpline.GaussLegendre(2)
    gl3 = BasicBSpline.GaussLegendre(3)
    gl4 = BasicBSpline.GaussLegendre(4)

    I = 2..8
    coef = rand(10)

    f(n,x) = sum(coef[i]*x^(i-1) for i in 1:n)
    F(n,x) = sum(coef[i]*x^(i)/i for i in 1:n)

    @test F(1,maximum(I)) - F(1,minimum(I)) ≈ BasicBSpline.integrate(x->f(1,x), I, gl1)
    @test F(2,maximum(I)) - F(2,minimum(I)) ≈ BasicBSpline.integrate(x->f(2,x), I, gl1)
    @test F(3,maximum(I)) - F(3,minimum(I)) ≈ BasicBSpline.integrate(x->f(3,x), I, gl2)
    @test F(4,maximum(I)) - F(4,minimum(I)) ≈ BasicBSpline.integrate(x->f(4,x), I, gl2)
    @test F(5,maximum(I)) - F(5,minimum(I)) ≈ BasicBSpline.integrate(x->f(5,x), I, gl3)
    @test F(6,maximum(I)) - F(6,minimum(I)) ≈ BasicBSpline.integrate(x->f(6,x), I, gl3)
    @test F(7,maximum(I)) - F(7,minimum(I)) ≈ BasicBSpline.integrate(x->f(7,x), I, gl4)
    @test F(8,maximum(I)) - F(8,minimum(I)) ≈ BasicBSpline.integrate(x->f(8,x), I, gl4)
end


######### StanSample example  ###########

using StanSample

bernoulli_model = "
data {
  int<lower=1> N;
  int<lower=0,upper=1> y[N];
}
parameters {
  real<lower=0,upper=1> theta;
}
model {
  theta ~ beta(1,1);
  y ~ bernoulli(theta);
}
";

bernoulli_data = [
  Dict("N" => 10, "y" => [0, 1, 0, 1, 0, 0, 0, 0, 0, 1]),
  Dict("N" => 10, "y" => [0, 1, 0, 0, 1, 0, 0, 0, 0, 1]),
  Dict("N" => 10, "y" => [0, 1, 0, 1, 0, 0, 0, 0, 1, 0]),
  Dict("N" => 10, "y" => [0, 0, 0, 1, 0, 0, 1, 0, 0, 1]),
]

# Keep tmpdir identical across multiple runs to prevent re-compilation
stanmodel = SampleModel(
  "bernoulli", bernoulli_model;
  method =  StanSample.Sample(adapt=StanSample.Adapt(delta=0.85)))

rc = stan_sample(stanmodel, data=bernoulli_data)

if success(rc)

  # Fetch the same output in the `sdf` ChainDataFrame
  sdf = read_summary(stanmodel)

  @test sdf[sdf.parameters .== :theta, :mean][1] ≈ 0.33 rtol=0.05

end


using LinearAlgebra

const n = 8
a, b = legendre_coefs(Float64, n)

Tn = SymTridiagonal(copy(a), b[2:n])
F = eigen(Tn)

v = zeros(n)
special_eigenproblem!(a, b, v, 30)

idx = sortperm(a)
@test a[idx] ≈ F.values
@test abs.(v[idx]) ≈ abs.(F.vectors[1,:])


# indicator of the ball of matrices with (at most) a given rank

"""
  IndBallRank(r::Int=1)

Returns the function `g = ind{X : rank(X) ⩽ r}`, for an integer parameter `r > 0`.
"""

immutable IndBallRank{I <: Integer} <: IndicatorNonconvex
  r::I
  function IndBallRank(r::I)
    if r <= 0
      error("parameter r must be a positive integer")
    else
      new(r)
    end
  end
end

IndBallRank{I <: Integer}(r::I=1) = IndBallRank{I}(r)

function (f::IndBallRank){T <: RealOrComplex}(x::AbstractArray{T,2})
  maxr = minimum(size(x))
  if maxr <= f.r return 0.0 end
  svdobj = svds(x, nsv=f.r+1)[1]
  # the tolerance in the following line should be customizable
  if svdobj[:S][end]/svdobj[:S][1] <= 1e-14
    return 0.0
  end
  return +Inf
end

function prox!{T <: Real}(f::IndBallRank, x::AbstractArray{T,2}, y::AbstractArray{T,2}, gamma::Real=1.0)
  maxr = minimum(size(x))
  if maxr <= f.r
    y[:] = x
    return 0.0
  end
  svdobj = svds(x, nsv=f.r)[1]
  for i = 1:size(x,1)
    for j = 1:size(x,2)
      y[i,j] = 0.0
      for k = 1:f.r
          if VERSION >= v"0.6.0-dev.2071"
              y[i,j] += svdobj[:U][i,k]*svdobj[:S][k]*svdobj[:Vt][k,j]
          else
              y[i,j] += svdobj[:U][i,k]*svdobj[:S][k]*svdobj[:Vt][j,k]
          end
      end
    end
  end
  return 0.0
end

function prox!{T <: Complex}(f::IndBallRank, x::AbstractArray{T,2}, y::AbstractArray{T,2}, gamma::Real=1.0)
  maxr = minimum(size(x))
  if maxr <= f.r
    y[:] = x
    return 0.0
  end
  svdobj = svds(x, nsv=f.r)[1]
  for i = 1:size(x,1)
    for j = 1:size(x,2)
      y[i,j] = 0.0 + 0.0im
      for k = 1:f.r
          if VERSION >= v"0.6.0-dev.2071"
              y[i,j] += svdobj[:U][i,k]*svdobj[:S][k]*svdobj[:Vt][k,j]
          else
              y[i,j] += svdobj[:U][i,k]*svdobj[:S][k]*conj(svdobj[:Vt][j,k])
          end
      end
    end
  end
  return 0.0
end

fun_name(f::IndBallRank) = "indicator of the set of rank-r matrices"
fun_dom(f::IndBallRank) = "AbstractArray{Real,2}, AbstractArray{Complex,2}"
fun_expr(f::IndBallRank) = "x ↦ 0 if rank(x) ⩽ r, +∞ otherwise"
fun_params(f::IndBallRank) = "r = $(f.r)"

function prox_naive{T <: RealOrComplex}(f::IndBallRank, x::AbstractArray{T,2}, gamma::Real=1.0)
  maxr = minimum(size(x))
  if maxr <= f.r
    y = x
    return y, 0.0
  end
  U, S, V = svd(x)
  M = U[:,1:f.r]*spdiagm(S[1:f.r])
  y = M*V[:,1:f.r]'
  return y, 0.0
end


export σx, σy, σz, 𝟙, up, down, ⊗, speye, identity_op, single_spin_op, correlator, op_list, symmetrize_state, symmetrize_op, nspins

const σx = sparse([0 1; 1 0])
const σy = sparse([0 -im; im 0])
const σz = sparse([1 0; 0 -1])
const 𝟙 = spdiagm([1, 1])

const up = [1,0]
const down = [0,1]

const ⊗ = kron

speye(k::Integer) = spdiagm([1 for _ in 1:k])
identity_op(k::Integer) = speye(2^k)

"""
    nspins(obj)

Give number of spins in the system this object belongs to.
"""
function nspins(system_size::Int)
    N = Int(ceil(log2(system_size)))
    2^N == system_size || throw(ArgumentError("$(system_size) is not a power of 2!"))
    N
end
nspins(state::AbstractVector) = nspins(length(state))
nspins(op::AbstractMatrix) = size(op,1) == size(op, 2) ? nspins(size(op, 1)) : throw(ArgumentError("The operator is not a square-matrix!"))

single_spin_op(op, k::Integer, N::Integer) = identity_op(k-1) ⊗ op ⊗ identity_op(N-k)
correlator(op, i::Integer, j::Integer, N::Integer) = i > j ? correlator(op, j, i, N) : identity_op(i-1) ⊗ op ⊗ identity_op(j-i-1) ⊗ op ⊗ identity_op(N-j)

op_list(op, N::Integer) = [single_spin_op(op, k, N) for k in 1:N]


# # Welford's algorithm
#
# * Welford, B. P. (1962). "Note on a method for calculating corrected sums of squares and products". Technometrics. 4 (3): 419–420. doi:10.2307/1266577. JSTOR 1266577
# * [Welford's online algorithm - Algorithms for calculating variance - Wikipedia](https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Welford's_online_algorithm)

using Transducers
using Transducers: R_, start, next, complete, inner, xform, wrap, unwrap, wrapping

struct MeanVar <: Transducer
end

function Transducers.start(rf::R_{MeanVar}, result)
    private_state = (0, 0.0, 0.0)
    return wrap(rf, private_state, start(inner(rf), result))
end

function Transducers.next(rf::R_{MeanVar}, result, input)
    wrapping(rf, result) do st, iresult
        (n, μ, M2) = st
        n += 1
        δ = input - μ
        μ += δ/n
        δ2 = input - μ
        M2 += δ*δ2
        iinput = (μ, M2 / (n-1))
        iresult = next(inner(rf), iresult, iinput)
        return (n, μ, M2), iresult
    end
end

function Transducers.complete(rf::R_{MeanVar}, result)
    _private_state, inner_result = unwrap(rf, result)
    return complete(inner(rf), inner_result)
end

collect(MeanVar(),randn(10))

@time foldl(right, MeanVar(), 1:100)


module ReactiveMPModelsLGSSMTest

using Test, InteractiveUtils
using Rocket, ReactiveMP, GraphPPL, Distributions
using BenchmarkTools, Random, Plots, Dates, LinearAlgebra, StableRNGs

## Model definition
## -------------------------------------------- ##
@model function univariate_lgssm_model(n, x0, c_, P_)

    x_prior ~ NormalMeanVariance(mean(x0), cov(x0))

    x = randomvar(n)
    c = constvar(c_)
    P = constvar(P_)
    y = datavar(Float64, n)

    x_prev = x_prior

    for i in 1:n
        x[i] ~ x_prev + c
        y[i] ~ NormalMeanVariance(x[i], P)
        x_prev = x[i]
    end

    return x, y
end
## -------------------------------------------- ##
@model function multivariate_lgssm_model(n, x0, A, B, Q, P)

    # We create constvar references for better efficiency
    cA = constvar(A)
    cB = constvar(B)
    cQ = constvar(Q)
    cP = constvar(P)

    # `x` is a sequence of hidden states
    x = randomvar(n)
    # `y` is a sequence of "clamped" observations
    y = datavar(Vector{Float64}, n)

    x_prior ~ MvNormalMeanCovariance(mean(x0), cov(x0))
    x_prev = x_prior

    for i in 1:n
        x[i] ~ MvNormalMeanCovariance(cA * x_prev, cQ)
        y[i] ~ MvNormalMeanCovariance(cB * x[i], cP)
        x_prev = x[i]
    end

    return x, y
end
## -------------------------------------------- ##
## Inference definition
## -------------------------------------------- ##
function univariate_lgssm_inference(data, x0, c, P)
    n = length(data)

    model, (x, y) = univariate_lgssm_model(n, x0, c, P);

    x_buffer = buffer(Marginal, n)
    fe       = keep(Float64)

    x_sub = subscribe!(getmarginals(x), x_buffer)
    f_sub = subscribe!(score(Float64, BetheFreeEnergy(), model), fe)

    update!(y, data)

    unsubscribe!((x_sub, f_sub))

    return x_buffer, fe
end
## -------------------------------------------- ##
function multivariate_lgssm_inference(data, x0, A, B, Q, P)
    n = length(data)

    model, (x, y) = multivariate_lgssm_model(n, x0, A, B, Q, P, options = (limit_stack_depth = 100, ));

    xbuffer = buffer(Marginal, n)
    fe      = keep(Float64)

    x_sub = subscribe!(getmarginals(x), xbuffer)
    f_sub  = subscribe!(score(Float64, BetheFreeEnergy(), model), fe)

    update!(y, data)

    unsubscribe!((x_sub, f_sub))

    return xbuffer, fe
end
## -------------------------------------------- ##

@testset "Linear Gaussian State Space Model" begin

    @testset "Univariate" begin
        ## -------------------------------------------- ##
        ## Data creation
        ## -------------------------------------------- ##
        rng = StableRNG(123)
        P   = 100.0
        n   = 500
        hidden   = collect(1:n)
        data     = hidden + rand(rng, Normal(0.0, sqrt(P)), n);
        x0_prior = NormalMeanVariance(0.0, 10000.0)
        ## -------------------------------------------- ##
        ## Inference execution
        x_estimated, fe = univariate_lgssm_inference(data, x0_prior, 1.0, P);
        ## -------------------------------------------- ##
        ## Test inference results
        @test length(x_estimated) === n
        @test all((mean.(x_estimated) .- 3 .* std.(x_estimated)) .< hidden .< (mean.(x_estimated) .+ 3 .* std.(x_estimated)))
        @test all(var.(x_estimated) .> 0.0)
        @test length(fe) === 1
        @test abs(last(fe) - 1854.297647) < 0.01
        ## -------------------------------------------- ##
        ## Form debug output
        base_output = joinpath(pwd(), "_output", "models")
        mkpath(base_output)
        timestamp        = Dates.format(now(), "dd-mm-yyyy-HH-MM")
        plot_output      = joinpath(base_output, "lgssm_univariate_plot_$(timestamp)_v$(VERSION).png")
        benchmark_output = joinpath(base_output, "lgssm_univariate_benchmark_$(timestamp)_v$(VERSION).txt")
        ## -------------------------------------------- ##
        ## Create output plots
        subrange = 200:215
        m = mean.(x_estimated)[subrange]
        s = std.(x_estimated)[subrange]
        p = plot()
        p = plot!(subrange, m, ribbon = s, label = "Estimated signal")
        p = plot!(subrange, hidden[subrange], label = "Hidden signal")
        p = scatter!(subrange, data[subrange], label = "Observations")
        savefig(p, plot_output)
        ## -------------------------------------------- ##
        ## Create output benchmarks
        benchmark = @benchmark univariate_lgssm_inference($data, $x0_prior, 1.0, $P) seconds=15
        open(benchmark_output, "w") do io
            show(io, MIME("text/plain"), benchmark)
            versioninfo(io)
        end
        ## -------------------------------------------- ##
    end

    @testset "Multivariate" begin
        ## -------------------------------------------- ##
        ## Data creation
        ## -------------------------------------------- ##
        function generate_data(rng, A, B, Q, P)
            x_prev = [ 10.0, -10.0 ]

            x = Vector{Vector{Float64}}(undef, n)
            y = Vector{Vector{Float64}}(undef, n)

            for i in 1:n
                x[i] = rand(rng, MvNormal(A * x_prev, Q))
                y[i] = rand(rng, MvNormal(B * x[i], P))
                x_prev = x[i]
            end

            return x, y
        end
        ## -------------------------------------------- ##
        # Seed for reproducibility
        rng = StableRNG(1234)
        # We will model 2-dimensional observations with rotation matrix `A`
        # To avoid clutter we also assume that matrices `A`, `B`, `P` and `Q`
        # are known and fixed for all time-steps
        θ = π / 35
        A = [ cos(θ) -sin(θ); sin(θ) cos(θ) ]
        B = diageye(2)
        Q = diageye(2)
        P = 25.0 .* diageye(2)
        n = 1_000 # Number of observations
        x, y = generate_data(rng, A, B, Q, P)
        x0 = MvNormalMeanCovariance(zeros(2), 100.0 * diageye(2));
        ## -------------------------------------------- ##
        ## Inference execution
        xmarginals, fe = multivariate_lgssm_inference(y, x0, A, B, Q, P);
        ## Test inference results
        @test length(xmarginals) === n
        # We use 3.0var instead of 3.0std here for easier dot broadcasting with mean
        @test all((mean.(xmarginals) .- 3.0 .* var.(xmarginals)) .< x .< (mean.(xmarginals) .+ 3.0 .* var.(xmarginals)))
        @test all(isposdef.(cov.(xmarginals)))
        @test length(fe) === 1
        @test abs(last(fe) - 6275.9015944677) < 0.01
        ## -------------------------------------------- ##
        ## Form debug output
        base_output = joinpath(pwd(), "_output", "models")
        mkpath(base_output)
        timestamp        = Dates.format(now(), "dd-mm-yyyy-HH-MM")
        plot_output      = joinpath(base_output, "lgssm_multivariate_plot_$(timestamp)_v$(VERSION).png")
        benchmark_output = joinpath(base_output, "lgssm_multivariate_benchmark_$(timestamp)_v$(VERSION).txt")
        ## -------------------------------------------- ##
        ## Create output plots
        slicedim(dim) = (a) -> map(e -> e[dim], a)

        subrange = 100:500
        px = plot()

        px = plot!(px, x[subrange] |> slicedim(1), label = "Hidden Signal (dim-1)", color = :orange)
        px = plot!(px, x[subrange] |> slicedim(2), label = "Hidden Signal (dim-2)", color = :green)

        px = plot!(px, mean.(xmarginals)[subrange] |> slicedim(1), ribbon = var.(xmarginals)[subrange] |> slicedim(1) .|> sqrt, fillalpha = 0.5, label = "Estimated Signal (dim-1)", color = :teal)
        px = plot!(px, mean.(xmarginals)[subrange] |> slicedim(2), ribbon = var.(xmarginals)[subrange] |> slicedim(2) .|> sqrt, fillalpha = 0.5, label = "Estimated Signal (dim-1)", color = :violet)

        savefig(px, plot_output)
        ## -------------------------------------------- ##
        ## Create output benchmarks
        benchmark = @benchmark multivariate_lgssm_inference($y, $x0, $A, $B, $Q, $P) seconds=15
        open(benchmark_output, "w") do io
            show(io, MIME("text/plain"), benchmark)
            versioninfo(io)
        end
        ## -------------------------------------------- ##
    end

end

end


# # Time Dependent Problems
#
# ![](transient_heat.gif)
#
# *Figure 1*: Visualization of the temperature time evolution on a unit
# square where the prescribed temperature on the upper and lower parts
# of the boundary increase with time.
#
#-
#md # !!! tip
#md #     This example is also available as a Jupyter notebook:
#md #     [`transient_heat_equation.ipynb`](@__NBVIEWER_ROOT_URL__/examples/transient_heat_equation.ipynb).
#-
#
# ## Introduction
#
# In this example we extend the heat equation by a time dependent term, i.e.
# ```math
#  \frac{\partial u}{\partial t}-\nabla \cdot (k \nabla u) = f  \quad x \in \Omega,
# ```
#
# where $u$ is the unknown temperature field, $k$ the heat conductivity,
# $f$ the heat source and $\Omega$ the domain. For simplicity we set $f = 1$
# and $k = 1$. We define homogeneous Dirichlet boundary conditions along the left and right edge of the domain.
# ```math
# u(x,t) = 0 \quad x \in \partial \Omega_1,
# ```
# where $\partial \Omega_1$ denotes the left and right boundary of $\Omega$.
#
# Further, we define heterogeneous Dirichlet boundary conditions at the top and bottom edge $\partial \Omega_2$.
# We choose a linearly increasing function $a(t)$ that describes the temperature at this boundary
# ```math
# u(x,t) = a(t) \quad x \in \partial \Omega_2.
# ```
# The semidiscrete weak form is given by
# ```math
# \int_{\Omega}\frac{\partial u}{\partial t}\cdot v \ d\Omega + \int_{\Omega} \nabla v \cdot \nabla u \ d\Omega = \int_{\Omega} v \ d\Omega,
# ```
# where $v$ is a suitable test function. Now, we still need to discretize the time derivative. An implicit Euler scheme is applied,
# which yields:
# ```math
# \int_{\Omega}u_{n+1}\cdot v \ d\Omega + \Delta t\int_{\Omega} \nabla v \cdot \nabla u_{n+1} \ d\Omega = \Delta t\int_{\Omega} v \ d\Omega + \int_{\Omega} u_{n}\cdot v \ d\Omega.
# ```
# If we assemble the discrete operators, we get the following algebraic system:
# ```math
# \mathbf{M} \mathbf{u}_{n+1} + Δt \mathbf{K} \mathbf{u}_{n+1} = Δt \mathbf{f} + \mathbf{M} \mathbf{u}_{n}
# ```
# In this example we apply the boundary conditions to the assembled discrete operators (mass matrix $\mathbf{M}$ and stiffnes matrix $\mathbf{K}$)
# only once. We utilize the fact that in finite element computations Dirichlet conditions can be applied by
# zero out rows and columns that correspond
# to a prescribed dof in the system matrix ($\mathbf{A} = Δt \mathbf{K} + \mathbf{M}$) and setting the value of the right-hand side vector to the value
# of the Dirichlet condition. Thus, we only need to apply in every time step the Dirichlet condition to the right-hand side of the problem.
#-
# ## Commented Program
#
# Now we solve the problem in Ferrite. What follows is a program spliced with comments.
#md # The full program, without comments, can be found in the next [section](@ref heat_equation-plain-program).
#
# First we load Ferrite, and some other packages we need.
using Ferrite, SparseArrays
# We create the same grid as in the heat equation example.
grid = generate_grid(Quadrilateral, (100, 100));

# ### Trial and test functions
# Again, we define the structs that are responsible for the `shape_value` and `shape_gradient` evaluation.
dim = 2
ip = Lagrange{dim, RefCube, 1}()
qr = QuadratureRule{dim, RefCube}(2)
cellvalues = CellScalarValues(qr, ip);

# ### Degrees of freedom
# After this, we can define the `DofHandler` and distribute the DOFs of the problem.
dh = DofHandler(grid)
push!(dh, :u, 1)
close!(dh);

# By means of the `DofHandler` we can allocate the needed `SparseMatrixCSC`.
# `M` refers here to the so called mass matrix, which always occurs in time related terms, i.e.
# ```math
# M_{ij} = \int_{\Omega} u_i \cdot v_j \ d\Omega,
# ```
# where $u_i$ and $v_j$ are trial and test functions, respectively.
K = create_sparsity_pattern(dh);
M = create_sparsity_pattern(dh);
# We also preallocate the right hand side
f = zeros(ndofs(dh));

# ### Boundary conditions
# In order to define the time dependent problem, we need some end time `T` and something that describes
# the linearly increasing Dirichlet boundary condition on $\partial \Omega_2$.
max_temp = 100
Δt = 1
T = 200
ch = ConstraintHandler(dh);

# Here, we define the boundary condition related to $\partial \Omega_1$.
∂Ω₁ = union(getfaceset.((grid, ), ["left", "right"])...)
dbc = Dirichlet(:u, ∂Ω₁, (x, t) -> 0)
add!(ch, dbc);
# While the next code block corresponds to the linearly increasing temperature description on $\partial \Omega_2$.
∂Ω₂ = union(getfaceset.((grid, ), ["top", "bottom"])...)
dbc = Dirichlet(:u, ∂Ω₂, (x, t) -> t*(max_temp/T))
add!(ch, dbc)
close!(ch)
update!(ch, 0.0);

# ### Assembling the linear system
# As in the heat equation example we define a `doassemble!` function that assembles the diffusion parts of the equation:
function doassemble_K!(K::SparseMatrixCSC, f::Vector, cellvalues::CellScalarValues{dim}, dh::DofHandler) where {dim}

    n_basefuncs = getnbasefunctions(cellvalues)
    Ke = zeros(n_basefuncs, n_basefuncs)
    fe = zeros(n_basefuncs)

    assembler = start_assemble(K, f)

    @inbounds for cell in CellIterator(dh)

        fill!(Ke, 0)
        fill!(fe, 0)

        reinit!(cellvalues, cell)

        for q_point in 1:getnquadpoints(cellvalues)
            dΩ = getdetJdV(cellvalues, q_point)

            for i in 1:n_basefuncs
                v  = shape_value(cellvalues, q_point, i)
                ∇v = shape_gradient(cellvalues, q_point, i)
                fe[i] += v * dΩ
                for j in 1:n_basefuncs
                    ∇u = shape_gradient(cellvalues, q_point, j)
                    Ke[i, j] += (∇v ⋅ ∇u) * dΩ
                end
            end
        end

        assemble!(assembler, celldofs(cell), fe, Ke)
    end
    return K, f
end
#md nothing # hide
# In addition to the diffusive part, we also need a function that assembles the mass matrix `M`.
function doassemble_M!(M::SparseMatrixCSC, cellvalues::CellScalarValues{dim}, dh::DofHandler) where {dim}

    n_basefuncs = getnbasefunctions(cellvalues)
    Me = zeros(n_basefuncs, n_basefuncs)

    assembler = start_assemble(M)

    @inbounds for cell in CellIterator(dh)

        fill!(Me, 0)

        reinit!(cellvalues, cell)

        for q_point in 1:getnquadpoints(cellvalues)
            dΩ = getdetJdV(cellvalues, q_point)

            for i in 1:n_basefuncs
                v  = shape_value(cellvalues, q_point, i)
                for j in 1:n_basefuncs
                    u = shape_value(cellvalues, q_point, j)
                    Me[i, j] += (v ⋅ u) * dΩ
                end
            end
        end

        assemble!(assembler, celldofs(cell), Me)
    end
    return M
end
#md nothing # hide
# ### Solution of the system
# We first assemble all parts in the prior allocated `SparseMatrixCSC`.
K, f = doassemble_K!(K, f, cellvalues, dh)
M = doassemble_M!(M, cellvalues, dh)
A = (Δt .* K) + M;
# Now, we need to save all boundary condition related values of the unaltered system matrix `A`, which is done
# by `get_rhs_data`. The function returns a `RHSData` struct, which contains all needed informations to apply
# the boundary conditions solely on the right-hand-side vector of the problem.
rhsdata = get_rhs_data(ch, A);
# We set the initial time step, denoted by uₙ,  to $\mathbf{0}$.
uₙ = zeros(length(f));
# Here, we apply **once** the boundary conditions to the system matrix `A`.
apply!(A, ch);

# To store the solution, we initialize a `paraview_collection` (.pvd) file.
pvd = paraview_collection("transient-heat.pvd");

# At this point everything is set up and we can finally approach the time loop.
for t in 0:Δt:T
    #First of all, we need to update the Dirichlet boundary condition values.
    update!(ch, t)

    #Secondly, we compute the right-hand-side of the problem.
    b = Δt .* f .+ M * uₙ
    #Then, we can apply the boundary conditions of the current time step.
    apply_rhs!(rhsdata, b, ch)

    #Finally, we can solve the time step and save the solution afterwards.
    u = A \ b;

    vtk_grid("transient-heat-$t", dh) do vtk
        vtk_point_data(vtk, dh, u)
        vtk_save(vtk)
        pvd[t] = vtk
    end
   #At the end of the time loop, we set the previous solution to the current one and go to the next time step.
   uₙ .= u
end
# In order to use the .pvd file we need to store it to the disk, which is done by:
vtk_save(pvd);

#md # ## [Plain program](@id transient_heat_equation-plain-program)
#md #
#md # Here follows a version of the program without any comments.
#md # The file is also available here:
#md # [`transient_heat_equation.jl`](transient_heat_equation.jl).
#md #
#md # ```julia
#md # @__CODE__
#md # ```


using jInv.Mesh;
using FactoredEikonalFastMarching;

include("../examples/getAnalyticalMediums.jl");

function runSensitivityTest()
tic()
println("########### 2D TEST SETTING ##############################");
n = [100,100]
Omega = [0.0,4.0,0.0,8.0];
Msh = getRegularMesh(Omega,n-1);

(kappaSquared,src,T_exact) = getAnalyticalConstGradInv2D(Msh.n+1,Msh.h);
T0 = getAnalytic2DeikonalSolution(n,Msh.h,src)[1];
println("First order test");
doSensTest(kappaSquared,src,T_exact,Msh,T0,false);
println("Second order test");
doSensTest(kappaSquared,src,T_exact,Msh,T0,true);
##########################################################
println("########### 3D TEST SETTING ##############################");
n = [64,64,32]
Omega = [0.0,1.0,0.0,1.0,0.0,0.5];
Msh = getRegularMesh(Omega,n-1);
(kappaSquared,src,T_exact) = getAnalyticalConstGradInv3D(Msh.n+1,Msh.h);
T0 = getAnalytic3DeikonalSolution(n,Msh.h,src)[1];
println("First order test");
doSensTest(kappaSquared,src,T_exact,Msh,T0,false);
println("Second order test");
doSensTest(kappaSquared,src,T_exact,Msh,T0,true);

##########################################################
toc()
return;
end

function doSensTest(kappaSquared,src,T_exact,Msh,T0,HO)
n = Msh.n+1;
pMem = getEikonalTempMemory(n);
pEik = getEikonalParam(Msh,kappaSquared,src,HO);
solveFastMarchingUpwindGrad(pEik,pMem);
T = T0[:].*pEik.T1[:];
println("Maximum error norm: ",maximum(abs((T_exact[:] - T))));
println("Sensitivity Check:");
delta_kappa = 0.05*randn(size(kappaSquared))*mean(kappaSquared[:]);
Tlin = zeros(prod(n));
for k = 1:10
	delta_kappa *= 0.5;
	new_kappa = kappaSquared + delta_kappa;
	pEikNew = getEikonalParam(Msh,new_kappa,src,HO);
	solveFastMarchingUpwindGrad(pEikNew,pMem);
	getSensMatVecEik(delta_kappa[:],Tlin,pEik);
	Tlin += T;
	print("||dm||: ");
	print(norm(delta_kappa[:]))
	print(". ||T(m + dm) - T(m)|| ");
	Tnew = T0[:].*pEikNew.T1[:];
	err0 = (T - Tnew);
	print(norm(err0));
	print(". ||T(m + dm) - (T(m) + J*dm)||: ");
	err = Tnew - Tlin;
	println(norm(err));

end
v = rand(prod(n));
Jv = zeros(prod(n));
getSensMatVecEik(v,Jv,pEik);
Z = rand(size(Jv));
JtZ = rand(prod(n));
getSensTMatVecEik(Z,JtZ,pEik);
if abs(dot(JtZ[:],v) - dot(Jv[:],Z[:])) > 1e-8
	println("Checking that (J^T)^T = J for eikonal solver falied: ",abs(dot(JtZ[:],v) - dot(Jv[:],Z[:])) );
end
return;
end