### A Pluto.jl notebook ###
# v0.16.0

using Markdown
using InteractiveUtils

# This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error).
macro bind(def, element)
    quote
        local el = $(esc(element))
        global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : missing
        el
    end
end

# ╔═╡ 7224ff21-b911-417d-b42c-b2aac200d930
begin
	using PlutoUI
end

# ╔═╡ 66ee80df-26f5-4d6b-8d2b-deba85ef5a2e
md"""

# Selecting and Filtering Patients Based on Data Values

In this notebook we will be exploring tools we will be using during the next two weeks to split patients into different groups based on properties that the patients do or do not share. We will start with the conceptual basis for this using __sets__. We will then look at how we can achieve these sets using __Boolean logic__.

"""

# ╔═╡ 025f1c63-7f8e-4b8c-b470-b35f86f19901
md"""
# The Beautiful (and Famous) Julia Set

![Julia Set](https://upload.wikimedia.org/wikipedia/commons/thumb/b/bd/Julia_set_(indigo).png/640px-Julia_set_(indigo).png)
"""

# ╔═╡ 1da07d3d-3dd5-48e8-a679-fbeef8128091
md"""

## The Key Idea of Sets: Commonality

To define a set we first need to select the property or properties that __all__ members (elements) of the set have in __common__. The properties need to be such that we can ask quesitons about that property that result in a __YES__ (__True__) or __NO__ (__False__) answer.

### Examples using Numbers:

- The set of all even integers ("Is 3 even? No (False)", "Is 6 even? Yes (True)")
- The set of all odd integers ("Is 3 odd?", "Is 6 odd?")
- The set of all integers greater than 6. ("Is 5 greatr than 6?", "Is 10 greater than six?")

#### Exercise

Can you think of at least two more questions you could ask about numbers to create sets of numbers that create a set? (Remember, all members of sets have at least one property in common.)

$(@bind numex1 TextField((80,7)))

"""

# ╔═╡ 51c6ff9c-dc99-422d-83d4-fece9bb0068d
md"""

### Example: Game of Set

The game Set consists of cards that have four properties

1. Shape
1. Fill Pattern
1. Color
1. Number

![game of Set](https://upload.wikimedia.org/wikipedia/commons/8/8f/Set-game-cards.png)

A set consists of three cards which satisfy ALL of these conditions:

* They all have the same number, __OR__ they have three different numbers.

__AND__

* They all have the same symbol, __OR__ they have three different symbols.

__AND__

* They all have the same shading, __OR__ they have three different shadings.

__AND__

* They all have the same color, __OR__ they have three different colors. ([From wikipedia](https://en.wikipedia.org/wiki/Set_(game)))

This game explicitly several of the concepts that we will use to group patients.

First, notice that every condition for the cards can be stated as a __YES__ or __NO__ question? For example,

- "Do the three cards all have the same shading?"
- "Do the three cards have three different symbols?"

Further, the conditions introduce two of the key words and the corresponding set concepts that we will use to group patients:

- __AND__: This corresponds to the set concept of __INTERSECTION__ (frequently denoted with the symbol __$\cap$__)
- __OR__: This corresponds to the set concept of __UNION__ (frequently denoted with the symbol __$\cup$__)

### One Last Word

In order to construct our sets we need one last word that is

__NOT__ (frequently denoted with the symbol __!__ or __$\neg$__)

To help understand these concepts we will use [Venn diagrams](https://en.wikipedia.org/wiki/Venn_diagram). (Venn diagrams have limited utility for more then three sets.)
"""

# ╔═╡ 6d1b3eb5-6f2b-48c6-b8dd-73fbf6c38130
html"""

If you have forgotten your childhood Venn diagram lessons, watch this short video.

"""

# ╔═╡ 9917ccf9-1503-4dd9-9bd7-c9a28e9ef7a6
md"""

### Intersection (AND) of Two sets

#### Example

- Let set __F__ be the set of all patients who are Female
- Let set __O__ be the set of all patients with a BMI > 30.

Consider the question

- "Who are the female patients with BMI greater than 30?

We can rewrite this as

- "Who are the patients who are female (members of set F) __AND__ have a BMI greater than 30 (members of set O)?"

This would be written mathematically as

- $\bf{F} \cap \bf{O}?$

and represented in the yellow region in the Venn diagram below.

![Intersection of two sets](https://github.com/chapmanbe/raw-stuff/blob/main/EulerVenn/venn_intersect_fo.png?raw=true)

### Union (OR) of Two Sets

#### Example

- Let set __N__ be the set of all patients who have [disorder of kidney due to diabetes mellitus (nephropathy)](http://purl.bioontology.org/ontology/SNOMEDCT/127013003).
- Let set __R__ be the set of all patients with a retinopathy.

Consider the question

- "Who are the patients with nephropathy __OR__ [retinopathy due to diabetes mellitus (retinopathy)](http://purl.bioontology.org/ontology/SNOMEDCT/4855003)?

This would be written mathematically as

- $\bf{N} \cup \bf{R}?$

and represented in the green region in the Venn diagram below.

![Union of two sets](https://github.com/chapmanbe/raw-stuff/blob/main/EulerVenn/venn_union_nr.png?raw=true)

#### Example

- Let set __F__ be the set of all patients who are Female
- Let set __O__ be the set of all patients with a BMI > 30.

Consider the question

- "Who are the female patients who are not obese?"

This could be rewritten as

- "Who are the patients who are female __AND NOT__ obese?"

This can be written mathematically as

- $\bf{F} \cap \neg \bf{O}?$

![Difference of two sets](https://github.com/chapmanbe/raw-stuff/blob/main/EulerVenn/venn_diff_fo.png?raw=true)
"""

# ╔═╡ 95d7109f-314d-4851-ac59-4bf059626d24
md"""

## Exploration with the [MIMIC II](https://archive.physionet.org/mimic2/) Demo Data Set

MIMIC (Medical Information Mart for Intensive Care) is a fully de-identified data set consisting of hospital-generated data collected on patients with an ICU admission. The current release of MIMIC is [MIMIC-IV](https://mimic.mit.edu/docs/iv/)

### MIMIC-II

MIMIC-II is an earlier relesae of MIMIC

>The MIMIC II...Databases contain physiologic signals and vital signs time series captured from patient monitors, and comprehensive clinical data obtained from hospital medical information systems, for tens of thousands of Intensive Care Unit (ICU) patients*. Data were collected between 2001 and 2008 from a variety of ICUs (medical, surgical, coronary care, and neonatal) in a single tertiary teaching hospital.

Access to the full MIMIC data sets requires ethics training and a data use agreement. However, the MIMIC-II data set includes a demo data set that consists of the data for 4000 deceased inviduals and is available without any restrictions. Even as a small data set there is a nearly overwhelming amount of data.

- 5074 admissions
- 5844 ICU stays
- 3740682 lab orders
- 1048968 medication orders
- 90000 microbiology orders
- 34240621 chart events
- 2471191 io events
- 171927 notes

#### Diabetes in MIMIC-II Demo

For illustrative purposes I extracted some of the data that might be similar to data in our diabetes project. It is also illustrative of how standards can help work with data and how we create sets with YES/NO questions

- __Diagnoses:__ Diagnoses were determined using the ICD9 codes assigned to the patient at discharge.
  - Does the patient have a type I diabetes diagnosis?
  - Does the patient have a type II diabetes diagnosis?
  - Number of non-diabetes diagnoses
- A1C Values
- Glucose Blood
- marital_status
- ethnicty
- payor
- religion
- admission
- sex
- age at death
- hospitaldeath
- DaysInHospital
- height
- weight
- bmi

"""

# ╔═╡ 646ceac0-b72c-4b87-a3a1-0888d9bd4a83
md"""
### Exercise:

In this and the following examples the sets are represented using Euler-Venn diagrams, which try to represent not just the intersection of sets but the relative sizes of the sets by the areas of the circles.

Given the two set _TypeI_ and _TypeII_, what combinations of __AND__, __OR__, and or __NOT__ would yield the set represented by the crosshatched area?

![all diabetics](https://github.com/chapmanbe/raw-stuff/blob/main/EulerVenn/Alldiabetics.png?raw=true)
"""

# ╔═╡ a3ed9688-4459-4270-856d-bfb9bcd9a562

HTML("My Answer
TypeI OR Type II
")

# ╔═╡ 9ec579ae-059a-46e3-b73c-089271a4b83f
md"""

### Exercise:

Given the two set _TypeI_ and _TypeII_, what combinations of __AND__, __OR__, and or __NOT__ would yield the set represented by the crosshatched area?

![all diabetics](https://github.com/chapmanbe/raw-stuff/blob/main/EulerVenn/DiabeticAndObese.png?raw=true)
"""

# ╔═╡ 4ed2f1f8-1249-4679-b93b-a54684163729

HTML("My Answers
Obese AND (TypeI OR Type II)
")

# ╔═╡ 5aef28d5-8ffd-45f9-a547-6bb41e359bc0
md"""

### Exercise:

Given the two set _TypeI_ and _TypeII_, what combinations of __AND__, __OR__, and or __NOT__ would yield the set represented by the crosshatched area?

![all diabetics](https://github.com/chapmanbe/raw-stuff/blob/main/EulerVenn/Diedobesediabetic.png?raw=true)
"""

# ╔═╡ 392c63d4-75aa-4df1-b0fc-40702c8745b8

HTML("My Answers
Obese AND Type II AND DiedAtHospital
")

# ╔═╡ 47c88a48-5392-4b2f-abe6-8593657587bf
md"""## Selecting based on Numeric Values

All of our examples so far with the MIMIC data set have been asking questions of categorical data (e.g. sex, died at the hospital). But as we showed with the numbers earlier we can still form YES/NO questions with numeric values. Here is an example.

#### Example

A1c above 7 (HighA1c) is viewed as problematic and is a basis for diagnosing diabetes. The biological significance of low A1c (below 4) is not well known. However, a [recent study](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4734630/), found that low A1c was associated with increased mortality. So we might consider A1c vaues < 4 or greater than 7 as problematic:

- "What patients have TypeII diabetes __AND__ have A1c values less than 4.0 __OR__ have A1c values greater than 7?"

- $\text{TypeII} \cup ((\text{A1C} < 4) \cup (\text{A1C} > 7))$

![a1c problems](https://github.com/chapmanbe/raw-stuff/blob/main/EulerVenn/diabetes4.png?raw=true)

"""

# ╔═╡ c51953b1-d267-4206-879d-63a139d12afa
md"""
# Additional set information

If you are interested in more complete (yet still brief) review of set concepts and notation, check out [this page](https://plato.stanford.edu/entries/set-theory/basic-set-theory.html).
"""

# ╔═╡ b481618c-13b7-4fd3-bf21-e736b6902fb9
md"""
### Answers in set notation

- $\text{TypeI} \cup \text{TypeII}$
- $\text{Obese} \cap (\text{TypeI} \cup \text{TypeII})$
- $\text{DiedInHospital} \cap \text{Obese} \cap \text{TypeII}$
"""

# ╔═╡ 00000000-0000-0000-0000-000000000001
PLUTO_PROJECT_TOML_CONTENTS = """
[deps]
PlutoUI = "7f904dfe-b85e-4ff6-b463-dae2292396a8"

[compat]
PlutoUI = "~0.7.9"
"""

# ╔═╡ 00000000-0000-0000-0000-000000000002
PLUTO_MANIFEST_TOML_CONTENTS = """
# This file is machine-generated - editing it directly is not advised

[[Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[JSON]]
deps = ["Dates", "Mmap", "Parsers", "Unicode"]
git-tree-sha1 = "8076680b162ada2a031f707ac7b4953e30667a37"
uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
version = "0.21.2"

[[Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"

[[Parsers]]
deps = ["Dates"]
git-tree-sha1 = "438d35d2d95ae2c5e8780b330592b6de8494e779"
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
version = "2.0.3"

[[PlutoUI]]
deps = ["Base64", "Dates", "InteractiveUtils", "JSON", "Logging", "Markdown", "Random", "Reexport", "Suppressor"]
git-tree-sha1 = "44e225d5837e2a2345e69a1d1e01ac2443ff9fcb"
uuid = "7f904dfe-b85e-4ff6-b463-dae2292396a8"
version = "0.7.9"

[[Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[Random]]
deps = ["Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[Reexport]]
git-tree-sha1 = "45e428421666073eab6f2da5c9d310d99bb12f9b"
uuid = "189a3867-3050-52da-a836-e630ba90ab69"
version = "1.2.2"

[[Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[Suppressor]]
git-tree-sha1 = "a819d77f31f83e5792a76081eee1ea6342ab8787"
uuid = "fd094767-a336-5f1f-9728-57cf17d0bbfb"
version = "0.2.0"

[[Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"
"""

# ╔═╡ Cell order:
# ╟─66ee80df-26f5-4d6b-8d2b-deba85ef5a2e
# ╟─025f1c63-7f8e-4b8c-b470-b35f86f19901
# ╟─1da07d3d-3dd5-48e8-a679-fbeef8128091
# ╟─51c6ff9c-dc99-422d-83d4-fece9bb0068d
# ╟─6d1b3eb5-6f2b-48c6-b8dd-73fbf6c38130
# ╟─9917ccf9-1503-4dd9-9bd7-c9a28e9ef7a6
# ╟─95d7109f-314d-4851-ac59-4bf059626d24
# ╟─646ceac0-b72c-4b87-a3a1-0888d9bd4a83
# ╟─a3ed9688-4459-4270-856d-bfb9bcd9a562
# ╟─9ec579ae-059a-46e3-b73c-089271a4b83f
# ╟─4ed2f1f8-1249-4679-b93b-a54684163729
# ╟─5aef28d5-8ffd-45f9-a547-6bb41e359bc0
# ╟─392c63d4-75aa-4df1-b0fc-40702c8745b8
# ╠═47c88a48-5392-4b2f-abe6-8593657587bf
# ╟─7224ff21-b911-417d-b42c-b2aac200d930
# ╟─c51953b1-d267-4206-879d-63a139d12afa
# ╟─b481618c-13b7-4fd3-bf21-e736b6902fb9
# ╟─00000000-0000-0000-0000-000000000001
# ╟─00000000-0000-0000-0000-000000000002


using SchattenNorms
using Test
using LinearAlgebra

import Random
import LinearAlgebra

Random.seed!(31415926)

@testset "snorm sanity checks" begin
    @test isapprox(snorm([1 0; 0 -1]),sqrt(2.0))
    @test isapprox(snorm([1 0; 0 -1],1.0),2.0)
    @test isapprox(snorm([1 0; 0 -1],Inf),1.0)
end

@testset "snorm" begin
    for i=1:100
        for d=2:4
            R = randn(d,d)
            U = LinearAlgebra.svd(randn(d,d)).U
            V = LinearAlgebra.svd(randn(d,d)).U
            p = sort(abs.(randn(3)) .+ 1.0)
            @test isapprox(snorm(R),snorm(R,2.0))
            @test isapprox(snorm(U*R*V,1),snorm(R,1))
            @test isapprox(snorm(U*R*V),snorm(R,2.0))
            @test isapprox(snorm(U*R*V,Inf),snorm(R,Inf))
            @test isapprox(snorm(U*R*V,p[1]),snorm(R,p[1]))
            @test snorm(R,1) >= snorm(R,2) >= snorm(R,Inf)
            @test snorm(R,p[1]) >= snorm(R,p[2]) >= snorm(R,p[3])
            @test snorm(R,1.0) == snorm(R,1) == trnorm(R) == nucnorm(R)
        end
    end
end

# E
# U,V
# @test isapprox(cbnorm(E),dnorm(E'))
# @test isapprox(cbnorm(E),cbnorm(U*E*V))
# @test isapprox(dnorm(E),dnorm(U*E*V))

include("test-dnorm.jl")


function f2(x)
  x[1]^2 + (2.0 - x[2])^2
end
function g2(x, storage)
  storage[1] = 2.0 * x[1]
  storage[2] = -2.0 * (2.0 - x[2])
end
d2 = DifferentiableFunction(f2, g2)
initial_x = [100.0, 100.0]

results = Optim.bfgs(d2, initial_x)
@assert length(results.trace.states) == 0
@assert results.converged
@assert norm(results.minimum - [0.0, 2.0]) < 0.01


# Test of Min-Cut and maximum adjacency visit

using LightGraphs
using Base.Test

g = Graph(8)

# Original example by Stoer

wedges = [
    (1, 2, 2.),
    (1, 5, 3.),
    (2, 3, 3.),
    (2, 5, 2.),
    (2, 6, 2.),
    (3, 4, 4.),
    (3, 7, 2.),
    (4, 7, 2.),
    (4, 8, 2.),
    (5, 6, 3.),
    (6, 7, 1.),
    (7, 8, 3.) ]

m = length(wedges)
eweights = spzeros(nv(g),nv(g))

for (s, d, w) in wedges
    add_edge!(g, s, d)
    eweights[s, d] = w
    eweights[d, s] = w
end

@test nv(g) == 8
@test ne(g) == m

parity, bestcut = mincut(g, eweights)

@test length(parity) == 8
@test parity == [2, 2, 1, 1, 2, 2, 1, 1]
@test bestcut == 4.0

parity, bestcut = mincut(g)

@test length(parity) == 8
@test parity == [2, 1, 1, 1, 1, 1, 1, 1]
@test bestcut == 2.0

v = maximum_adjacency_visit(g)

@test v == Vector{Int64}([1, 2, 5, 6, 3, 7, 4, 8])


@testset "azimuth.jl" begin
    p1 = rad2deg(Point(0.5, 1.0))
    p2 = rad2deg(Point(1.0, 0.5))
    p3 = rad2deg(Point(0.6, 1.0))  # above p1

    rs = Arc(p1,p2)
    @test azimuth(p1, p2) ≈ rad2deg(5.81412807071) atol = 0.001
    @test azimuth(rs) ≈ rad2deg(5.81412807071) atol = 0.001
    @test azimuth(p3, p1) ≈ 180.0 atol = 0.001

    @test final_azimuth(p1, p2) ≈ rad2deg(5.45864813531) atol = 0.001
    @test final_azimuth(rs) ≈ rad2deg(5.45864813531) atol = 0.001

    angular_distance_x = 34.49
    angular_distance_y = 28.11
    angular_distance_z = 29.0

    @test spherical_angle(angular_distance_z, angular_distance_y, angular_distance_x) ≈ 76.6 atol = 0.1
    @test spherical_angle(angular_distance_x, angular_distance_z, angular_distance_y) ≈ 54.1 atol = 0.1
    @test spherical_angle(angular_distance_y, angular_distance_x, angular_distance_z) ≈ 56.4 atol = 0.1

    @test spherical_excess(angular_distance_z, angular_distance_y, angular_distance_x) ≈ 7.1 atol = 0.1

    angular_distance_x2 = 35.0
    angular_distance_y2 = 35.0
    angular_distance_z2 = 0.1

    @test spherical_angle(angular_distance_z2, angular_distance_y2, angular_distance_x2) ≈ 89.9 atol = 0.1
    @test spherical_angle(angular_distance_x2, angular_distance_z2, angular_distance_y2) ≈ 89.9 atol = 0.1
    @test spherical_angle(angular_distance_y2, angular_distance_x2, angular_distance_z2) ≈ 0.174 atol = 0.001

end


export entropy,spectrum
"""
    Calculate the von-Neumann entanglement entropy of an MPS at bond_id
"""
function entropy(mps::AbstractMPS,bond_id::Int)
    S = spectrum(mps,bond_id)
    S = S./sqrt(sum(S.*S))  #normalize MPS
    S = S .+ 1E-100
    return sum(- S.*S .* log.(S.*S))
end

"""
    Calculate von-Neumann entanglement entropy of every bonds
"""
function entropy(mps::AbstractMPS,epsilon=1E-13)
    function spectrum2entropy(S::AbstractVector)
        t = S./sqrt(sum(S.*S))  #normalize MPS
        t = t .+ 1E-100
        return sum(- t.*t .* log.(t.*t))
    end
    # Calculate the von-Neumann entanglement entropy of an MPS
    #from left to right, qr
    v_entropy = zeros(Float64,mps.L-1)
    sweep!(mps,mps.L-1,LeftNormalization())
    #print (mps.bdim)
    #from right to left, svd
    for site = mps.L:-1:2
        S = single_tensor_spectrum!(mps,site)
        v_entropy[site-1] = spectrum2entropy(S)
    end
    return v_entropy
end

"""
Calculate Spectrum in MPS: from site-bond_id to site-bond_id+1
"""
function spectrum(mps::AbstractMPS,bond_id::Int)
    res = 0.0
    @assert bond_id>0 && bond_id <= mps.L
    normalization!(mps,bond_id,MixNormalization())
    U,S,V = svd(reshape(mps[bond_id],mps.bdim[bond_id-1]*mps.S,mps.bdim[bond_id]))
    return S
end


# Cost functions
"""
    mse(ŷ, y)

Return the mean squared error `sum((ŷ .- y).^2) / length(y)`.
"""
mse(ŷ, y) = sum((ŷ .- y).^2) * 1 // length(y)

function _crossentropy(ŷ::AbstractVecOrMat, y::AbstractVecOrMat, weight::Nothing)
  return -sum(y .* log.(ŷ)) * 1 // size(y, 2)
end

function _crossentropy(ŷ::AbstractVecOrMat, y::AbstractVecOrMat, weight::Number)
  return -sum(y .* log.(ŷ)) .* weight * 1 // size(y, 2)
end

function _crossentropy(ŷ::AbstractVecOrMat, y::AbstractVecOrMat, weight::AbstractVector)
  return -sum(y .* log.(ŷ) .* weight) * 1 // size(y, 2)
end

"""
    crossentropy(ŷ, y; weight=1)

Return the crossentropy computed as `-sum(y .* log.(ŷ) .* weight) / size(y, 2)`.

See also [`logitcrossentropy`](@ref), [`binarycrossentropy`](@ref).
"""
crossentropy(ŷ::AbstractVecOrMat, y::AbstractVecOrMat; weight=nothing) = _crossentropy(ŷ, y, weight)

"""
    logitcrossentropy(ŷ, y; weight=1)

Return the crossentropy computed after a [softmax](@ref) operation:

  -sum(y .* logsoftmax(ŷ) .* weight) / size(y, 2)

See also [`crossentropy`](@ref), [`binarycrossentropy`](@ref).
"""
function logitcrossentropy(ŷ::AbstractVecOrMat, y::AbstractVecOrMat; weight = 1)
  return -sum(y .* logsoftmax(ŷ) .* weight) * 1 // size(y, 2)
end

"""
    binarycrossentropy(ŷ, y; ϵ=eps(ŷ))

Return `-y*log(ŷ + ϵ) - (1-y)*log(1-ŷ + ϵ)`. The ϵ term provides numerical stability.

Typically, the prediction `ŷ` is given by the output of a [`sigmoid`](@ref) activation.
"""
binarycrossentropy(ŷ, y; ϵ=eps(ŷ)) = -y*log(ŷ + ϵ) - (1 - y)*log(1 - ŷ + ϵ)

# Re-definition to fix interaction with CuArrays.
CuArrays.@cufunc binarycrossentropy(ŷ, y; ϵ=eps(ŷ)) = -y*log(ŷ + ϵ) - (1 - y)*log(1 - ŷ + ϵ)

"""
    logitbinarycrossentropy(ŷ, y)

`logitbinarycrossentropy(ŷ, y)` is mathematically equivalent to `binarycrossentropy(σ(ŷ), y)`
but it is more numerically stable.

See also [`binarycrossentropy`](@ref), [`sigmoid`](@ref), [`logsigmoid`](@ref).
"""
logitbinarycrossentropy(ŷ, y) = (1 - y)*ŷ - logσ(ŷ)

# Re-definition to fix interaction with CuArrays.
CuArrays.@cufunc logitbinarycrossentropy(ŷ, y) = (1 - y)*ŷ - logσ(ŷ)

"""
    normalise(x; dims=1)

Normalises `x` to mean 0 and standard deviation 1, across the dimensions given by `dims`. Defaults to normalising over columns.

```julia-repl
julia> a = reshape(collect(1:9), 3, 3)
3×3 Array{Int64,2}:
  1  4  7
  2  5  8
  3  6  9

julia> normalise(a)
3×3 Array{Float64,2}:
  -1.22474  -1.22474  -1.22474
  0.0       0.0       0.0
  1.22474   1.22474   1.22474

julia> normalise(a, dims=2)
3×3 Array{Float64,2}:
  -1.22474  0.0  1.22474
  -1.22474  0.0  1.22474
  -1.22474  0.0  1.22474
```
"""
function normalise(x::AbstractArray; dims=1)
  μ′ = mean(x, dims = dims)
  σ′ = std(x, dims = dims, mean = μ′, corrected=false)
  return (x .- μ′) ./ σ′
end

"""
    kldivergence(ŷ, y)

KLDivergence is a measure of how much one probability distribution is different from the other.
It is always non-negative and zero only when both the distributions are equal everywhere.
[KL Divergence](https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence).
"""
function kldivergence(ŷ, y)
  entropy = sum(y .* log.(y)) *1 //size(y,2)
  cross_entropy = crossentropy(ŷ, y)
  return entropy + cross_entropy
end

"""
    poisson(ŷ, y)

Poisson loss function is a measure of how the predicted distribution diverges from the expected distribution.
[Poisson Loss](https://peltarion.com/knowledge-center/documentation/modeling-view/build-an-ai-model/loss-functions/poisson).
"""
poisson(ŷ, y) = sum(ŷ .- y .* log.(ŷ)) *1 // size(y,2)

"""
    hinge(ŷ, y)

Measures the loss given the prediction `ŷ` and true labels `y` (containing 1 or -1).
[Hinge Loss](https://en.wikipedia.org/wiki/Hinge_loss).
"""
hinge(ŷ, y) = sum(max.(0, 1 .-  ŷ .* y)) *1 // size(y,2)


__precompile__()

module EgyptianFractions

using Base.Iterators: repeated

export efgreedy, efoddgreedy, efharmonic, engelexpand, efengel

function _prep(r::Rational)
  @assert(denominator(r) != 0, "denominator must be > 0")
  rem = abs(big(r))
  ef = BigInt[]
  if rem ≥ 1
    f = floor(rem)
    rem -= f
    append!(ef, collect(repeated(1, Int(f))))
  end
  return (rem, ef)
end

function _greedyloop!(ef::Vector{BigInt}, r::Rational{BigInt})
  c = ceil(1//r)
  push!(ef, c)
  return r - 1//c
end

"""
Performs a greedy (Fibonacci–Sylvester) expansion of the given rational number into a sum of fractions of the form `1/a_1 + 1/a_2 + ...`

This function returns only the denominators of the expansion (i.e., only `[a_1, a_2, ...]`).  If the given rational number `r` satisfies `r ≥ 1`, the first `n` elements of the expansion will be 1, where `n = floor(r)`.  If the given rational number `r` satisfies `r < 0`, all returned denominators will be less than 0, so that the relationship

    r == sum(1 .// efgreedy(r))

is always true.
"""
function efgreedy(r::Rational; nmax::Int = typemax(Int))
  (rem, ef) = _prep(r)
  i = nmax
  while rem != 0 && i > 0
    rem = _greedyloop!(ef, rem)
    i -= 1
  end
  if r < 0
    ef .*= -1
  end
  return ef
end

efgreedy(r::Real; nmax::Int = typemax(Int)) = efgreedy(Rational(big(r)), nmax=nmax)

function _oddgreedyloop!(ef::Vector{BigInt}, r::Rational{BigInt})
  c = ceil(1//r)
  if iseven(numerator(c))
    c += 1
  end
  push!(ef, c)
  return r - 1//c
end

"""
Performs a greedy (Fibonacci–Sylvester) expansion of the given rational number into a sum of fractions of the form `1/a_1 + 1/a_2 + ...`, but only using odd denominators.

It can be shown that, for any rational number `x/y` where `y` is odd, you can write this as a finite sum of fractions where all denominators are odd.  This method sometimes produces an expansion with fewer elements than the traditional greedy algorithm.

This function will throw an exception if the denominator of the rational number supplied is even.

This function returns only the denominators of the expansion (i.e., only `[a_1, a_2, ...]`).  If the given rational number `r` satisfies `r ≥ 1`, the first `n` elements of the expansion will be 1, where `n = floor(r)`.  If the given rational number `r` satisfies `r < 0`, all returned denominators will be less than 0, so that the relationship

    r == sum(1 .// efoddgreedy(r))

is always true.
"""
function efoddgreedy(r::Rational; nmax::Int = typemax(Int))
  @assert isodd(denominator(r)) "denominator of rational ($(denominator(r))) must be odd"
  (rem, ef) = _prep(r)
  i = nmax
  while rem != 0 && i > 0
    rem = _oddgreedyloop!(ef, rem)
    i -= 1
  end
  if r < 0
    ef .*= -1
  end
  return ef
end

efoddgreedy(r::Real; nmax::Int = typemax(Int)) = efoddgreedy(Rational(big(r)), nmax=nmax)

"""
Performs a harmonic expansion of the given rational number into a sum of fractions of the form `1/2 + 1/3 + ...`, and concludes the remainder of the expansion using `efgreedy` if the given rational number cannot be represented as a sum in the harmonic sequence.

If the second argument is specified, the expansion will begin using that value as the first denominator.  In other words, while `efharmonic(r)` will return an array beginning `[2, 3, ...]`, `efharmonic(r, 3)` will return an array beginning `[3, 4, ...]`, and so on.

This function will throw an exception if the second argument `f ≤ 1`.

This function returns only the denominators of the expansion (i.e., only `[a_1, a_2, ...]`).  If the given rational number `r` satisfies `r ≥ 1`, the first `n` elements of the expansion will be 1, where `n = floor(r)`.  If the given rational number `r` satisfies `r < 0`, all returned denominators will be less than 0, so that the relationship

    r == sum(1 .// efharmonic(r))

is always true.
"""
function efharmonic(r::Rational, first::Int = 2; nmax::Int = typemax(Int))
  @assert(first ≥ 2, "harmonic series must start at 2 or greater ($first given)")
  (rem, ef) = _prep(r)
  s = Rational{BigInt}(0)
  i = big(first)
  j = nmax
  while s ≤ rem && j > 0
    hh = 1//i
    if s + hh > rem
      break
    end
    push!(ef, i)
    s += hh
    i += 1
    j -= 1
  end
  rem -= s
  while rem != 0 && j > 0
    rem = _greedyloop!(ef, rem)
    j -= 1
  end
  if r < 0
    ef .*= -1
  end
  return ef
end

efharmonic(r::Real, first::Int = 2; nmax::Int = typemax(Int)) = efharmonic(Rational(big(r)), first, nmax=nmax)

function _engelloop!(ef::Vector{BigInt}, r::Rational{BigInt})
  c = ceil(1//r)
  push!(ef, c)
  return r * c - 1
end

"""
Performs an Engel expansion of the given rational number into a sum of fractions of the form `1/a + 1/(a*b) + 1/(a*b*c)...`.

This function returns only the unique denominators of the expansion (i.e., only `[a, b, ...]`).  If the given rational number `r` satisfies `r ≥ 1`, the first `n` elements of the expansion will be 1, where `n = floor(r)`.
"""
function engelexpand(r::Rational; nmax::Int = typemax(Int))
  (rem, ef) = _prep(r)
  i = nmax
  while rem != 0 && i > 0
    rem = _engelloop!(ef, rem)
    i -= 1
  end
  # Convention:  the Engle expansion of a negative number will have ef[1] < 0.
  # This way, all elements of cumprod(ef) will be negative.
  if !isempty(ef) && r < 0
    ef[1] *= -1
  end
  return ef
end

engelexpand(r::Real; nmax::Int = typemax(Int)) = engelexpand(Rational(big(r)), nmax=nmax)

"""
Performs an Engel expansion of the given rational number into a sum of fractions of the form `1/a + 1/(a*b) + 1/(a*b*c)...`.

This function returns only the denominators of the expansion (i.e., only `[a, a*b, ...]`).  If the given rational number `r` satisfies `r ≥ 1`, the first `n` elements of the expansion will be 1, where `n = floor(r)`.  If the given rational number `r` satisfies `r < 0`, all returned denominators will be less than 0, so that the relationship

    r == sum(1 .// efengel(r))

is always true.
"""
function efengel(r::Rational; nmax::Int = typemax(Int))
  ef = engelexpand(r, nmax=nmax)
  cumprod!(ef, ef)
  return ef
end

efengel(r::Real; nmax::Int = typemax(Int)) = efengel(Rational(big(r)), nmax=nmax)

end # module


function qrsolve!(A::AbstractMatrix{T},b::AbstractVector{T}) where {T}
    # Fast least-squares solver for full column rank Hessenberg-like matrices
    m, n = size(A)
    m < n && error("Column dimension exceeds row dimension")
    _, τ = LinearAlgebra.LAPACK.geqrf!(A)
    T <: Complex ? tran = 'C' : tran = 'T'
    LinearAlgebra.LAPACK.ormqr!('L',tran,A,τ,view(b,:,1:1))
    return UpperTriangular(triu(A[1:n,:]))\b[1:n]
end

function poldeg1(v::Vector{T}) where {T <: Number}
    # Degree of a polynomial plus one.
    n1 = findlast(!iszero,v) # degree(v) + 1
    n1 === nothing && (n1 = 0)
    return n1
end
poldeg(v::Vector{T}) where {T <: Number} = poldeg1(v)-1  # Degree of a polynomial
function poldiv(a::Vector{T}, b::Vector{S}) where {T <: Number, S <: Number }
    # Quotient of the exact division of two polynomials a and b.
    # q = poldiv(a,b) returns the quotient q = a/b, which is computed by solving the
    # linear least-square problem ||C(b)*q - a|| = min, where C(b) is the convolution
    # matrix of appropriate size.

    na1 = poldeg1(a) # degree(a) + 1
    nb1 = poldeg1(b) # degree(b) + 1
    nb1 == 0 && throw(DivideError())
    na1 < nb1 && error("Degree of second polynomial exceeds the degree of first polynomial")
    a1, b1  = promote(float(a[1:na1]), float(b[1:nb1]))
    #return Matrix(Toeplitz([den;zeros(n1-length(den))],[den[1];zeros(n1-m1)]))\num[1:n1]
    #return convmtx(b[1:nb1],na1-nb1+1)\a[1:na1]
    return qrsolve!(convmtx(b1,na1-nb1+1),a1)
end
function convmtx(v::Vector{T}, n::Int) where {T <: Number}

    #   Convolution matrix.
    #   C = convmtx(v,n) returns the convolution matrix C for a vector v.
    #   If q is a column vector of length n, then C*q is the same as conv(v,q).

    #   Form C as the Toeplitz matrix
    #   C = Toeplitz([v; zeros(n-1)],[v[1]; zeros(n-1));  put Toeplitz code inline

    nv = length(v)-1
    C = zeros(T, n+nv, n)
    @inbounds for j = 1:n
        C[j:j+nv,j] = v
    end
    return C
end
function poldivrem(a::Vector{T}, b::Vector{S}) where {T <: Number, S <: Number }
    # Quotient and remainder of the division of two polynomials a and b.
    # q, r = poldivrem(a,b) returns the quotient q and remainder r such that a = q*b + r.
    na1 = poldeg1(a) # degree(a) + 1
    nb1 = poldeg1(b) # degree(b) + 1
    nb1 == 0 && throw(DivideError())

    R = eltype(one(T)/one(S))

    if na1 < nb1
       return zeros(R, 1), a
    end
    m = nb1-1
    q = zeros(R, na1-m)
    r = R[ a[i] for i in 1:na1 ]
    #r = copy_oftype(num[1:n1],R)

    @inbounds for i in na1:-1:nb1
        s = r[i] / b[nb1]
        q[i-m] = s
        @inbounds for j in 1:m
            r[i-nb1+j] -= b[j] * s
        end
    end

    return q, m == 0 ? zeros(R,1) : r[1:m]
end
function polgcdvw(a::Vector{T}, b::Vector{S}; atol::Real = 0, rtol::Real = Base.rtoldefault(float(real(T))), maxnit::Int = 0) where {T <: Number, S <: Number }
    # Greatest common divisor of two polynomials.
    #     polgcdvw(a, b; atol = 0, rtol, maxnit = 0) -> (d, v, w, δ)
    # Compute the greatest common divisor `d` of two polynomials `a` and `b`, and
    # the polynomials v and w such that a = d * v * ka and b = d * w *kb, where ka and kb
    # are scalar scaling factors (ka and kb are not explicitly provided).
    # The accuracy estimate δ is computed as δ = norm([a/ka-d*v; b/kb-d*w]).
    # A SVD-based method adapted from [1] is employed and an iterative accuracy refinement
    # of δ, with a maximum number of maxnit iterations, is additionally performed.
    #  [1] Z. Zeng, The numerical greatest common divisor of univariate polynomials,
    #      in: L. Gurvits, P. Pébay, J.M. Rojas, D. Thompson (Eds.),
    #      Randomization, Relaxation,and Complexity in Polynomial Equation Solving,
    #      Contemporary Mathematics,vol.556,AMS,2011,pp.187–217.

    na1 = poldeg1(a) # degree(a) + 1
    nb1 = poldeg1(b) # degree(b) + 1

    R = eltype(one(T)/one(S))

    a1, b1  = promote(float(a[1:na1]), float(b[1:nb1]))

    na1 <= 0 && (return ones(R,1), zeros(R,1), b1, zero(R))
    nb1 <= 0 && (return ones(R,1), a1, zeros(R,1), zero(R))
    a1 = a1/norm(a1)
    b1 = b1/norm(b1)
    switch = (na1 < nb1)
    switch && ((a1, b1, na1, nb1) = (b1, a1, nb1, na1))

    atol == 0 ? tol = rtol : tol = atol
    # determine the degree of GCD as the nullity of the Sylvester matrix
    nd = na1 + nb1 - 2 - rank([convmtx(a1,nb1-1) convmtx(b1,na1-1)], atol = tol)
    nd == 0 && (switch ? (return [one(R)], b1, a1, zero(R)) : (return [one(R)], a1, b1, zero(R)))
    # determine [w; -v] from an orthogonal/unitary nullspace basis of dimension 1
    # of a reduced Sylvester matrix using the last row of Vt from its (partial) SVD
    _, sv, Vt = LAPACK.gesvd!('N', 'A', [convmtx(a1,nb1-nd) convmtx(b1,na1-nd)])
    k = na1 + nb1 -2*nd - count(x -> x > tol, sv) # expected nullity k = 1
    k == 1 || error("GCD computation failure")
    wv = Vt[end,:]
    # wv = LAPACK.gesvd!('N', 'A', [convmtx(a1,nb1-nd) convmtx(b1,na1-nd)])[3][end,:]
    eltype(wv) <: Complex && (wv = conj(wv))
    v = -wv[nb1-nd+1:end]
    w = wv[1:nb1-nd]
    # determine the GCD d with high accuracy as the solution of a well-conditioned
    # linear least-squares problem
    # d = convmtx(w,nd+1)\b1
    # d = [convmtx(v,nd+1) ; convmtx(w,nd+1)] \ [a1; b1]
    d = qrsolve!([convmtx(v,nd+1) ; convmtx(w,nd+1)],[a1; b1])
    if maxnit > 0
       d, v, w, δ = gcdvwupd(a1, b1, d, v, w, maxnit = maxnit)
    else
       δ = norm( [a1; b1] - [ conv(v,d); conv(w,d) ])
    end

    switch ? (return d, w, v, δ) : (return d, v, w, δ)
end
function gcdvwupd(a::Vector{T}, b::Vector{T}, d::Vector{T}, v::Vector{T}, w::Vector{T}; maxnit::Int = 10) where {T <: Number}
    # Iterative refinement of the accuracy of the greatest common divisor of two polynomials.
    #     gcdvwupd(a, b, d, v, w; maxnit = 0) -> (dupd, vupd, wupd, δ)
    # Given the polynomials a and b, and a triple of polynomials (d, v, w) such that
    # `d` is an approximation of a greatest common divisor of `a` and `b`, and
    # v and w are approximate quotients of a/d and b/d, respectively, compute the updated triple
    # (dupd, vupd, wupd) which minimizes the accuracy estimate δ = norm([a-dupd*vupd; b-dupd*wupd]).
    # A maximum number of maxnit iterations are performed using the Gauss-Newton iteration method,
    # in the form proposed in [1].
    #
    #  [1] Z. Zeng, The numerical greatest common divisor of univariate polynomials,
    #      in: L. Gurvits, P. Pébay, J.M. Rojas, D. Thompson (Eds.),
    #      Randomization, Relaxation,and Complexity in Polynomial Equation Solving,
    #      Contemporary Mathematics,vol.556,AMS,2011,pp.187–217.

    na1 = length(a)
    nb1 = length(b)
    nd1 = length(d)
    nv1 = length(v)
    nw1 = length(w)
    na1 == nd1+nv1-1 || error("Incompatible dimensions between a, d and v")
    nb1 == nd1+nw1-1 || error("Incompatible dimensions between b, d and w")

    h = d/norm(d)^2
    f = [a;b;fill(one(T),1)]
    fh = [conv(v,d); conv(w,d); fill(dot(h,d),1)]
    Δf = f-fh;
    δ = norm(Δf)
    d0 = copy(d); v0 = copy(v); w0 = copy(w);
    nvw1 = nv1+nw1
    ndvw1 = nd1+nvw1
    nab1 = na1+nb1
    Jh = [zeros(nab1,ndvw1); zeros(1,nvw1) h' ]

    j1 = 1:nv1
    j2 = nv1+1:nvw1
    j3 = nvw1+1:ndvw1
    i1 = 1:nv1+nd1-1
    i2 = nv1+nd1:nvw1+2*nd1-2
    Jh11 = view(Jh,i1,j1)
    Jh22 = view(Jh,i2,j2)
    Jh13 = view(Jh,i1,j3)
    Jh23 = view(Jh,i2,j3)

    for it = 1:maxnit
        # Jh = [ convmtx(d0,nv1)          0     convmtx(v0,nd1) ]
        #      [  0             convmtx(u0,nw1) convmtx(w0,nd1) ]
        #      [  0                       0            h'       ]
        Jh11[:,:] = convmtx(d0,nv1)
        Jh22[:,:] = convmtx(d0,nw1)
        Jh13[:,:] = convmtx(v0,nd1)
        Jh23[:,:] = convmtx(w0,nd1)
        # Δz = Jh\Δf;   # this does not exploit the zero structure
        Δz = qrsolve!(copy(Jh),copy(Δf));  # this is probably the fastest
        # Δz = qr(Jh)\Δf;  # this is usually slower, but faster for full matrices
        v1 = v0 + Δz[j1];
        w1 = w0 + Δz[j2];
        d1 = d0 + Δz[j3];
        fh1 = [conv(v1,d1); conv(w1,d1); fill(dot(h,d1),1)];
        Δf1 = f-fh1;
        δ1 = norm(Δf1)
        if δ1 < δ
           d0 = copy(d1); v0 = copy(v1); w0 = copy(w1); δ = copy(δ1); Δf = copy(Δf1);
        else
           break
        end
    end
    return d0, v0, w0, δ
end
function pollcm(a::Vector{T}, b::Vector{S}; atol::Real = 0, rtol::Real = Base.rtoldefault(float(real(T))), maxnit::Int = 10) where {T <: Number, S <: Number }
    # Least common multiple of two polynomials.
    #     m = pollcm(a,b; atol = 0, rtol, maxnit = 0)
    # Compute the least common multiple `m` of two polynomials `a` and `b`. Employ iterative refinement of
    # the accuracy of GCD(a,b) if maxnit > 0.

    na1 = poldeg1(a) # degree(a) + 1
    nb1 = poldeg1(b) # degree(b) + 1

    na1 == 0 && (return zeros(T,1))
    nb1 == 0 && (return zeros(S,1))
    na1 == 1 && (return b)
    nb1 == 1 && (return a)

    return conv(a, polgcdvw(a, b, atol = atol, rtol = rtol, maxnit = maxnit)[3])
end
function conv(a::Vector{T}, b::Vector{S}) where {T <: Number, S <: Number }
    # Convolution of two vectors (or product of two polynomials).
    # c = conv(a,b) returns the convolution of vectors a and b.

    na1 = length(a)
    nb1 = length(b)

    R = promote_type(T, S)
    c = zeros(R, na1 + nb1 - 1)
    for i in 1:na1, j in 1:nb1
        @inbounds c[i + j - 1] += a[i] * b[j]
    end
    return c
end
function polcoeffval(r::AbstractVector{T},val::Number) where {T}
    # Compute the coefficients of a polynomial from its roots and evaluate the polynomial
    # for a given value of its argument. Both are equal to one for an empty vector r.
    T1 = promote_type(T,eltype(val))
    n = length(r)
    c = zeros(T1, n+1)
    ONE = one(T1)
    c[1] = ONE
    pval = ONE
    for j = 1:n
        pval = pval*(val-r[j])
        for i = j:-1:1
            c[i+1] = c[i+1]-r[j]*c[i]
        end
    end
    return reverse(c), pval
 end


export sinc_interpolate, sinc_interpolate_sum, downsample
export resample

"""
    resample(arr, new_size [, normalize]; take_real=true, boundary_handling=true)

Calculates the `sinc` interpolation of an `arr` on a new array size
`new_size`.
It is a re-evaluation of the Fourier series at new grid points.
`new_size` can be arbitrary. Each dimension is then independently either up or downsampled.

This method is based on FFTs and therefore implicitly assumes periodic
boundaries and a finite frequency support.
`normalize=true` by default multiplies by an appropriate factor so that
the average intensity stays the same.
If `size(new_size)[i] > size(arr)[i]`, we apply zero padding in Fourier space.
If `size(new_size)[i] < size(arr)[i]`, we cut out a centered part of the
Fourier spectrum.

We apply some tricks at the boundary to increase accuracy of highest frequencies.
If you set the keyword argument `boundary_handling=false` you turn this boundary handling off.
This can increase inaccuracies for the highest frequency (not a big problem in real images/signals).
However, `resample` is then faster on CPUs and especially on CUDA GPUs.

# Examples
```jldoctest
julia> resample([1.0, 2.0, 3.0, 4.0], 8)
8-element Array{Float64,1}:
 1.0
 1.085786437626905
 2.0
 2.5
 3.0
 3.914213562373095
 4.0
 2.5

julia> resample([1.0  2.0; 3.0 4.0], (4,4))
4×4 Array{Float64,2}:
 1.0  1.5  2.0  1.5
 2.0  2.5  3.0  2.5
 3.0  3.5  4.0  3.5
 2.0  2.5  3.0  2.5

julia> resample([1.0, 0.0, 1.0, 0.0, 1.0, 0.0], (3))
3-element Array{Float64,1}:
 0.5
 0.5
 0.5

julia> resample([1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0], (6))
6-element Array{Float64,1}:
  1.0
 -0.3333333333333333
  1.0
 -0.3333333333333333
  1.0
 -0.3333333333333333

julia> resample([1 2 3; 4 5 6], (3, 2))
3×2 Array{Float64,2}:
 1.0   3.0
 3.25  5.25
 3.25  5.25
```
"""
function resample(arr::AbstractArray{T, N}, new_size, normalize=true; take_real=true, boundary_handling=true) where {T<:Real, N}

    if typeof(new_size) <: Number
        new_size = Tuple(new_size)
    end

    # go to fourier space
    arr_f = fftshift(fft(arr))
    # create fourier space new array

    # the idea is the following
    # 1) first we handle all the upsampling dimensions. By doing so, we leave the size
    # or increase it. downsampling is done afterwards. Hence we use max to create
    # a new array which is a least as large as the initial one or larger
    # 2) we then handle all the downsampling dimensions
    new_size_interp = Tuple(max(x[1], x[2]) for x in zip(size(arr), new_size))
    out_f = similar(arr_f, new_size_interp)
    fill!(out_f, zero(eltype(arr_f)))

    # change arr_f to be a hermitian array because we want a purely real result
    # after iffting

    # in that case, make_hermitian is useless since we only cut out
    if boundary_handling && ~(all(new_size < size(arr)))
        arr_f = make_hermitian(arr_f)
    end

    # it can happen, that arr_f was now padded with an extra row
    # but there is no extra column in out_f
    # therefore, cut it
    inds = []
    for (a, o) in zip(size(arr_f), size(out_f))
        push!(inds, 1:min(a, o))
    end
    arr_f = arr_f[inds...]

    # set the old array into the new 0-padded array
    center_set!(out_f, arr_f)
    arr_f = out_f

    # part 2). We handle now the downsampling
    # if the new_size[d] is even, we need to add the highest positive frequency
    # of the initial spectrum
    # to the highest negative one. In that way, we get a purely real result
    if boundary_handling
        arr_f = add_high_frequencies(size(arr_f), arr_f, new_size, N)
    end
    # return arr_out_f

    # do the cutting in Fourier space
    arr_f_n = center_extract(arr_f, new_size)
    # back to real space
    arr_out = ifft(ifftshift(arr_f_n))

    if normalize
        arr_out .*= length(arr_out) ./ length(arr)
    end
    if take_real
        return real(arr_out)
    else
        return arr_out
    end
end

 # for a complex signal: split into real and imaginary part and solve for each
 # part individually
function resample(arr::AbstractArray{T}, new_size, normalize=true) where T<:Complex
    # array of same shape but with Complex element type
    arr_r = resample(real(arr), new_size, normalize)
    arr_i = 1im .* resample(imag(arr), new_size, normalize)
    return arr_r .+ arr_i
end

"""
    sinc_interpolate_sum(arr, new_length)

Calculates the `sinc` interpolation of an 1D `arr` on a new array size
`new_size`.
This method is slow, because of an explicit sum evalulation and
not a FFT based evaluation.

# Examples
```jldoctest
julia> sinc_interpolate_sum([1.0, 2.0, 3.0, 4.0], 8)
8-element Array{Float64,1}:
 1.0
 1.7825353626292277
 2.0
 2.1220659078919377
 3.0
 4.1592491794681985
 4.0
 2.0735615442829793
```
"""
function sinc_interpolate_sum(arr, new_size)
    out = zeros(eltype(arr), new_size)
    T = 1 / size(arr)[1]
    t_arr = range(0, 1.0, length=size(out)[1]+1)[1:size(out)[1]]
    for (j, t) in enumerate(t_arr)
        v = zero(eltype(arr))
        for n = 1:size(arr)[1]
            v += arr[n] * sinc((t - (n-1) * T) / T)
        end
        out[j] = v
    end
    return out
end


using Ipopt

# hs071
# min x1 * x4 * (x1 + x2 + x3) + x3
# st  x1 * x2 * x3 * x4 >= 25
#     x1^2 + x2^2 + x3^2 + x4^2 = 40
#     1 <= x1, x2, x3, x4 <= 5
# Start at (1,5,5,1)
# End at (1.000..., 4.743..., 3.821..., 1.379...)

function eval_f(x::Vector{Float64})
  return x[1] * x[4] * (x[1] + x[2] + x[3]) + x[3]
end

function eval_g(x::Vector{Float64}, g::Vector{Float64})
  # Bad: g    = zeros(2)  # Allocates new array
  # OK:  g[:] = zeros(2)  # Modifies 'in place'
  g[1] = x[1]   * x[2]   * x[3]   * x[4]
  g[2] = x[1]^2 + x[2]^2 + x[3]^2 + x[4]^2
end

function eval_grad_f(x::Vector{Float64}, grad_f::Vector{Float64})
  # Bad: grad_f    = zeros(4)  # Allocates new array
  # OK:  grad_f[:] = zeros(4)  # Modifies 'in place'
  grad_f[1] = x[1] * x[4] + x[4] * (x[1] + x[2] + x[3])
  grad_f[2] = x[1] * x[4]
  grad_f[3] = x[1] * x[4] + 1
  grad_f[4] = x[1] * (x[1] + x[2] + x[3])
end

function eval_jac_g(x::Vector{Float64}, mode, rows::Vector{Int32}, cols::Vector{Int32}, values::Vector{Float64})
  if mode == :Structure
    # Constraint (row) 1
    rows[1] = 1; cols[1] = 1
    rows[2] = 1; cols[2] = 2
    rows[3] = 1; cols[3] = 3
    rows[4] = 1; cols[4] = 4
    # Constraint (row) 2
    rows[5] = 2; cols[5] = 1
    rows[6] = 2; cols[6] = 2
    rows[7] = 2; cols[7] = 3
    rows[8] = 2; cols[8] = 4
  else
    # Constraint (row) 1
    values[1] = x[2]*x[3]*x[4]  # 1,1
    values[2] = x[1]*x[3]*x[4]  # 1,2
    values[3] = x[1]*x[2]*x[4]  # 1,3
    values[4] = x[1]*x[2]*x[3]  # 1,4
    # Constraint (row) 2
    values[5] = 2*x[1]  # 2,1
    values[6] = 2*x[2]  # 2,2
    values[7] = 2*x[3]  # 2,3
    values[8] = 2*x[4]  # 2,4
  end
end

function eval_h(x::Vector{Float64}, mode, rows::Vector{Int32}, cols::Vector{Int32}, obj_factor::Float64, lambda::Vector{Float64}, values::Vector{Float64})
  if mode == :Structure
    # Symmetric matrix, fill the lower left triangle only
    idx = 1
    for row = 1:4
      for col = 1:row
        rows[idx] = row
        cols[idx] = col
        idx += 1
      end
    end
  else
    # Again, only lower left triangle
    # Objective
    values[1] = obj_factor * (2*x[4])  # 1,1
    values[2] = obj_factor * (  x[4])  # 2,1
    values[3] = 0                      # 2,2
    values[4] = obj_factor * (  x[4])  # 3,1
    values[5] = 0                      # 3,2
    values[6] = 0                      # 3,3
    values[7] = obj_factor * (2*x[1] + x[2] + x[3])  # 4,1
    values[8] = obj_factor * (  x[1])  # 4,2
    values[9] = obj_factor * (  x[1])  # 4,3
    values[10] = 0                     # 4,4

    # First constraint
    values[2] += lambda[1] * (x[3] * x[4])  # 2,1
    values[4] += lambda[1] * (x[2] * x[4])  # 3,1
    values[5] += lambda[1] * (x[1] * x[4])  # 3,2
    values[7] += lambda[1] * (x[2] * x[3])  # 4,1
    values[8] += lambda[1] * (x[1] * x[3])  # 4,2
    values[9] += lambda[1] * (x[1] * x[2])  # 4,3

    # Second constraint
    values[1]  += lambda[2] * 2  # 1,1
    values[3]  += lambda[2] * 2  # 2,2
    values[6]  += lambda[2] * 2  # 3,3
    values[10] += lambda[2] * 2  # 4,4
  end
end

function intermediate(alg_mod::Int, iter_count::Int,
  obj_value::Float64, inf_pr::Float64, inf_du::Float64, mu::Float64,
  d_norm::Float64, regularization_size::Float64, alpha_du::Float64, alpha_pr::Float64,
  ls_trials::Int)
  println("Iteration $iter_count, objective value is $obj_value.")
  return true
end

n = 4
x_L = [1.0, 1.0, 1.0, 1.0]
x_U = [5.0, 5.0, 5.0, 5.0]

m = 2
g_L = [25.0, 40.0]
g_U = [2.0e19, 40.0]

prob = createProblem(n, x_L, x_U, m, g_L, g_U, 8, 10,
                     eval_f, eval_g, eval_grad_f, eval_jac_g, eval_h)

prob.x = [1.0, 5.0, 5.0, 1.0]
stat = solveProblem(prob)

@test Ipopt.ApplicationReturnStatus[stat] == :Solve_Succeeded
@test_approx_eq_eps prob.x[1] 1.0000000000000000 1e-5
@test_approx_eq_eps prob.x[2] 4.7429996418092970 1e-5
@test_approx_eq_eps prob.x[3] 3.8211499817883077 1e-5
@test_approx_eq_eps prob.x[4] 1.3794082897556983 1e-5
@test_approx_eq_eps prob.obj_val 17.014017145179164 1e-5


"""
solve the 1D linear advection equation
```math
∂q/∂t=−u∂q/∂x
```
in a periodic domain, where ``q`` is the quantity being advected,
``t`` is time, ``x`` is the spatial coordinate and ``u`` is the velocity,
which is constant with ``x``.
"""
function lax_wendroff!(nt::Int, c, q_init::AbstractVector{T}, q::AbstractVector{T}) where T
    nx = length(q)
    flux = zeros(T, nx-1)   # Fluxes between boxes
    @inbounds for i=1:nx
        q[i] = q_init[i] # Initialize q
    end
    @inbounds for j=1:nt  # Main loop in time
        for i=1:nx-1
            flux[i] = 0.5*c*(q[i]+q[i+1]+c*(q[i]-q[i+1]))
        end
        for i=2:nx-1
            q[i] += flux[i-1]-flux[i]
        end
        q[1] = q[nx-1]; q[nx] = q[2] # Treat boundary conditions
    end
    return q
end

using Random
Random.seed!(2)
q_init = randn(100)
q = zeros(100)
@show lax_wendroff!(2000, 1.0, q_init, zero(q_init))
using BenchmarkTools
@benchmark lax_wendroff!(2000, 1.0, $q_init, x) setup=(x=zero(q_init))
@time lax_wendroff!(2000, 1.0, q_init, q)

using NiLang
@i function i_lax_wendroff!(nt::Int, c, q_init::AbstractVector{T}, q::AbstractVector{T},
        cache::AbstractMatrix{T}) where T
    nx ← length(q)
    @inbounds for i=1:nx
        q[i] += q_init[i] # Initialize q
    end
    @inbounds for j=1:nt  # Main loop in time
        for i=1:nx-1
            @routine begin
                @zeros T anc1 anc2 anc3
                anc1 += 0.5 * c
                anc2 += q[i] - q[i+1]
                anc3 += q[i] + q[i+1]
                anc3 += c * anc2
            end
            cache[i,j] += anc1 * anc3
            ~@routine
        end
        for i=2:nx-1
            q[i] += cache[i-1,j]-cache[i,j]
        end
        # Treat boundary conditions
        cache[nx,j] += q[nx-1]
        SWAP(q[1], cache[nx,j])
        cache[nx+1,j] += q[2]
        SWAP(q[nx], cache[nx+1,j])
    end
end
nt = 2000
i_lax_wendroff!(nt, 1.0, q_init, zero(q_init), zeros(length(q_init)+1,nt))


function fit(m::MultinomialNB, X::Matrix{Int64}, y::Vector{C}) where C
    ensure_data_size(X, y)
    for j=1:size(X, 2)
        c = y[j]
        m.c_counts[c] += 1
        m.x_counts[c] .+= X[:, j]
        m.x_totals += X[:, j]
        m.n_obs += 1
    end
    return m
end

"""Calculate log P(x|C)"""
function logprob_x_given_c(m::MultinomialNB, x::Vector{Int64}, c::C) where C
    x_priors_for_c = m.x_counts[c] ./ sum(m.x_counts[c])
    x_probs_given_c = x_priors_for_c .^ x
    logprob = sum(log(x_probs_given_c))
    return logprob
end

"""Calculate log P(x|C)"""
function logprob_x_given_c(m::MultinomialNB, X::Matrix{Int64}, c::C) where C
    x_priors_for_c = m.x_counts[c] ./ sum(m.x_counts[c])
    x_probs_given_c = x_priors_for_c .^ X
    logprob = sum(log.(x_probs_given_c), dims=1)
    return dropdims(logprob, dims=1)
end


export randomM

"""
function randomM(m,n;p=2,N=1)

Generate samples for M from uniform distribution with non-repeated columns.

Input:
    m - number of blocks
    n - number of columns
    p - number of categories in each block (default=2)
    N - number of samples (default=1)

Output:
    M - BitArray of size (m*(p-1),n,N), or (m*(p-1),n) if N==1
"""
function randomM(m::Int64,n::Int64;p::Int64=2,N::Int64=1)
    if log(n) > m*log(p)
        error("unique columns in M not possible if n > p^m")
    end

    # store all feasible blocks of M in a matrix B of size ((p-1)*p^n, n)
    B = getB(n,p)

    M = falses(m*(p-1),n,N)
    for idx = 1:N
        # rejection sampling: check uniqueness of columns
        firstFlag = true # (this should be a do-while loop)
        while firstFlag || size(unique(M[:,:,idx],2),2) < n
            firstFlag = false
            # randomly choose m blocks from B
            blockIdB = rand(1:p^n, m)
            for blockIdM = 1:m
                subsB = (blockIdB[blockIdM]-1)*(p-1)+1:blockIdB[blockIdM]*(p-1)
                subsM = (blockIdM-1)*(p-1)+1:blockIdM*(p-1)
                M[subsM,:,idx] = B[subsB,:]
            end
        end
    end

    # output
    if N==1
        M = squeeze(M,3)
    end
    return M
end


using DifferentialEquations
using Plots

# BM model for beta(t): d beta(t) = beta(t) d W(t)
# Function to compute rates of change
spsir_bm = function(du, u, p, t)
    S = u[1]
    I = u[2]
    R = u[3]
    beta = max(0., u[4])

    N = S + I + R
    gamma = p[1]

    du[1] = -beta * S * I / N
    du[2] = beta * S * I / N - gamma * I
    du[3] = gamma * I
    du[4] = 0.
end

# Function to add noise
sigma_spsir = function(du, u, p, t )
    sigma = p[2]

    du[1] = 0.
    du[2] = 0.
    du[3] = 0.
    du[4] = sigma
end

# BM for logbeta(t), with drift
spsir_logbm_drift = function(du, u, p, t)
    S = u[1]
    I = u[2]
    R = u[3]
    beta = exp( u[4] )

    N = S + I + R
    gamma = p[1]
    alpha = p[3]

    du[1] = -beta * S * I / N
    du[2] = beta * S * I / N - gamma * I
    du[3] = gamma * I
    du[4] = -alpha * I
end

# set random seed
srand( 1111 )

## Simulation of BM model
# starting conditions
u0 = [50.;1.0;0.0;2.0]
tspan = (0.0,10.0)
# parameters gamma sigma
p = [1.; 1.]
spsir_bm_prob = SDEProblem(spsir_bm, sigma_spsir, u0, tspan, p)
spsir_bm_sol = solve(spsir_bm_prob)

## Simulation of log-BM with drift model

# starting condtions
u0 = [50.;1.0;0.0;log(3.) ]
tspan = (0.0,10.0)
# parameters gamma sigma alpha
p = [1.; 1.; 0.1]
spsir_logbm_drift_prob = SDEProblem(spsir_logbm_drift, sigma_spsir, u0, tspan, p)
spsir_logbm_drift_sol = solve(spsir_logbm_drift_prob)

## Plotting for BM model
# Plot evolution of number infected
plot( spsir_bm_sol , vars = 2 )

# Plot evolution of transmission rate
plot( spsir_bm_sol , vars = 4 )

## Plotting for BM with drift model
# Plot evolution of number infected
plot( spsir_logbm_drift_sol , vars = 2 )

# Plot evolution of transmission rate
plot( spsir_logbm_drift_sol , vars = 4 )


"""
    euler_method(f, x0, span, h=1.0e-2)

Calculate the solution to a differential equation using forward euler method.
"""
function euler_method(f, x0, span, h=1.0e-2)
    s, e = span
    steps = floor((e - s) / h) |> Int
    x = Vector{Float64}(undef, steps)
	t = Vector{Float64}(undef, steps)
    x[1] = x0
	t[1] = 0
    for i in 1:(steps - 1)
        x[i + 1] = x[i] + h * f(x[i], t[i])
		t[i + 1] = t[i] + h
    end
    return x, t
end


include("utils.jl")

"""
Computes the jacobian of a layer `l` with respect to `wrt`, evaluated at the
point `x`, using 2nd order finite differences of precision `h`. The resulting
matrix contains the entries of `wrt` along the column and the entries of `l`'s
output along the rows, e.g.:

    J_{i,j} = ∂_i f_j (x)

The flattening of `wrt` and `l`'s output happens in column-major order.
Example:

    x = FloatX[0 1 2]
    l = xavier!(Linear{FloatX}(1,1))
    Jw = jacobian_fwd(l, x, l.w, 1e-5)
    Jx = jacobian_fwd(l, x, x, 1e-5)
"""
function jacobian_fwd(l, x, wrt, h=0)
    out = fprop!(l, x)
    jacobian = Array(eltype(out), (length(out), length(wrt)))

    # Slightly perturb ("wiggle") each parameter individually.
    for (i,idx) in enumerate(eachindex(wrt))
        orig = wrt[idx]

        # See https://codewords.recurse.com/issues/four/hack-the-derivative
        # and the link to lecture notes therein.
        eff_h = h == 0 ? √eps(typeof(orig)) * max(abs(orig), one(orig)) : h

        wrt[idx] = orig + eff_h
        right = vec(fprop!(l, x))

        wrt[idx] = orig - eff_h
        left = vec(fprop!(l, x))

        wrt[idx] = orig
        jacobian[:,i] = (right - left)/2eff_h
    end

    jacobian
end

jacobian_fwd(l, x) = jacobian_fwd(l, x, x)

"""
Uses backprop to compute the jacobian of a layer `l` with respect to `wrt`,
evaluated at the point `x`. The resulting matrix contains the entries of `wrt`
along the column and the entries of `l`'s output along the rows, e.g.:

    J_{i,j} = ∂_i f_j (x)

The flattening of `wrt` and `l`'s output happens in column-major order.
The last parameter, `dwrt` is the buffer of the layer containing the
derivative. This should be automated somehow in the future.

Example:

    x = FloatX[0 1 2]
    l = xavier!(Linear{FloatX}(1,1))
    Jw = jacobian_backward(l, x, l.w, l.dw)
    Jx = jacobian_backward(l, x, x, nothing)
"""
function jacobian_bwd(l, x, wrt, dwrt)
    out = fprop!(l, x)
    jacobian = Array(eltype(out), (length(out), length(wrt)))

    # Set each output to 1, individually, and backprop from it.
    for (i,idx) in enumerate(eachindex(out))
        dout = zeros(out)
        dout[idx] = 1

        zero_grads!(l)

        din = bprop!(l, dout)
        if is(wrt, x)
            jacobian[i,:] = vec(din)
        else
            jacobian[i,:] = vec(dwrt)
        end
    end

    jacobian
end

jacobian_bwd(l, x) = jacobian_bwd(l, x, x, nothing)


"Computes Gregory-type quadrature rules."
module Gregory

export integrate

"""
    B = calcBernoulli(n)

Return the `n`th Bernoulli number.  Follows the convention that `n=1`
Bernoulli number is equal to -1/2.
"""
function calcBernoulli(n)
    A = Vector{Rational{BigInt}}(undef, n + 1)
    for m = 0:n
        A[m + 1] = 1 // (m + 1)
        for j = m:-1:1
            A[j] = j * (A[j] - A[j + 1])
        end
    end
    if n == 1
        return -A[1]
    else
        return A[1]
    end
end

"""
    w = calcBoundaryWeights(numbnd, order)

Compute the boundary weights for a Gregory rule of order `order` that using
`numbnd` boundary points.
"""
function calcBoundaryWeights(numbnd::Int, order::Int)
    @assert( numbnd >= order-1,
             "number of boundary nodes must be larger than order-1")
    # Form the linear system to solve
    A = zeros(Rational{BigInt}, order-1, numbnd)
    b = zeros(Rational{BigInt}, order-1)
    for j = 1:order-1
        for i = 1:numbnd
            A[j,i] = j*(numbnd - (i-1))^(j-1)
        end
        b[j] = numbnd^j - ((-1)^j)*calcBernoulli(j)
    end
    w = A\b
    return w
end

"""
    val = integrate(data, numbnd, order[, h=1.0])

Integrate uniformly spaced `data` using a Gregory quadrature rule of order
`order` with `numbnd` boundary points.  The mesh spacing is defined by `h`.
"""
function integrate(data::Array{T,1}, numbnd::Int, order::Int;
                   h::T=one(T)) where {T<:Number}
    @assert( size(data,1) >= 2*numbnd,
             "number of data points must be larger than 2*numbnd")
    w = calcBoundaryWeights(numbnd, order)
    integral = zero(T)
    for i = 1:numbnd
        integral += w[i]*(data[i] + data[end-i+1])
    end
    for i = numbnd+1:size(data,1)-numbnd
        integral += data[i]
    end
    return integral*h
end

end # module Gregory


### A Pluto.jl notebook ###
# v0.15.1

using Markdown
using InteractiveUtils

# ╔═╡ 9daa72fd-1145-4733-a21e-bd209367380a
using Pkg; Pkg.develop(path=".."); Pkg.activate("..")

# ╔═╡ 947fbdc9-3c48-44f3-b4d3-183f70e2194e
using LinearAlgebra

# ╔═╡ a465636e-bc12-4e8f-9db2-1a20ea771504
begin
	𝐱 = rand(100, 3)
	𝐱 = hcat(ones(size(𝐱, 1)), 𝐱)
	y = 𝐱 * [1, 2, 3, 4] + 1e-3randn(size(𝐱, 1))
end;

# ╔═╡ 833c56c2-d2ab-4f94-968b-b5771c9fa945
function irls(𝐱, y; p=2, n_iter=100)
	w = ones(size(𝐱, 1))
	𝐰 = diagm(w)
	β = inv(𝐱' * 𝐰 * 𝐱) * 𝐱' * 𝐰 * y

	if p == 2
		return β
	end

	for _ in 1:n_iter
		w = abs.(y - 𝐱 * β).^(p-2)
		𝐰 = diagm(w)
		β = inv(𝐱' * 𝐰 * 𝐱) * 𝐱' * 𝐰 * y
	end

	return β
end

# ╔═╡ 2c663777-b6e3-42b2-a279-d7b42dd27d70
irls(𝐱, y, p=4, n_iter=10)

# ╔═╡ Cell order:
# ╟─9daa72fd-1145-4733-a21e-bd209367380a
# ╠═947fbdc9-3c48-44f3-b4d3-183f70e2194e
# ╠═a465636e-bc12-4e8f-9db2-1a20ea771504
# ╠═833c56c2-d2ab-4f94-968b-b5771c9fa945
# ╠═2c663777-b6e3-42b2-a279-d7b42dd27d70


# This script solves an elliptic partial differential equation

using Printf
using LinearAlgebra

# Define the paths
CURRENT_DIR = @__DIR__
ROOT_DIR = basename(CURRENT_DIR) == "scripts" ? dirname(CURRENT_DIR) : CURRENT_DIR
TABLES = joinpath(ROOT_DIR, "tables")

# Prepare the output directories
mkpath(TABLES)

# Define the problem
l_x = 1
l_y = 1
p(x, y) = 1
q(x, y) = 1
u(x, y) = x^3 * y + y^2 * x
f(x, y) = -(6 * x * y + 2 * x)

# Define the grids
N = [5, 10, 20]
M = copy(N)

# Set the expected precision of the results
ε = 1e-4
digits = 4

"Compute the value of the differential operator L"
function L(U, x, y, h_x, h_y, i, j)::Float64
    return Λ₁(U, x, y, h_x, i, j) + Λ₂(U, x, y, h_y, i, j)
end

"Compute the value of the differential operator Λ₁"
function Λ₁(U, x, y, h_x, i, j)::Float64
    return p(x[i] + h_x / 2, y[j]) * (U[i+1, j] - U[i, j]) / h_x^2 -
           p(x[i] - h_x / 2, y[j]) * (U[i, j] - U[i-1, j]) / h_x^2
end

"Compute the value of the differential operator Λ₂"
function Λ₂(U, x, y, h_y, i, j)::Float64
    return q(x[i], y[j] + h_y / 2) * (U[i, j+1] - U[i, j]) / h_y^2 -
           q(x[i], y[j] - h_y / 2) * (U[i, j] - U[i, j-1]) / h_y^2
end

"Compute ξ, which is needed to compute an optimal number of iterations"
function compute_xi(l_x, l_y, h_x, h_y)::Float64
    rx = 0:0.001:l_x
    ry = 0:0.001:l_y
    pairs = [(x, y) for x in rx, y in ry]
    c₁ = minimum(args -> p(args...), pairs)
    c₂ = maximum(args -> p(args...), pairs)
    d₁ = minimum(args -> q(args...), pairs)
    d₂ = maximum(args -> q(args...), pairs)
    k₁ = 4 / h_x^2
    k₂ = 4 / h_y^2
    arg₁ = (π * h_x) / (2 * l_x)
    arg₂ = (π * h_y) / (2 * l_y)
    δ = c₁ * k₁ * sin(arg₁)^2 +
        d₁ * k₂ * sin(arg₂)^2
    Δ = c₂ * k₁ * cos(arg₁)^2 +
        d₂ * k₂ * cos(arg₂)^2
    return δ / Δ
end

"Initialize a new matrix and compute the boundary values"
function boundary_values(N, M, x, y)::Matrix{Float64}
    # Prepare a matrix for the solution
    U = zeros(N + 1, M + 1)
    # Compute the boundary values
    U[:, 1] .= u.(x, 0)
    U[:, M+1] .= u.(x, l_y)
    U[1, 2:M] .= u.(0, y[2:M])
    U[N+1, 2:M] .= u.(l_x, y[2:M])
    return U
end

"Iterate using the simple iteration method"
function simple_iteration(Uₖ₋₁, ξ, ε, N, M, h_x, h_y, x, y)::Tuple{Matrix{Float64},Int}
    # Prepare a new matrix
    Uₖ = copy(Uₖ₋₁)
    # Compute the optimal number of iterations
    m = ceil(Int, log(1 / ε) / (2ξ))
    # Iterate enough times to achieve desired precision
    for _ in 1:m
        # Compute values at the inner nodes
        for i = 2:N, j = 2:M
            k₁ = p(x[i] - h_x / 2, y[j]) / h_x^2
            k₂ = p(x[i] + h_x / 2, y[j]) / h_x^2
            k₃ = q(x[i], y[j] - h_y / 2) / h_y^2
            k₄ = q(x[i], y[j] + h_y / 2) / h_y^2
            Uₖ[i, j] = (k₁ * Uₖ₋₁[i-1, j] +
                        k₂ * Uₖ₋₁[i+1, j] +
                        k₃ * Uₖ₋₁[i, j-1] +
                        k₄ * Uₖ₋₁[i, j+1] +
                        f(x[i], y[j])) /
                       (k₁ + k₂ + k₃ + k₄)
        end
        # Reassign the current iteration as the previous one
        Uₖ₋₁ = Uₖ
    end
    return Uₖ, m
end

"Iterate using the Seidel's method"
function seidel(Uₖ₋₁, ξ, ε, N, M, h_x, h_y, x, y)::Tuple{Matrix{Float64},Int}
    # Prepare a new matrix
    Uₖ = copy(Uₖ₋₁)
    # Compute the optimal number of iterations
    m = ceil(Int, log(1 / ε) / (4ξ))
    # Iterate enough times to achieve desired precision
    for _ in 1:m
        # Compute values at the inner nodes
        for i = N:-1:2, j = M:-1:2
            k₁ = p(x[i] - h_x / 2, y[j]) / h_x^2
            k₂ = p(x[i] + h_x / 2, y[j]) / h_x^2
            k₃ = q(x[i], y[j] - h_y / 2) / h_y^2
            k₄ = q(x[i], y[j] + h_y / 2) / h_y^2
            Uₖ[i, j] = (k₁ * Uₖ[i-1, j] +
                        k₂ * Uₖ₋₁[i+1, j] +
                        k₃ * Uₖ[i, j-1] +
                        k₄ * Uₖ₋₁[i, j+1] +
                        f(x[i], y[j])) /
                       (k₁ + k₂ + k₃ + k₄)
        end
        # Reassign the current iteration as the previous one
        Uₖ₋₁ = Uₖ
    end
    return Uₖ, m
end

"Iterate using the upper relaxation method"
function upper_relaxation(Uₖ₋₁, ξ, ε, N, M, h_x, h_y, x, y)::Tuple{Matrix{Float64},Int}
    # Prepare a new matrix
    Uₖ = copy(Uₖ₋₁)
    # Compute the optimal number of iterations
    m = ceil(Int, log(1 / ε) / sqrt(ξ))
    # Define ω in (0,2), which affects the speed of convergence
    ω = 1.0
    # Iterate enough times to achieve desired precision
    for _ in 1:m
        # Compute values at the inner nodes
        for i = N:-1:2, j = M:-1:2
            k₁ = p(x[i] + h_x / 2, y[j]) / h_x^2
            k₂ = p(x[i] - h_x / 2, y[j]) / h_x^2
            k₃ = q(x[i], y[j] + h_y / 2) / h_y^2
            k₄ = q(x[i], y[j] - h_y / 2) / h_y^2
            Uₖ[i, j] = Uₖ₋₁[i, j] +
                       ω * (k₁ * (Uₖ₋₁[i+1, j] - Uₖ₋₁[i, j]) -
                            k₂ * (Uₖ₋₁[i, j] - Uₖ[i-1, j]) +
                            k₃ * (Uₖ₋₁[i, j+1] - Uₖ₋₁[i, j]) -
                            k₄ * (Uₖ₋₁[i, j] - Uₖ[i, j-1]) +
                            f(x[i], y[j])) /
                       (k₁ + k₂ + k₃ + k₄)
        end
        # Reassign the current iteration as the previous one
        Uₖ₋₁ = Uₖ
    end
    return Uₖ, m
end

"Iterate, using the alternately triangular iterative method"
function triangular(Uₖ₋₁, ε, N, M, h_x, h_y, x, y)::Tuple{Matrix{Float64},Int}
    # Prepare a new matrix
    Uₖ = copy(Uₖ₋₁)
    # Compute the coefficients
    pairs = [(x, y) for x in 0:0.001:l_x, y in 0:0.001:l_y]
    c₁ = minimum(args -> p(args...), pairs)
    c₂ = maximum(args -> p(args...), pairs)
    d₁ = minimum(args -> q(args...), pairs)
    d₂ = maximum(args -> q(args...), pairs)
    k₁ = 4 / h_x^2
    k₂ = 4 / h_y^2
    δ = c₁ * k₁ * sin((π * h_x) / (2 * l_x))^2 +
        d₁ * k₂ * sin((π * h_y) / (2 * l_y))^2
    Δ = c₂ * k₁ + d₂ * k₂
    ω = 2 / sqrt(δ * Δ)
    η = δ / Δ
    γ₁ = δ / (2 + 2 * sqrt(η))
    γ₂ = δ / (4 * sqrt(η))
    ξ = γ₁ / γ₂
    κ₁ = ω / h_x^2
    κ₂ = ω / h_y^2
    τ = 2 / (γ₁ + γ₂)
    # Compute the optimal number of iterations
    m = ceil(Int, log(1 / ε) / log((1 + ξ) / (1 - ξ)))
    # Iterate enough times to achieve desired precision
    for _ in 1:m
        # Prepare intermediate matrices
        Ũ = zeros(size(Uₖ₋₁)...)
        U̅ = zeros(size(Uₖ₋₁)...)
        # Compute values at the inner nodes of the first intermediate matrix
        for i = 2:N, j = 2:M
            k₁ = κ₁ * p(x[i] - h_x / 2, y[j])
            k₂ = κ₂ * q(x[i], y[j] - h_y / 2)
            Ũ[i, j] = (k₁ * Ũ[i-1, j] + k₂ * Ũ[i, j-1] +
                        L(Uₖ₋₁, x, y, h_x, h_y, i, j) + f(x[i], y[j])) /
                       (1 + k₁ + k₂)
        end
        # Compute values at the inner nodes of the second intermediate matrix
        for i = N:-1:2, j = M:-1:2
            k₁ = κ₁ * p(x[i] + h_x / 2, y[j])
            k₂ = κ₂ * q(x[i], y[j] + h_y / 2)
            U̅[i, j] = (k₁ * U̅[i+1, j] + k₂ * U̅[i, j+1] + Ũ[i, j]) /
                       (1 + k₁ + k₂)
        end
        # Add up the matrices
        Uₖ[2:N, 2:M] .= Uₖ₋₁[2:N, 2:M] .+ τ .* U̅[2:N, 2:M]
        # Reassign the current iteration as the previous one
        Uₖ₋₁ = Uₖ
    end
    return Uₖ, m
end

"Iterate using the variable directions method"
function variable_directions(Uₖ₋₁, ε, N, M, h_x, h_y, x, y)::Tuple{Matrix{Float64},Int}
    # Prepare a new matrix and an intermediate matrix
    Uₖ = copy(Uₖ₋₁)
    U̅ = copy(Uₖ₋₁)
    # Compute the optimal number of iterations
    m = ceil(Int, N / (2 * π) * log(1 / ε))
    # Compute the coefficients
    pairs = [(x, y) for x in 0:0.001:l_x, y in 0:0.001:l_y]
    c₁ = minimum(args -> p(args...), pairs)
    c₂ = maximum(args -> p(args...), pairs)
    d₁ = minimum(args -> q(args...), pairs)
    d₂ = maximum(args -> q(args...), pairs)
    k₁ = 4 / h_x^2
    k₂ = 4 / h_y^2
    arg₁ = (π * h_x) / (2 * l_x)
    arg₂ = (π * h_y) / (2 * l_y)
    δ₁ = c₁ * k₁ * sin(arg₁)^2
    δ₂ = d₁ * k₂ * sin(arg₂)^2
    Δ₁ = c₂ * k₁ * cos(arg₁)^2
    Δ₂ = d₂ * k₁ * cos(arg₂)^2
    δ = min(δ₁, δ₂)
    Δ = max(Δ₁, Δ₂)
    τ = 2 / sqrt(δ * Δ)
    # Iterate enough times to achieve desired precision
    for _ in 1:m
        # Compute values at the inner nodes of the intermediate matrix
        for j = 2:M
            # Compute the linear system's matrix
            Ũ = Tridiagonal(
                # A's
                [repeat([τ / (2 * h_x^2)], N - 1); 0],
                # B's
                [1; repeat([-τ / h_x^2 + 1], N - 1); 1],
                # C's
                [0; repeat([τ / (2 * h_x^2)], N - 1)],
            )
            # Compute the linear system's right-hand side vector
            g = [
                Uₖ₋₁[1, j]
                [-Uₖ₋₁[i, j] -
                 τ / 2 * (Λ₂(Uₖ₋₁, x, y, h_y, i, j) +
                          f(x[i], y[j]))
                 for i in 2:N
                ]
                Uₖ₋₁[N+1, j]]
            # Compute the solution of the linear system
            U̅[:, j] .= Ũ \ g
        end
        # Compute values at the inner nodes of the input matrix
        for i = 2:N
            # Compute the linear system's matrix
            Ũ = Tridiagonal(
                # A's
                [repeat([τ / (2 * h_y^2)], N - 1); 0],
                # B's
                [1; repeat([-τ / h_y^2 + 1], N - 1); 1],
                # C's
                [0; repeat([τ / (2 * h_y^2)], N - 1)],
            )
            # Compute the linear system's right-hand side vector
            g = [
                Uₖ₋₁[i, 1]
                [-U̅[i, j] -
                 τ / 2 * (Λ₁(U̅, x, y, h_x, i, j) +
                          f(x[i], y[j]))
                 for j in 2:M
                ]
                Uₖ₋₁[i, M+1]]
            # Compute the solution of the linear system
            Uₖ[i, :] .= Ũ \ g
        end
        # Reassign the current iteration as the previous one
        Uₖ₋₁ = Uₖ
    end
    return Uₖ, m
end

"Create the TeX tables and write them to disk"
function tables(dir, U, N, M; digits = 8)
    # Prepare the output directory
    OUTPUT_DIR = joinpath(TABLES, dir)
    mkpath(OUTPUT_DIR)
    # Prepare the ranges
    ri = Int.([1:N/5:N+1]...)
    rj = Int.([1:M/5:M+1]...)
    # Create and write the table with the solution grid
    open(joinpath(OUTPUT_DIR, "$N, $M.tex"), "w") do io
        for i in ri
            s = ""
            for j in rj
                number = round(U[i, j]; digits)
                s = "$(s)& \$ $(Printf.format(Printf.Format("%.0$(digits)f"), number == 0 ? 0 : number)) \$ "
            end
            s = "$(s)\\\\"
            println(io, s)
        end
    end
end

"""
Compare the approximate solution to the exact one,
find the biggest absolute difference between the nodes
"""
function max_difference(U, x, y)::Float64
    M1, N1 = size(U)
    # Find the maximum
    Δu = 0
    for i = 1:N1, j = 1:M1
        _Δu = abs(u(x[i], y[j]) - U[i, j])
        if _Δu > Δu
            Δu = _Δu
        end
    end
    return Δu
end

# For each grid
for l in eachindex(N)
    # Compute the steps
    h_x = l_x / N[l]
    h_y = l_y / M[l]
    # Print the numbers of nodes
    println(
        '\n',
        " "^5, "N: ", N[l], '\n',
        " "^5, "M: ", M[l]
    )
    # Compute the nodes
    x = collect(0:h_x:1.0)
    y = collect(0:h_y:1.0)
    # Compute ξ
    ξ = compute_xi(l_x, l_y, h_x, h_y)
    # Create a new matrix and compute the boundary values
    U = boundary_values(N[l], M[l], x, y)
    # Iterate, using the simple iteration method
    U, m = simple_iteration(U, ξ, ε, N[l], M[l], h_x, h_y, x, y)
    # Create and write the TeX tables
    tables("simple_iteration", U, N[l], M[l]; digits)
    # Print the info
    println(
        '\n',
        " "^5, "> Simple iteration:", '\n',
        " "^5, "  m = ", m, '\n',
        " "^5, "  Δu = ", max_difference(U, x, y)
    )
    # Iterate, using the Seidel's method
    U, m = seidel(U, ξ, ε, N[l], M[l], h_x, h_y, x, y)
    # Create and write the TeX tables
    tables("seidel", U, N[l], M[l]; digits)
    # Print the info
    println(
        '\n',
        " "^5, "> Seidel:", '\n',
        " "^5, "  m = ", m, '\n',
        " "^5, "  Δu = ", max_difference(U, x, y)
    )
    # Iterate, using the upper relaxation method
    U, m = upper_relaxation(U, ξ, ε, N[l], M[l], h_x, h_y, x, y)
    # Create and write the TeX tables
    tables("upper_relaxation", U, N[l], M[l]; digits)
    # Print the info
    println(
        '\n',
        " "^5, "> Upper relaxation:", '\n',
        " "^5, "  m = ", m, '\n',
        " "^5, "  Δu = ", max_difference(U, x, y)
    )
    # Iterate, using the alternately triangular iterative method
    U, m = triangular(U, ε, N[l], M[l], h_x, h_y, x, y)
    # Create and write the TeX tables
    tables("triangular", U, N[l], M[l]; digits)
    # Print the info
    println(
        '\n',
        " "^5, "> Alternately triangular iterative method:", '\n',
        " "^5, "  m = ", m, '\n',
        " "^5, "  Δu = ", max_difference(U, x, y)
    )
    # Iterate, using the variable directions method
    U, m = variable_directions(U, ε, N[l], M[l], h_x, h_y, x, y)
    # Create and write the TeX tables
    tables("variable_directions", U, N[l], M[l]; digits)
    # Print the info
    println(
        '\n',
        " "^5, "> Variable directions method:", '\n',
        " "^5, "  m = ", m, '\n',
        " "^5, "  Δu = ", max_difference(U, x, y)
    )
end

println()


# This is an exact model of a velocity controlled differential drive
# under the following assumptions:
# - the control was applied at the beginning of the time slot
# - the control took effect immediately
# - the wheels don't slip

L = 0.13    # m : distance between the wheels
r = 0.033   # m : wheel radius

# Creates a new state based upon the previous one assuming that the
# specified control that was executed for the specified time.
function new_state(state, control, elapsed_time)
    x, y, θ = state
    ω_left, ω_right = control

    θ⁺ = θ + Δθ(ω_left, ω_right, elapsed_time)
    x⁺ = x + Δx(ω_left, ω_right, θ, θ⁺, elapsed_time)
    y⁺ = y + Δy(ω_left, ω_right, θ, θ⁺, elapsed_time)

    return [x⁺ y⁺ θ⁺]
end

# Change in the x position after the elapsed time.
function Δx(ω_left, ω_right, θ_init, θ_new, elapsed_time)
    if ω_left != ω_right
        L/2.0 * (ω_left + ω_right)/(ω_right - ω_left) * (sin(θ_new) - sin(θ_init))
    else
        r/2.0 * (ω_left + ω_right) * cos(θ_init) * elapsed_time
    end
end

# Change in the y position after the elapsed time.
function Δy(ω_left, ω_right, θ_init, θ_new, elapsed_time)
    if ω_left != ω_right
        L/2.0 * (ω_left + ω_right)/(ω_right - ω_left) * (cos(θ_init) - cos(θ_new))
    else
        r/2.0 * (ω_left + ω_right) * sin(θ_init) * elapsed_time
    end
end

# Change in the orientation after the elapsed time.
Δθ(ω_left, ω_right, elapsed_time) = r/L * (ω_right - ω_left) * elapsed_time

# Creates a new state given with the specified x, y and orientation
# components. The components should be in identical coordinate systems,
# e.g. world.
initial_state(x_world, y_world, θ_world) = [x_world y_world θ_world]


# This file is a part of SimilaritySearch.jl

export CosineDistance, AngleDistance, NormalizedCosineDistance, NormalizedAngleDistance
using LinearAlgebra
import Distances: evaluate

"""
   CosineDistance()

The cosine is defined as:
```math
\\cos(u, v) = \\frac{\\sum_i u_i v_i}{\\sqrt{\\sum_i u_i^2} \\sqrt{\\sum_i v_i^2}}
```

The cosine distance is defined as ``1 - \\cos(u,v)``
"""
struct CosineDistance <: PreMetric end

"""
   AngleDistance()

The angle distance is defined as:
```math
∠(u, v)= \\arccos(\\cos(u, v))
```

"""
struct AngleDistance <: PreMetric end
"""
    NormalizedCosineDistance()

Similar to [`CosineDistance`](@ref) but suppose that input vectors are already normalized

```math
1 - \\sum_i {u_i v_i}
```

"""
struct NormalizedCosineDistance <: PreMetric end

"""
    NormalizedAngleDistance()

Similar to [`AngleDistance`](@ref) but suppose that input vectors are already normalized

```math
\\arccos \\sum_i {u_i v_i}
```

"""
struct NormalizedAngleDistance <: PreMetric end

const π_2 = π / 2

function fastacos(d)
    if d <= -1.0
        π
    elseif d >= 1.0
        0.0
    elseif d == 0  # turn around for zero vectors, in particular for denominator=0
        π_2
    else
        acos(d)
    end
end

"""
    evaluate(::NormalizedCosineDistance, a, b)

Computes the cosine distance between two vectors, it expects normalized vectors (see [normalize!](@ref) method).
Please use NormalizedAngleDistance if you are expecting a metric function (cosine_distance is a faster
alternative whenever the triangle inequality is not needed)
"""
evaluate(::NormalizedCosineDistance, a, b) = one(eltype(a)) - dot(a, b)

"""
    evaluate(::AngleDistance, a, b)

Computes the angle  between twovectors. It supposes that all vectors are normalized (see `normalize!` function)

"""
evaluate(::NormalizedAngleDistance, a, b) = fastacos(dot(a, b))

"""
    evaluate(::CosineDistance, a, b)

Computes the cosine distance between two vectors.
Please use AngleDistance if you are expecting a metric function (cosine_distance is a faster
alternative whenever the triangle inequality is not needed)
"""
evaluate(::CosineDistance, a, b) = one(eltype(a)) - dot(a, b) / (norm(a) * norm(b))

"""
    evaluate(::AngleDistance, a, b)

Computes the angle  between twovectors.

"""
function evaluate(::AngleDistance, a, b)
    d = dot(a, b) / (norm(a) * norm(b))
    fastacos(d)
end


module CircleFit
import StatsBase
import StatsBase: RegressionModel, residuals, coef, coefnames, dof
import Statistics: var, cov, stdm

export circfit, Circle, algorithm

"""
Circle fit model

Currently only in 2D

* position: center position of the fitted circle
* radius: radius of the fitted circle
* points: the data to fit to. Points are stored as a matrix (number of points, number of dimensions)
* alg: algorithm to use. Possible options are :kasa, :pratt, :graf and :taubin

To get the coefficients one can use StatsBase.coef
The coeffient names are provived by StatsBase.coefnames
"""
struct Circle <: RegressionModel
    position::AbstractArray
    radius
    points::AbstractArray
    alg::Symbol
end

"""
Get the algorithm used in the fit
"""
algorithm(model::Circle) = model.alg

# StatsBase methods

StatsBase.coef(fit::Circle) = (fit.position..., fit.radius)
StatsBase.coefnames(fit::Circle) = (("center position x".*string.(1:length(fit.position)))..., "radius")
StatsBase.dof(fit::Circle) = size(fit.points,1) - length(coef(fit))
function StatsBase.residuals(fit::Circle)
    rs = @. hypot(fit.points[:,1] - fit.position[1], fit.points[:,2] - fit.position[2])
    rs .- fit.radius
end
StatsBase.rss(fit::Circle) = sum(abs2.(residuals(fit)))

function StatsBase.fit(::Type{Circle},x::AbstractArray,y::AbstractArray;alg=:kasa)
    x0,y0,r = if alg == :taubin
        taubin(x,y)
    elseif alg == :pratt
        pratt(x,y)
    elseif alg == :graf
        p0 = collect(kasa(x,y))
        GRAF(x,y,p0)
    else
        kasa(x,y)
    end
    Circle([x0,y0],r,[x y],alg)
end

# Old method interface

"""
Fit a circle to points provided as arrays of x and y coordinates

Example
```
x = [-1.0,0,0,1]
y = [0.0,1,-1,0]
x0,y0,radius = circfit(x,y)
```
"""
circfit(x,y) = kasa(x,y)

@deprecate circfit(x,y) StatsBase.fit(Circle,x,y) false

"""
Fit a circle to the points provided as arrays of x and y coordinates

This method uses [Kåsa's method](https://doi.org/10.1109/TIM.1976.6312298)
The result is a GeometryBasics::Circle
"""
function kasa(x::AbstractArray, y::AbstractArray)
    x² = x.^2
    y² = y.^2

    A = var(x)
    B = cov(x, y)
    C = var(y)
    D = cov(x, y²) + cov(x, x²)
    E = cov(y, x²) + cov(y, y²)

    ACB2 = 2 * (A * C - B^2)
    am = (D * C - B * E) / ACB2
    bm = (A * E - B * D) / ACB2
    rk = hypot(stdm(x, am, corrected=false), stdm(y, bm, corrected=false))

    (am, bm, rk)
end

using LinearAlgebra

"""
Fit a circle by using Taubin's method
https://doi.org/10.1007/s10851-005-0482-8
Warning: not optimized
"""
function taubin(x,y)

    z = x.^2 .+ y.^2
    Mx = sum(x)
    My = sum(y)
    Mz = sum(z)
    Mxx = sum(x.^2)
    Myx = Mxy = sum(x.*y)
    Mzx = Mxz = sum(x.*z)
    Myy = sum(y.^2)
    Mzy = Myz = sum(y.*z)
    Mzz = sum(z.^2)
    n = length(x)

    C = [4Mz 2Mx 2My 0
         2Mx n   0   0
         2My 0   n   0
         0   0   0   0]

    M = [Mzz Mxz Myz Mz
         Mxz Mxx Mxy Mx
         Myz Mxy Myy My
         Mz  Mx  My  n]

    F = eigen(M,C)

    values = F.values
    values[values .< 0] .= Inf
    i = argmin(values)

    A,B,C,D = F.vectors[:,i]

    a = -B/(2*A)
    b = -C/(2*A)
    r = sqrt((B^2+C^2-4*A*D)/(4*A^2))

    (a, b, r)
end

"""
Fit a circle by using the method of Pratt
https://doi.org/10.1007/s10851-005-0482-8
Warning: not optimized
"""
function pratt(x,y)
    z = x.^2 .+ y.^2
    Mx = sum(x)
    My = sum(y)
    Mz = sum(z)
    Mxx = sum(x.^2)
    Myx = Mxy = sum(x.*y)
    Mzx = Mxz = sum(x.*z)
    Myy = sum(y.^2)
    Mzy = Myz = sum(y.*z)
    Mzz = sum(z.^2)
    n = length(x)

    B = [0  0  0 -2
         0  1  0  0
         0  0  1  0
        -2  0  0  0]

    M = [Mzz Mxz Myz Mz
         Mxz Mxx Mxy Mx
         Myz Mxy Myy My
         Mz  Mx  My  n]

    F = eigen(M,B)

    values = F.values
    values[values .< 0] .= Inf
    i = argmin(values)

    A,B,C,D = F.vectors[:,i]

    a = -B/(2*A)
    b = -C/(2*A)
    r = sqrt((B^2+C^2-4*A*D)/(4*A^2))

    (a, b, r)
end

import LsqFit: levenberg_marquardt, OnceDifferentiable, minimizer

"""
Gradient weighted algebraic fit
* x: vector of x coordinates
* y: vector of y coordiantes
* p0: starting values for the fit parameters(position x, position y , radius)
* kwargs are passed to `LsqFit.levenberg_marquardt`

return (position x, position y , radius)
"""
function GRAF(x,y,p0;kwargs...)
    x1 = x
    x2 = y
    z = @. x1^2 + x2^2

    model_inplace = (F, p) -> begin
        B,C,D = p
        A = 1
        @. F = (A*z + B*x1 + C*x2 + D) / (4*A*(A*z+B*x1+C*x2+D)+B^2+C^2-4*A*D)
    end
    jacobian_inplace = (F::Array{Float64,2},p) -> begin
        A = 1
        B,C,D = p

        # dA
        #@. F[:,1] = z / (4*A*(A*z+B*x1+C*x2+D)+B^2+C^2-4*A*D) - (A*z + B*x1 + C*x2 + D) / (4*A*(A*z+B*x1+C*x2+D)+B^2+C^2-4*A*D)^2 * (8*A*z-4*D)
        # dB
        @. F[:,1] = x1 / (4*A*(A*z+B*x1+C*x2+D)+B^2+C^2-4*A*D) - (A*z + B*x1 + C*x2 + D) / (4*A*(A*z+B*x1+C*x2+D)+B^2+C^2-4*A*D)^2 * (4*A*x1+2*B)
        # dC
        @. F[:,2] = x2 / (4*A*(A*z+B*x1+C*x2+D)+B^2+C^2-4*A*D) - (A*z + B*x1 + C*x2 + D) / (4*A*(A*z+B*x1+C*x2+D)+B^2+C^2-4*A*D)^2 * (4*A*x2+2*C)
        # dD
        @. F[:,3] = 1 / (4*A*(A*z+B*x1+C*x2+D)+B^2+C^2-4*A*D)
    end
    p0_ext = [abr_to_BCD(p0...)...]
    R = OnceDifferentiable(model_inplace, jacobian_inplace, p0_ext, similar(x); inplace = true)
    results = levenberg_marquardt(R, p0_ext; kwargs...)
    coef = minimizer(results)
    BCD_to_abr(coef[1:end]...)
end

"""
convert the parametric form of
z+B*x+C*y+D -> (x-a)²+(y-b)²-r²
"""
function BCD_to_abr(B,C,D)
    [-B/2,-C/2,sqrt(B^2/4+C^2/4-D)]
end

"""
convert the parametric form of
z+B*x+C*y+D <- (x-a)²+(y-b)²-r²
"""
function abr_to_BCD(a,b,r)
    [-2a,-2b,a^2+b^2-r^2]
end

end # module


### A Pluto.jl notebook ###
# v0.11.4

using Markdown
using InteractiveUtils

# ╔═╡ 5a0472c2-f40b-11ea-231b-a52e7382c1bc
md"""Recall the mean value theorem:

Let $f(x)$ be a function that is continuous on $[a,b]$ and differentiable on $(a,b)$.

Then there is a number $c$ in between $a,b$ such that:

$$f'(c) = \frac{f(b)-f(a)}{b-a}$$.

This means that there is a point with a derivative equal to the slope between the boundary points.

Let's take the following function:

$$f(x) = x^3+2x-x \text{ on } [-1,2]$$"""

# ╔═╡ bc85d0e0-f40d-11ea-2344-dd5ea4b66cde
md""" By the mean value theorem:

$$\frac{f(b)-f(a)}{b-a}= \frac{(10)-(-2)}{2-(-1)} = 4 = f'(c)$$

for some $c$ in $(-1,2)$.

With some algebra and rearranging, we find

$$c = \frac{-4+\sqrt{76}}{6}= 0.7863$$.

Graph:
"""

# ╔═╡ 40d54e00-f40b-11ea-3ddf-a39ce834ea41
using Gadfly

# ╔═╡ 54dca870-f411-11ea-1f67-ad216b5b2e49
set_default_plot_size(20cm, 15cm)

# ╔═╡ dd853160-f40c-11ea-318e-5751cdc6ffdf
begin
	p = plot((x)->(x^3+2x-x),-1,2)
	push!(p,Coord.cartesian(fixed=true))
	push!(p,Guide.title("f(x)"))
end

# ╔═╡ 5e773960-f40f-11ea-1894-0b4feb0e99e3
begin
	push!(p,layer(x=[0.7863],y=[1.2724], size=[0.25], color=[colorant"gold"]))
	push!(p,layer((x)->(4x-1.8728),0,2))
	push!(p,layer((x)->(4x+2),-1,2))
end

# ╔═╡ Cell order:
# ╟─40d54e00-f40b-11ea-3ddf-a39ce834ea41
# ╠═5a0472c2-f40b-11ea-231b-a52e7382c1bc
# ╟─54dca870-f411-11ea-1f67-ad216b5b2e49
# ╟─dd853160-f40c-11ea-318e-5751cdc6ffdf
# ╠═bc85d0e0-f40d-11ea-2344-dd5ea4b66cde
# ╠═5e773960-f40f-11ea-1894-0b4feb0e99e3


using CSV, DataFrames, StatsBase
using StanSample, StatsPlots
using StatisticalRethinkingPlots, StatisticalRethinking

df = CSV.read(sr_datadir("WaffleDivorce.csv"), DataFrame);
df[!, :M] = zscore(df.Marriage)
df[!, :A] = zscore(df.MedianAgeMarriage)
df[!, :D] = zscore(df.Divorce)
data = (N=size(df, 1), D=df.D, A=df.A, M=df.M)

stan5_1 = "
data {
    int < lower = 1 > N; // Sample size
    vector[N] D; // Outcome
    vector[N] A; // Predictor
}
parameters {
    real a; // Intercept
    real bA; // Slope (regression coefficients)
    real < lower = 0 > sigma;    // Error SD
}
transformed parameters {
    vector[N] mu;               // mu is a vector
    for (i in 1:N)
        mu[i] = a + bA * A[i];
}
model {
    a ~ normal(0, 0.2);         //Priors
    bA ~ normal(0, 0.5);
    sigma ~ exponential(1);
    D ~ normal(mu , sigma);     // Likelihood
}
generated quantities {
    vector[N] log_lik;
    for (i in 1:N)
        log_lik[i] = normal_lpdf(D[i] | mu[i], sigma);
}
";

stan5_2 = "
data {
    int N;
    vector[N] D;
    vector[N] M;
}
parameters {
    real a;
    real bM;
    real<lower=0> sigma;
}
transformed parameters {
    vector[N] mu;
    for (i in 1:N)
        mu[i]= a + bM * M[i];

}
model {
    a ~ normal( 0 , 0.2 );
    bM ~ normal( 0 , 0.5 );
    sigma ~ exponential( 1 );
    D ~ normal( mu , sigma );
}
generated quantities {
    vector[N] log_lik;
    for (i in 1:N)
        log_lik[i] = normal_lpdf(D[i] | mu[i], sigma);
}
";

stan5_3 = "
data {
  int N;
  vector[N] D;
  vector[N] M;
  vector[N] A;
}
parameters {
  real a;
  real bA;
  real bM;
  real<lower=0> sigma;
}
transformed parameters {
    vector[N] mu;
    for (i in 1:N)
        mu[i] = a + bA * A[i] + bM * M[i];
}
model {
  a ~ normal( 0 , 0.2 );
  bA ~ normal( 0 , 0.5 );
  bM ~ normal( 0 , 0.5 );
  sigma ~ exponential( 1 );
  D ~ normal( mu , sigma );
}
generated quantities{
    vector[N] log_lik;
    for (i in 1:N)
        log_lik[i] = normal_lpdf(D[i] | mu[i], sigma);
}
";

fname = joinpath(@__DIR__, "coeftab_plot.png")
isfile(fname) && rm(fname)

m5_1s = SampleModel("m5.1s", stan5_1)
rc5_1s = stan_sample(m5_1s; data)

m5_2s = SampleModel("m5.2s", stan5_2)
rc5_2s = stan_sample(m5_2s; data)

m5_3s = SampleModel("m5.3s", stan5_3)
rc5_3s = stan_sample(m5_3s; data)
models = [m5_1s, m5_2s, m5_3s]

if success(rc5_1s) && success(rc5_2s) && success(rc5_3s)

    m5_1s_df = read_samples(m5_1s, :dataframe)
    m5_2s_df = read_samples(m5_2s, :dataframe)
    m5_3s_df = read_samples(m5_3s, :dataframe)

    if isinteractive()

        coeftab_plot(m5_1s_df, m5_2s_df, m5_3s_df; pars=(:bA, :bM),
            names=["m5.1", "m5.2", "m5.3"])
        savefig(joinpath(@__DIR__, "coeftab_plot.png"))

    end
end


using FFTW, BenchmarkTools, FourierFlows, Random
using LinearAlgebra: mul!, ldiv!
using Random: seed!

seed!(1234) # for reproducibility

Lx, Ly = 2π, 3π
nx, ny = 256, 256
T, tolerance = Float64, 1e-12
# T, tolerance = Float32, 1e-6

gr =  TwoDGrid(nx, Lx, ny, Ly, T=T)

# test function used in bench tests
f = real(ifft(rand(T, (nx, ny)) + im*rand(T, (nx, ny))))
fcomplex  = Complex.(f)

# initialize empty arrays
dfdx = zeros(T, (nx, ny))
dfdx_c = zeros(Complex{T}, (nx, ny))
 fh  = zeros(Complex{T}, (gr.nk, gr.nl))
 fhr = zeros(Complex{T}, (gr.nkr, gr.nl))

# various ways of computing ∂f/∂x

function dx_using_fft(f)
  fh = fft(f)
  dfdx = real(ifft(im*gr.k .* fh))
  return dfdx
end

function dx_using_rfft(f)
  fhr = rfft(f)
  dfdx = irfft(im*gr.kr .* fhr, nx)
  return dfdx
end

effort = FFTW.PATIENT
FFTW.set_num_threads(Sys.CPU_THREADS)
fftplan = plan_fft(Array{T, 2}(undef, nx, ny), flags=effort)
rfftplan = plan_rfft(Array{T, 2}(undef, nx, ny), flags=effort)

function dx_using_fftplan(f)
  fh = fftplan*f
  @. fh = im*gr.k * fh
  dfdx = fftplan \ fh
  return real.(dfdx)
end

function dx_using_rfftplan(f)
  fhr = rfftplan*f
  @. fhr = im*gr.kr * fhr
  dfdx = rfftplan \ fhr
  return dfdx
end

function dx_using_fftplan_mul(f)
  mul!(fh, fftplan, fcomplex) #fftplan within mul! only works if all arrays are complex-valued
  @. fh = im*gr.k * fh
  ldiv!(dfdx_c, fftplan, fh) #fftplan within ldiv! only works if all arrays are complex-valued
  return real.(dfdx_c)
end

function dx_using_rfftplan_mul(f)
  mul!(fhr, rfftplan, f)
  @. fhr = im*gr.kr * fhr
  ldiv!(dfdx, rfftplan, fhr)
  return dfdx
end

dfdx1 = dx_using_fft(f)
dfdx2 = dx_using_rfft(f)
dfdx3 = dx_using_fftplan(f)
dfdx4 = dx_using_rfftplan(f)
dfdx5 = dx_using_fftplan_mul(f)
dfdx6 = dx_using_rfftplan_mul(f)

if (isapprox(dfdx1, dfdx2, rtol=tolerance) && isapprox(dfdx1, dfdx3, rtol=tolerance)
    && isapprox(dfdx1, dfdx4, rtol=tolerance) && isapprox(dfdx1, dfdx5, rtol=tolerance)
    && isapprox(dfdx1, dfdx6, rtol=tolerance))
 #make sure that all functions compute ∂f/∂x the same

  println("Performing bench tests for 2D FFTs using nx=", nx, " and ny=", ny, " grid-points with ", T, " arithmetic.")
  println(" ")
  println("computing ∂f/∂x using fft")
  @btime dx_using_fft(f);
  println(" ")
  println("computing ∂f/∂x using rfft")
  @btime dx_using_rfft(f);
  println(" ")
  println("computing ∂f/∂x using fftplan")
  @btime dx_using_fftplan(f);
  println(" ")
  println("computing ∂f/∂x using rfftplan")
  @btime dx_using_rfftplan(f);
  println(" ")
  println("computing ∂f/∂x using fftplan & mul!")
  @btime dx_using_fftplan_mul(f);
  println(" ")
  println("computing ∂f/∂x using rfftplan & mul!")
  @btime dx_using_rfftplan_mul(f);
else
  error("something went wrong while computing the derivatives")
end


# Solve level-set equations in 2D using the Jiang-Lin-Tadmor method
# Alex Tam, 09/11/2021

"Function to solve level-set equation"
function level_set(V, ϕ, par, dx, dy, dt)
    ϕn = Array{Float64}(undef, par.Nx, par.Ny) # Pre-allocate array of ϕ
    # Compute solutions on staggered grid using Lin-Tadmor
    ϕs = lt_staggered(V, ϕ, par, dx, dy, dt)
    # Compute derivatives on staggered grid
    dxϕs = Array{Float64}(undef, par.Nx-1, par.Ny-1) # Pre-allocate x-derivatives at staggered grid points
    dyϕs = Array{Float64}(undef, par.Nx-1, par.Ny-1) # Pre-allocate y-derivatives at staggered grid points
    for i = 1:par.Nx-1
        for j = 1:par.Ny-1 # Loop over staggered grid points
            if i == 1
                dxϕs[i,j] = (ϕs[i+1,j]-ϕs[i,j])/dx # ϕ'_{i+1/2,j+1/2}/Δx
            elseif i == par.Nx-1
                dxϕs[i,j] = (ϕs[i,j]-ϕs[i-1,j])/dx # ϕ'_{i+1/2,j+1/2}/Δx
            else
                dxϕs[i,j] = minmod(par.θ*(ϕs[i+1,j]-ϕs[i,j])/dx, (ϕs[i+1,j]-ϕs[i-1,j])/(2*dx), par.θ*(ϕs[i,j]-ϕs[i-1,j])/dx) # ϕ'_{i+1/2,j+1/2}/Δx
            end
            if j == 1
                dyϕs[i,j] = (ϕs[i,j+1]-ϕs[i,j])/dy # ϕ`_{i+1/2,j+1/2}/Δy
            elseif j == par.Ny-1
                dyϕs[i,j] = (ϕs[i,j]-ϕs[i,j-1])/dy # ϕ`_{i+1/2,j+1/2}/Δy
            else
                dyϕs[i,j] = minmod(par.θ*(ϕs[i,j+1]-ϕs[i,j])/dy, (ϕs[i,j+1]-ϕs[i,j-1])/(2*dy), par.θ*(ϕs[i,j]-ϕs[i,j-1])/dy) # ϕ`_{i+1/2,j+1/2}/Δy
            end
        end
    end
    # Perform cell-averaging at non-staggered points (Jiang 1998)
    for i = 1:par.Nx
        for j = 1:par.Ny # Loop over non-staggered grid points
            if (i == 1) || (j == 1) || (i == par.Nx) || (j == par.Ny) # Ignore boundaries
                ϕn[i,j] = ϕ[i,j]
            else
                ϕn[i,j] = (ϕs[i,j] + ϕs[i-1,j] + ϕs[i,j-1] + ϕs[i-1,j-1])/4 +
                (dx*(dxϕs[i-1,j-1]-dxϕs[i,j-1]) + dx*(dxϕs[i-1,j]-dxϕs[i,j]) + dy*(dyϕs[i-1,j-1]-dyϕs[i-1,j]) + dy*(dyϕs[i,j-1]-dyϕs[i,j]))/16 # Compute non-staggered cell average
            end
        end
    end
    return ϕn
end

"Staggered Lin-Tadmor scheme"
function lt_staggered(V, ϕ, par, dx, dy, dt)
    ϕ_stag = Array{Float64}(undef, par.Nx-1, par.Ny-1) # Pre-allocate solution on staggered grid ϕ_stag[i,j] = ϕ[i+1/2, j+1/2], one-based indexing
    V_stag = Array{Float64}(undef, par.Nx-1, par.Ny-1) # Pre-allocate velocity on staggered grid V_stag[i,j] = ϕ[i+1/2, j+1/2], one-based indexing
    dxϕ = Array{Float64}(undef, par.Nx, par.Ny) # Pre-allocate x-derivatives at non-staggered grid points
    dyϕ = Array{Float64}(undef, par.Nx, par.Ny) # Pre-allocate y-derivatives at non-staggered grid points
    ϕ_mid = Array{Float64}(undef, par.Nx, par.Ny) # Pre-allocate mid-values at non-staggered grid points
    # Evaluate derivatives and missing mid-values on full non-staggered grid
    for i = 1:par.Nx
        for j = 1:par.Ny
            # Compute dϕ/dx
            if i == 1 # Boundary condition
                dxϕ[i,j] = (ϕ[i+1,j] - ϕ[i,j])/dx
            elseif i == par.Nx # Boundary condition
                dxϕ[i,j] = (ϕ[i,j] - ϕ[i-1,j])/dx
            else # Apply min-mod at interior grid points
                dxϕ[i,j] = minmod(par.θ*(ϕ[i+1,j]-ϕ[i,j])/dx, (ϕ[i+1,j]-ϕ[i-1,j])/(2*dx), par.θ*(ϕ[i,j]-ϕ[i-1,j])/dx)
            end
            # Compute dϕ/dy
            if j == 1 # Boundary condition
                dyϕ[i,j] = (ϕ[i,j+1] - ϕ[i,j])/dy
            elseif j == par.Ny # Boundary condition
                dyϕ[i,j] = (ϕ[i,j] - ϕ[i,j-1])/dy
            else # Apply min-mod at interior grid points
                dyϕ[i,j] = minmod(par.θ*(ϕ[i,j+1]-ϕ[i,j])/dy, (ϕ[i,j+1]-ϕ[i,j-1])/(2*dy), par.θ*(ϕ[i,j]-ϕ[i,j-1])/dy)
            end
            ϕ_mid[i,j] = ϕ[i,j] - dt/2*hamiltonian_ls(V[i,j], dxϕ[i,j], dyϕ[i,j]) # Missing mid-values
        end
    end
    # Obtain velocity field on staggered grid
    for i = 1:par.Nx-1
        for j = 1:par.Ny-1 # Loop over staggered grid points
            V_stag[i,j] = (V[i,j] + V[i+1,j] + V[i,j+1] + V[i+1,j+1])/4
        end
    end
    # Evaluate solution on staggered grid
    for i = 1:par.Nx-1
        for j = 1:par.Ny-1 # Loop over staggered grid points
            ϕ_stag[i,j] = (ϕ[i,j] + ϕ[i+1,j] + ϕ[i,j+1] + ϕ[i+1,j+1])/4 +
            (dxϕ[i,j]-dxϕ[i+1,j]+dxϕ[i,j+1]-dxϕ[i+1,j+1])*dx/16 +
            (dyϕ[i,j]-dyϕ[i,j+1]+dyϕ[i+1,j]-dyϕ[i+1,j+1])*dy/16 -
            dt/2*(hamiltonian_ls(V_stag[i,j], (ϕ_mid[i+1,j]-ϕ_mid[i,j])/dx, (ϕ_mid[i+1,j+1]-ϕ_mid[i+1,j])/dy) +
            hamiltonian_ls(V_stag[i,j], (ϕ_mid[i+1,j+1]-ϕ_mid[i,j+1])/dx, (ϕ_mid[i,j+1]-ϕ_mid[i,j])/dy))
        end
    end
    return ϕ_stag
end

"Minmod function for flux-limiter"
function minmod(a, b, c)
    if (a < 0) && (b < 0) && (c < 0)
        return max(a, b, c)
    elseif (a > 0) && (b > 0) && (c > 0)
        return min(a, b, c)
    else
        return 0.0
    end
end

"Evaluate Hamiltonian at one grid point"
function hamiltonian_ls(v, ϕx, ϕy)
    return v*sqrt(ϕx^2 + ϕy^2)
end


module DiffEqsSolver

export EulerDES, RK2DES, RK4DES, EulerCromerDES, EulerMidPiontDES, VerletDES, VelVerDES

function EulerDES(; ẋ::Function, x₀::Vector{T}, t₀::T, t₁::T, h::T) where {T<:AbstractFloat}
    tcoll = collect(t₀:h:t₁)
    xcoll = [x₀]
    x = x₀
    for t ∈ tcoll[2:end]
        x += h * ẋ(t, x)
        push!(xcoll, x)
    end
    return tcoll, xcoll
end

function UnstableDES(; ẋ::Function, x₀::Vector{T}, t₀::T, t₁::T, h::T) where {T<:AbstractFloat}
    tcoll = collect(t₀:h:t₁)
    xcoll = [x₀, x₀ + h * ẋ(t₀, x₀)]
    for t ∈ tcoll[3:end]
        xₙ₊₁ = 2 * h * ẋ(t, xcoll[end]) + xcoll[end-1]
        push!(xcoll, xₙ₊₁)
    end
    return tcoll, xcoll
end

function EulerYaghoubDES(; ẋ::Function, x₀::Vector{T}, t₀::T, t₁::T, h::T) where {T<:AbstractFloat}
    tcoll = collect(t₀:h:t₁)
    xcoll = [x₀]
    x = x₀
    ẋval = ẋ(t₀, x₀)
    for t ∈ tcoll[2:end]
        x += h * ẋval
        ẋval = ẋ(t, x)
        push!(xcoll, x)
    end
    return tcoll, xcoll
end

function RK2DES(; ẋ::Function, x₀::Vector{T}, t₀::T, t₁::T, h::T) where {T<:AbstractFloat}
    tcoll = collect(t₀:h:t₁)
    xcoll = [x₀]
    x = x₀
    for t ∈ tcoll[2:end]
        K₁ = ẋ(t, x) * h
        K₂ = h * ẋ(t + h / 2, x + (K₁ * h) / 2)
        x += K₂
        push!(xcoll, x)
    end
    return tcoll, xcoll
end

function RK4DES(; ẋ::Function, x₀::Vector{T}, t₀::T, t₁::T, h::T) where {T<:AbstractFloat}
    tcoll = collect(t₀:h:t₁)
    xcoll = [x₀]
    x = x₀
    for t ∈ tcoll[2:end]
        K₁ = ẋ(t, x)
        K₂ = ẋ(t + h / 2, x .+ h / 2 * K₁)
        K₃ = ẋ(t + h / 2, x .+ h / 2 * K₂)
        K₄ = ẋ(t + h, x .+ h * K₃)
        x += (K₁ + 2 * K₂ + 2 * K₃ + K₄) * h / 6
        push!(xcoll, x)
    end
    return tcoll, xcoll
end

function EulerCromerDES(; ẍ::Function, ẋ₀::Vector{T}, x₀::Vector{T}, t₀::T, t₁::T, h::T) where {T<:AbstractFloat}
    tcoll = collect(t₀:h:t₁)
    xcoll = [x₀]
    ẋcoll = [ẋ₀]
    x = x₀
    ẋ = ẋ₀
    for t ∈ tcoll[2:end]
        ẋ += h * ẍ(t, x, ẋ)
        x += h * ẋ
        push!(xcoll, x)
        push!(ẋcoll, ẋ)
    end

    return tcoll, xcoll, ẋcoll
end

function EulerMidPiontDES(; ẍ::Function, ẋ₀::Vector{T}, x₀::Vector{T}, t₀::T, t₁::T, h::T) where {T<:AbstractFloat}
    tcoll = collect(t₀:h:t₁)
    xcoll = [x₀]
    ẋcoll = [ẋ₀ - (ẍ(t₀, x₀, ẋ₀) * h / 2)]
    x = x₀
    ẋ = ẋ₀ - (ẍ(t₀, x₀, ẋ₀) * h / 2)

    for t ∈ tcoll[2:end]
        ẋ += h * ẍ(t, x, ẋ)
        x += h * ẋ

        push!(xcoll, x)
        push!(ẋcoll, ẋ)
    end

    return tcoll, xcoll, ẋcoll
end

function VerletDES(; ẍ::Function, ẋ₀::Vector{T}, x₀::Vector{T}, t₀::T, t₁::T, h::T) where {T<:AbstractFloat}
    tcoll = collect(t₀:h:t₁)
    xcoll = [x₀, x₀ + ẋ₀ * h + ẍ(t₀, x₀, ẋ₀) * (h^2) / 2]
    ẋcoll = [ẋ₀, ẋ₀ + ẍ(t₀, x₀, ẋ₀) * h]

    for i ∈ 3:length(tcoll)
        xᵢ = 2 * xcoll[i-1] - xcoll[i-2] + ẍ(tcoll[i-1], xcoll[i-1], ẋcoll[i-1]) * (h^2)
        push!(xcoll, xᵢ)
        ẋᵢ = (xcoll[i] - xcoll[i-1]) / h
        push!(ẋcoll, ẋᵢ)
    end

    return tcoll, xcoll, ẋcoll
end

function VelVerDES(; ẍ::Function, ẋ₀::Vector{T}, x₀::Vector{T}, t₀::T, t₁::T, h::T) where {T<:AbstractFloat}
    tcoll = collect(t₀:h:t₁)
    xcoll = [x₀]
    ẋcoll = [ẋ₀]
    x = x₀
    ẋ = ẋ₀

    for t ∈ tcoll[2:end]
        ẍₙ = ẍ(t, x, ẋ)
        x += ẋ * h + ẍₙ * (h^2) / 2
        ẍₙ₊₁ = ẍ(t + h, x, ẋ)
        ẋ += (ẍₙ + ẍₙ₊₁) * h / 2
        push!(xcoll, x)
        push!(ẋcoll, ẋ)
    end

    return tcoll, xcoll, ẋcoll
end

end


using Test
# using Revise
using CutCellDG
include("../useful_routines.jl")

l,m = 1.,2.
stiffness = CutCellDG.plane_strain_voigt_hooke_matrix(l,m)
teststiffness = [5. 1. 0.
                 1. 5. 0.
                 0. 0. 2.]
@test allapprox(teststiffness,stiffness)

stiffness = CutCellDG.HookeStiffness(l,m,l,m)
@test allapprox(teststiffness,stiffness[-1])
@test allapprox(teststiffness,stiffness[+1])

l1,m1 = 1.,2.
l2,m2 = 3.,4.
stiffness = CutCellDG.HookeStiffness(l1,m1,l2,m2)
teststiffness2 = [11.  3.  0.
                   3.  11. 0.
                   0.   0. 4.]
teststiffness1 = [5. 1. 0.
                  1. 5. 0.
                  0. 0. 2.]
@test allapprox(teststiffness1,stiffness[1])
@test allapprox(teststiffness2,stiffness[-1])

@test allapprox((l1,m1),CutCellDG.lame_coefficients(stiffness,1))
@test allapprox((l2,m2),CutCellDG.lame_coefficients(stiffness,-1))