# Bite Size Bayes

Copyright 2020 Allen B. Downey

License: [Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0)](https://creativecommons.org/licenses/by-nc-sa/4.0/)

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

## Fixing GPS

The Global Positioning System (GPS) is widely used satellite-based navigation system.  If you have a GPS receiver, you can use it to estimate your position on earth in three dimensions: latitude, longitude, and altitude.

Although GPS is accurate enough for applications like getting driving directions, is is subject to errors from several sources, [described in this Wikipedia article](https://en.wikipedia.org/wiki/Error_analysis_for_the_Global_Positioning_System).

For purposes of this problem, I'll assume that the distribution of errors in the horizontal directions is a normal distribution with mean 0 and standard deviation 5 m.  I'll also assume that errors in the $x$ and $y$ directions are independent.

In other words, if you stand on a survey marker placed by the [National Geodetic Survey (NGS)](https://en.wikipedia.org/wiki/U.S._National_Geodetic_Survey), so that your position is known with high accuracy, and take a single reading from a GPS receiver, the difference between the reading and the actual position would be well-modeled by a normal distribution.

SciPy provides an object called `norm` we can use to represent a normal distribution with a given mean and standard deviation.

from scipy.stats import norm

error_dist = norm(0, 5)

`norm` provides a function called `pdf` that compute the Probability Density Function (PDF) of the distribution.

xs = np.linspace(-15, 15)
ps = error_dist.pdf(xs)

error_series = pd.Series(ps, index=xs)

error_series.plot()
plt.xlabel('Error (m)')
plt.ylabel('Probability density')
plt.title('Error distribution');

def outer_product(s1, s2):
    """Compute the outer product of two Series.

    First Series goes down the rows;
    second goes across the columns.

    s1: Series
    s2: Series

    return: DataFrame
    """
    a = np.multiply.outer(s1.to_numpy(), s2.to_numpy())
    return pd.DataFrame(a, index=s1.index, columns=s2.index)

Here's what the distributions of width look like.

joint_dist = outer_product(error_series, error_series)

def plot_contour(joint):
    """Plot a joint distribution.

    joint: DataFrame representing a joint PMF
    """
    plt.contour(joint.index, joint.index, joint)
    plt.ylabel('y position in m')
    plt.xlabel('x position in m')

plot_contour(joint_dist)

## Inferring position

Now suppose I am standing in a field with a survey marker at the center.  I'll define the position of the marker to be the origin, where `x` and `y` positions are `0`, and I'll define east to be the positive `x` axis and north to be the positive `y` axis.

I don't know my actual location, but based on a single GPS reading, I compute that I am 10 m east and 20 m north of the marker.

Based on this reading, what is the posterior distribution of my actual location?

Well, it depends on how I got there.

To get started, let's assume that I am equally likely to be at any location in a large, square field.  In that case, `x` and `y` are independent and their distributions are uniform.

def make_pmf(xs, ps, **options):
    """Make a Series that represents a PMF.

    xs: sequence of values
    ps: sequence of probabilities
    options: keyword arguments passed to Series constructor

    returns: Pandas Series
    """
    pmf = pd.Series(ps, index=xs, **options)
    pmf /= pmf.sum()
    return pmf

xs = np.linspace(-50, 50, 101)
prior_x = make_pmf(xs, 1)

ys = np.linspace(-50, 50, 101)
prior_y = make_pmf(ys, 1)

I'll group the data by species and compute summary statistics.

To compute the likelihood of the `x` data, we can ask, "If my actual position were `x`, what is the probability my GPS reading would be 10 m?"

To answer that question, we can compute the hypothetical error for each value of `x`:

error_x = 10 - xs

Then we can use `error_dist` to compute the probability of the hypothetical errors.

likelihood = error_dist.pdf(error_x)

And the posterior distribution.

posterior_x = prior_x * likelihood
posterior_x /= posterior_x.sum()

prior_x.plot(label='prior')
posterior_x.plot(label='posterior')

plt.xlabel('Position (m)')
plt.ylabel('Probability')
plt.title('Prior distribution of position');

error_y = 20 - ys

likelihood = error_dist.pdf(error_y)

posterior_y = prior_y * likelihood
posterior_y /= posterior_y.sum()

joint_posterior1 = outer_product(posterior_y, posterior_x)
plot_contour(joint_posterior1)

This result might not come as a surprise to you.  With a uniform prior, the posterior is just a normalized version of the likelihood function.

Since the likelihood function is the PDF of a normal distribution, the posterior is a normal distribution with mean at the measured location, (10, 20), and the same standard deviation as the error distribution.

So we could have figured that out in our heads.  But now that we have done it the hard way, we are ready to solve harder problems.

## Parachuting in

Suppose I parachute into the field and try to land on the marker, and suppose my actualy landing position is well modeled by a normal distribution, along both axes, with mean 0 and standard deviation 15 m.

We can construct the marginal distributions for my positions along the `x` and `y` axes, like this:

landing_dist = norm(0, 15)

xs = np.linspace(-50, 50, 101)
ps = landing_dist.pdf(xs)
prior_x = make_pmf(xs, ps)

ys = np.linspace(-50, 50, 101)
ps = landing_dist.pdf(ys)
prior_y = make_pmf(ys, ps)

Assuming that my errors along the two axes are independent, we can use `outer_product` to construct the joint distribution for my landing spot.

joint_posterior = outer_product(prior_y, prior_x)

plot_contour(joint_posterior)

After landing, suppose I take a single GPS reading that indicates I am 10 m east and 20 m north of the marker.

I can use the GPS reading to update the distribution of my `x` coordinate, exactly as in the previous problem.

error_x = 10 - xs

likelihood = error_dist.pdf(error_x)

posterior_x = prior_x * likelihood
posterior_x /= posterior_x.sum()

Here's what the prior and posterior distributions look like for my `x` coordinate:

prior_x.plot(label='prior')
posterior_x.plot(label='posterior')

plt.xlabel('Position (m)')
plt.ylabel('Probability')
plt.title('Prior distribution of position')
plt.legend();

To compute the posterior mean, I'll use this function, which we've seen in a previous notebook.

def pmf_mean(pmf):
    """Compute the mean of a PMF.

    pmf: Series representing a PMF

    return: float
    """
    return np.sum(pmf.index * pmf)

The posterior mean is 9, a little less than the GPS reading, because in the prior distribution, values closer to 0 are more likely.

pmf_mean(posterior_x)

**Exercise:**  Update the `y` coordinate with the GPS read (20 m north) and compute the posterior mean.  We expect it to be a bit less than 20 m.

# Solution

error_y = 20 - ys

likelihood = error_dist.pdf(error_y)

posterior_y = prior_y * likelihood
posterior_y /= posterior_y.sum()

# Solution

pmf_mean(posterior_y)

In this version of the problem, the `x` and `y` coordinates are independent, so we can use `outer_product` to construct the joint posterior.

joint_posterior2 = outer_product(posterior_y, posterior_x)
plot_contour(joint_posterior2)

If you are not convinced that the coordinates are independent, just because I say say, you could update the joint prior distribution rather than the marginal distributions.  In the next example, we'll see how.

## Running away

Suppose I start out standing on the marker, spin until I am facing in a random direction, and then walk about 30 m, where my margin of error is 3 m.

So my distance from the origin is well modeled by a normal distribution with mean 30 and standard deviation 3.

Before I get a GPS reading, what is the prior distribution of my position?

In this example, `x` and `y` are not independent.  For example, if I know that `x` is near 30 m, that implies that `y` is near 0.  And if I know `y` is near 0, that implies that `x` is close to 30 m or -30 m.

To construct the prior distribution, I'll start with a joint distribution for `x` and `y` that is uniform and independent.  Then I'll do a Bayesian update with the data that I am 15 from the origin.

Here are the uniform marginal priors we've used before.

xs = np.linspace(-50, 50, 101)
prior_x = make_pmf(xs, 1)

ys = np.linspace(-50, 50, 101)
prior_y = make_pmf(ys, 1)

And here is the uniform joint prior.

joint_prior = outer_product(prior_x, prior_y)

To compute the distances from the center, I'll use the `outer` version of the NumPy function `hypot`, which computes the hypotenuse of a triangle with the given sides.

See how useful outer function are?

distances = np.hypot.outer(xs, ys)
distances.shape

Now, to compute a likelihood function, we can compute `error`, which is the difference between each hypothetical distance and the data, which is 30 m.

error = distances - 30
error.shape

The distribution of these errors is normal with standard deviation 3 m.

error_dist = norm(0, 3)

To get the likelihoods, we evaluate the normal PDF for each hypothetical error.

likelihood = error_dist.pdf(error)
likelihood.shape

As usual, the posterior distribution is the normalize product of the prior and the likelihood.

joint_posterior3 = joint_prior * likelihood
joint_posterior3 /= joint_posterior3.to_numpy().sum()

And here's what the posterior joint distribution looks like.

plot_contour(joint_posterior3)

plt.title('Joint distribution');

Remember that we did all that to compute the prior before we get a GPS reading.

Now suppose we take a reading and it indicates that we are 10 m east and 20 m north of the marker (as in the previous versions).

We'll use the posterior we just computed as the prior for the next update.

donut_prior = joint_posterior3

To compute the likelihoods, I'm going to cheat a little.

Remember that in the first version of the problem we started with a uniform prior and did an update with the same data.  The posterior we computed is just the normalized likelihood of the data, so we can use it as the likelihood for this update.

likelihood = joint_posterior1

As usual, the posterior is the normalized product of the prior and the likelihood.

donut_posterior = donut_prior * likelihood
donut_posterior /= donut_posterior.to_numpy().sum()

And here's what it looks like.

plot_contour(donut_posterior)

plt.title('Joint posterior distribution');

It reminds me of [The Bean](https://en.wikipedia.org/wiki/Cloud_Gate).

## Marginal posterior distributions

From the joint posterior distribution we can extract the marginal distributions for the coordinates.

def marginal(joint, axis):
    """Compute a marginal distribution.

    axis=0 returns the marginal distribution of the second variable
    axis=1 returns the marginal distribution of the first variable

    joint: DataFrame representing a joint PMF
    axis: int axis to sum along

    returns: Series representing a marginal PMF
    """
    return joint.sum(axis=axis)

donut_posterior_x = marginal(donut_posterior, 0)
donut_posterior_y = marginal(donut_posterior, 1)

Here's what they look like.

donut_posterior_x.plot(label='x')
donut_posterior_y.plot(label='y')

plt.xlabel('Position (m)')
plt.ylabel('Probability')
plt.title('Marginal posterior distributions')
plt.legend();

And here are the posterior means.

mean_x = pmf_mean(donut_posterior_x)
mean_x

mean_y = pmf_mean(donut_posterior_y)
mean_y

Here's the distance from the marker to the location of the posterior means:

np.hypot(mean_x, mean_y)

This distance represents a compromise between two sources of information:

* I started with the prior belief that I was 30 m from the marker, but

* My GPS reading suggests that I am only about 22 m from the marker.

np.hypot(10, 20)

The posterior mean distance is close to 30 m than 22 m because the margin of error in the prior, 3 m, is less than the margin of error of GPS, which is 5 m in each direction.

np.hypot(5, 5)

In this case, Bayes's theorem tells us that we should give more weight to out prior belief about the distance.

Actually, the posterior mean distance we just computed is not quite right because we computed the means of the marginal distributions independently.

To compute the posterior mean correctly, we have to compute the weighted sum of the hypothetical distances, using the posterior probabilities as weights.

qs = distances
ps = donut_posterior.values

np.sum(qs * ps)

## Summary


import numpy as np

# cross validation test
def multi_target_rel_l2_error(y_true,y_pred):
	if y_true.ndim == 2 and y_true.ndim == 2:
	    error_array =  np.linalg.norm(y_true - y_pred,axis=1)/np.linalg.norm(y_true,axis=1)
	    return error_array.mean()
	else:
		return np.linalg.norm(y_true - y_pred)/np.linalg.norm(y_true)

def plot_samples(Y,x=None,ax=None,q=[.005,.5,.995],show_mean=True,label=None,color=None,xlabel='',ylabel='',fs=10,alpha=.2,rotation=None,labelpad=10):
	assert Y.ndim == 2, "Y must contain samples and have 2 dimensions."
	nsamples, dim = Y.shape
	mu = np.mean(Y,axis=0)
	var = np.var(Y,axis=0)
	Q = np.quantile(Y,q=q,axis=0)
	if ax is None:
		fig,ax = mpl.subplots()
	if x is None:
		x = list(np.arange(dim))
	q_label = r"{}".format(label + ' 99% Q')
	# mu_label = r"{}".format(label + ' median')
	mu_label = None
	if q is None:
		ax.fill_between(x, mu - np.sqrt(var), mu + np.sqrt(var),color=color, alpha=alpha,label=label)
		if show_mean:
			ax.plot(x, mu, '--',color=color, lw=1.5, alpha=0.5,label=mu_label)
	else:
		ax.fill_between(x, Q[0], Q[2],color=color, alpha=0.2,label=q_label)
		if show_mean:
			ax.plot(x, Q[1], '--',color=color, lw=1.5, alpha=0.5,label=mu_label)

	ax.set_xlabel(r"{}".format(xlabel),fontsize=fs)
	ax.set_ylabel(r"{}".format(ylabel),fontsize=fs,rotation=rotation,labelpad=labelpad)
	return ax


import numpy as np
import cv2
import os

from fourier_transform import *

def img_FFT(img):
    if len(img.shape) == 2 or len(img.shape) == 3:
        return FFT_2D(img)
    else:
        raise ValueError("Please input a gray or RGB image!")

def img_FFT_inverse(img):
    if len(img.shape) == 2 or len(img.shape) == 3:
        return inverseFFT_2D(img)
    else:
        raise ValueError("Please input a gray or RGB image!")

def findpower2(num):
    """find the nearest number that is the power of 2"""
    if num & (num-1) == 0:
        return num

    bin_num = bin(num)
    origin_bin_num = str(bin_num)[2:]
    near_power2 = pow(10, len(origin_bin_num))
    near_power2 = "0b" + str(near_power2)
    near_power2 = int(near_power2, base=2)

    return near_power2

def image_padding(img):
    """ padding the image size to power of 2, for fft computation requirement"""
    if len(img.shape) == 2 or len(img.shape) == 3:
        h, w = img.shape[0], img.shape[1]

        h_pad = findpower2(h)-h
        w_pad = findpower2(w)-w

        img = np.pad(img, pad_width=((0, h_pad), (0, w_pad), (0, 0)), mode='constant')

        return img

def image_fft(img_path, result_folder_path="result/"):
    """ read, padding, fft, cut to origin size and save """
    data_root, img_name = os.path.split(img_path)

    if img_name[-3:] != "png" and img_name[-3:] != "tif":
        return 0

    if not os.path.exists(result_folder_path):
        os.mkdir(result_folder_path)

    img_origin = cv2.imread(img_path)
    img = image_padding(img_origin)

    img_fft = img_FFT(img)

    if len(img_origin) == 2:
        img_fft = img_fft[:img_origin.shape[0], :img_origin.shape[1]]
    else:
        img_fft = img_fft[:img_origin.shape[0], :img_origin.shape[1], :]

    img_fft_complex = img_fft.copy()

    # save real value for human seeing
    img_fft = np.real(img_fft)
    name, _ = img_name.split(".")
    save_img_name = result_folder_path + name + "_fft.png"
    cv2.imwrite(save_img_name, img_fft)

    return img_fft_complex

def image_fft_inverse(img_fft_complex, img_path, result_folder_path="result/"):
    """ inverse the read fft_img, cut to origin size and save """
    if not os.path.exists(result_folder_path):
        os.mkdir(result_folder_path)
    _, img_name = os.path.split(img_path)

    img_fft = image_padding(img_fft_complex)

    img_origin = img_FFT_inverse(img_fft)
    img_ifft = np.real(img_origin)

    name, _ = img_name.split(".")
    save_img_name = result_folder_path + name + "_inverse.png"

    if len(img_origin) == 2:
        img_ifft = img_ifft[:img_fft_complex.shape[0], :img_fft_complex.shape[1]]
    else:
        img_ifft = img_ifft[:img_fft_complex.shape[0], :img_fft_complex.shape[1], :]

    cv2.imwrite(save_img_name, img_ifft)

    return img_origin

if __name__ == '__main__':
    x = np.mgrid[:8, :8][0]
    # x = np.mgrid[:4, :4][0]
    print(x)
    print("-------------------")
    # print(np.allclose(FFT(x), np.fft.fft(x)))
    # print(np.allclose(FFT_2D(x), np.fft.fft2(x)))
    # print(FFT_2D(x))
    print(inverseFFT_2D(x))
    # print(inverseDFT_2D(x))

    print("-------------------")
    print(np.fft.ifft2(x))
    print("-------------------")
    # print(np.fft.fft(x))

    # print(np.allclose(np.fft.fft(x), np.fft.fft2(x)))


from numpy import random

class DistributionGenerator:
    """
    Generate random data within a given distribution
    """

    def __init__(self, num_samples, mean=0, sd=1, interval=1, n=1, p=0.5):
        """

        :param num_samples: Int The number of samples to be returned
        :param mean: The Mean value for a normal distribution
        :param sd: The Standard Deviation for a normal distribution
        :param interval: The Interval for a poisson distribution
        :param n: The number of samples for a binomial distribution
        :param p: The probability for a binomial distribution
        """
        self.num_samples = num_samples
        self.mean = mean
        self.sd = sd
        self.interval = interval
        self.n = n
        self.p = p

        self.data = None

    def normal(self):
        """ Produce an array of values in a normal distribution

        :return: ndarray
        """
        self.data = random.normal(size=self.num_samples, loc=self.mean, scale=self.sd)
        return self.data

    def poisson(self):
        """ Produce an array of values in a poisson distribution

        :return: ndarray
        """
        self.data = random.poisson(size=self.num_samples, lam=self.interval)
        return self.data

    def binomial(self):
        """ Produce an array of values in a binomial distribution

        :return: ndarray
        """
        self.data = random.binomial(size=self.num_samples, n=self.n, p=self.p)
        return self.data

    def summarise(self):
        """ Print summary statistics about the last dataset created

        :return:
        """
        if self.data is None:
            print("There is no current data")
        else:
            print("Min: " + str(self.data.min()))
            print("Max: " + str(self.data.max()))
            print("Mean: " + str(self.data.mean()))
            print("Standard deviation: " + str(self.data.std()))


# logisitc regression classifier for the XOR problem.
#
# the notes for this class can be found at:
# https://deeplearningcourses.com/c/data-science-logistic-regression-in-python
# https://www.udemy.com/data-science-logistic-regression-in-python

import numpy as np
import matplotlib.pyplot as plt

N = 4
D = 2

# XOR
X = np.array([
    [0, 0],
    [0, 1],
    [1, 0],
    [1, 1],
])
T = np.array([0, 1, 1, 0])

# add a column of ones
# ones = np.array([[1]*N]).T
ones = np.ones((N, 1))

# add a column of xy = x*y
xy = np.matrix(X[:,0] * X[:,1]).T
Xb = np.array(np.concatenate((ones, xy, X), axis=1))

# randomly initialize the weights
w = np.random.randn(D + 2)

# calculate the model output
z = Xb.dot(w)

def sigmoid(z):
    return 1/(1 + np.exp(-z))

Y = sigmoid(z)

# calculate the cross-entropy error
def cross_entropy(T, Y):
    E = 0
    for i in xrange(N):
        if T[i] == 1:
            E -= np.log(Y[i])
        else:
            E -= np.log(1 - Y[i])
    return E

# let's do gradient descent 100 times
learning_rate = 0.001
error = []
for i in xrange(10000):
    e = cross_entropy(T, Y)
    error.append(e)
    if i % 100 == 0:
        print e

    # gradient descent weight udpate with regularization
    # w += learning_rate * ( np.dot((T - Y).T, Xb) - 0.01*w )
    w += learning_rate * ( Xb.T.dot(T - Y) - 0.01*w )

    # recalculate Y
    Y = sigmoid(Xb.dot(w))

plt.plot(error)
plt.title("Cross-entropy per iteration")
plt.show()

print "Final w:", w
print "Final classification rate:", 1 - np.abs(T - np.round(Y)).sum() / N


# IMPORTS
import numpy as np
import scipy.stats as stats
import matplotlib.pyplot as plt
import matplotlib.style as style
from IPython.core.display import HTML

# PLOTTING CONFIG
%matplotlib inline
style.use('fivethirtyeight')
plt.rcParams["figure.figsize"] = (14, 7)
HTML("""
<style>
.output_png {
    display: table-cell;
    text-align: center;
    vertical-align: center;
}
</style>
""")
plt.figure(dpi=100)

# PDF k = 1
plt.plot(np.linspace(0, 15, 500),
         stats.chi2.pdf(np.linspace(0, 15, 500), df=1),
        )
plt.fill_between(np.linspace(0, 15, 500),
                 stats.chi2.pdf(np.linspace(0, 15, 500), df=1),
                 alpha=.15,
                )

# PDF k = 3
plt.plot(np.linspace(0, 15, 100),
         stats.chi2.pdf(np.linspace(0, 15, 100), df=3),
        )
plt.fill_between(np.linspace(0, 15, 100),
                 stats.chi2.pdf(np.linspace(0, 15, 100), df=3),
                 alpha=.15,
                )

# PDF k = 6
plt.plot(np.linspace(0, 15, 100),
         stats.chi2.pdf(np.linspace(0, 15, 100), df=6),
        )
plt.fill_between(np.linspace(0, 15, 100),
                 stats.chi2.pdf(np.linspace(0, 15, 100), df=6),
                 alpha=.15,
                )

# LEGEND
plt.text(x=.5, y=.7, s="$ k = 1$", rotation=-65, alpha=.75, weight="bold", color="#008fd5")
plt.text(x=1.5, y=.35, s="$ k = 3$", alpha=.75, weight="bold", color="#fc4f30")
plt.text(x=5, y=.2, s="$ k = 6$", alpha=.75, weight="bold", color="#e5ae38")

# TICKS
plt.tick_params(axis = 'both', which = 'major', labelsize = 18)
plt.axhline(y = 0, color = 'black', linewidth = 1.3, alpha = .7)

# TITLE, SUBTITLE & FOOTER
plt.text(x = -1.5, y = 2.8, s = "Chi-Squared Distribution - $ k $",
               fontsize = 26, weight = 'bold', alpha = .75)
plt.text(x = -1.5, y = 2.5,
         s = 'Depicted below are three Chi-Squared distributed random variables with varying $ k $. As one can\nsee the parameter $k$ smoothens the distribution and softens the skewness.',
         fontsize = 19, alpha = .85)
plt.text(x = -1.5,y = -0.4,
         s = '   ©Joshua Görner                                                                                github.com/jgoerner   ',
         fontsize = 14, color = '#f0f0f0', backgroundcolor = 'grey');


"""
Utility functions

Author: Shota Horii <sh.sinker@gmail.com>

References:

"""

import operator as op
from functools import reduce
import math
import numpy as np

def ncr(n, r):
    """
    Calculates nCr in efficient manner.
    This function is not my original code, but copied from below url.
    https://stackoverflow.com/questions/4941753/is-there-a-math-ncr-function-in-python

    Parameters
    ----------
    n, r: int
    """
    r = min(r, n-r)
    numer = reduce(op.mul, range(n, n-r, -1), 1)
    denom = reduce(op.mul, range(1, r+1), 1)
    return numer / denom

def flatten(l):
    """
    flatten a nested list.
    https://stackoverflow.com/questions/952914/how-to-make-a-flat-list-out-of-list-of-lists

    Parameters
    ----------
    l: array-like
    """
    return [item for sublist in l for item in sublist]

def split_array(a, n):
    """
    Split an array into n chunks.
    https://stackoverflow.com/questions/2130016/splitting-a-list-into-n-parts-of-approximately-equal-length

    Parameters
    ----------
    a: array-like
    n: int
        number of chunks
    """
    k, m = divmod(len(a), n)
    return (a[i * k + min(i, m):(i + 1) * k + min(i + 1, m)] for i in range(n))

def supremum_eigen(X):
    """
    Estimates approximate supremum of eigen values of a square matrix
    by Gershgorin circle theorem.
    Ref(in JP): https://qiita.com/fujiisoup/items/e7f703fc57e2dfc441ad

    Parameters
    ----------
    X: np.ndarray (d,d)
    """
    return np.max(np.sum(np.abs(X), axis=0))

def sigmoid(x):
    return 1 / (1 + np.exp(-x))

def softmax(x):
    if x.ndim == 1:
        x = np.array([x])

    # Avoid overflow
    x = x - np.array([x.max(axis=1)]).T

    exp_x = np.exp(x)
    sum_exp_x = np.sum(exp_x,axis=1)[np.newaxis].T
    return exp_x / sum_exp_x


# _*_ coding: utf-8 _*_
# source activate audio

##### Package necessary #####
import os
import numpy as np
import librosa
import matplotlib.pyplot as plt
import librosa.display
import IPython.display as ipd
from scipy.io import wavfile
from scipy.fftpack import fft
from scipy import signal
#############################

##### Package optional #####

# For 3D spectrogram plot # Attention: only work within jupyter notebook
# import plotly.graph_objs as go
# import plotly.offline as py
#############################

def example():
	filename = librosa.util.example_audio_file()
	y, sr = librosa.load(filename)
	return y, sr

def example2(T = 10.0, sr = 22050, freq_list = [600,400], coeffs_amplitude= None):
	"""
	Combination of two sinusoidal signals.

	T : length of sound

	"""
	final_x = 0
	t = np.linspace(0, T, int(T*sr), endpoint=False) # time variable
	if coeffs_amplitude is None:
		coeffs_amplitude = np.ones(len(freq_list))

	for index,freq in enumerate(freq_list):
		x = coeffs_amplitude[index]*np.sin(2*np.pi*freq*t) # pure sine wave at 'freq' Hz
		final_x += x

	return final_x, sr

def example3(T = 10.0, sr = 22050, freq_list = [600,400]):
	"""
	Inverse signal of example2().

	Demonstration of destructive wave: example2()+example3() ~= 0

	"""
	assert len(freq_list) == 2, "The sum of two sinusoidal functions."
	freq = freq_list[0]
	freq2 = freq_list[1]
	t = np.linspace(0, T, int(T*sr), endpoint=False) # time variable
	z = -2.0*np.sin(np.pi*(freq+freq2)*t)*np.cos(np.pi*(freq-freq2)*t)

	return z, sr

def plot_wave(y, sr):
	plt.figure(figsize=(12, 4))
	librosa.display.waveplot(y, sr=sr)
	plt.show()

def plot_spectrogram(y, sr, mode = "librosa"):

	if mode == "librosa":
		X = librosa.stft(y)
		Xdb = librosa.amplitude_to_db(X)
		plt.figure(figsize=(12, 5))
		librosa.display.specshow(Xdb, sr=sr, x_axis='time', y_axis='hz')
		plt.show()
	elif mode == "scipy":
		freqs, times, spectrogram = log_specgram(y, sr)
		plt.figure(figsize=(12, 5))
		librosa.display.specshow(spectrogram.T, sr=sr, x_axis='time', y_axis='hz')
		plt.show()
	else:
		raise ValueError("mode should be 'librosa' or 'scipy'.")

def plot_spectrogram_3D(y, sr):
	# TODO
	# X = librosa.stft(y)
	# Xdb = librosa.amplitude_to_db(X)
	freqs, times, spectrogram = log_specgram(y, sr)
	data = [go.Surface(z=spectrogram.T)]
	layout = go.Layout(
		title='Specgtrogram of "yes" in 3d',
		scene = dict(
		yaxis = dict(title='Frequencies', range=freqs),
		xaxis = dict(title='Time', range=times),
		zaxis = dict(title='Log amplitude'),
		),)

	fig = go.Figure(data=data, layout=layout)
	py.iplot(fig)
	py.show()

def custom_fft(y, fs):
	T = 1.0 / fs
	N = y.shape[0]
	yf = fft(y)
	xf = np.linspace(0.0, 1.0/(2.0*T), N//2)
	vals = 2.0/N * np.abs(yf[0:N//2])  # FFT is simmetrical, so we take just the first half
	# FFT is also complex, to we take just the real part (abs)
	# (Lu: J'ai pas très bien compris cela.)
	return xf, vals

def plot_fourier(y, sr):
	xf, vals = custom_fft(y, sr)
	plt.figure(figsize=(12, 4))
	plt.title('FFT of recording sampled with ' + str(sr) + ' Hz')
	plt.plot(xf, vals)
	plt.xlabel('Frequency')
	plt.grid()
	plt.show()

def plot_fourier_librosa(y, sr):
	# assert 1==2, "Not finished yey !"
	T = 1.0 / sr
	N = y.shape[0]
	xf = np.linspace(0.0, 1.0/(2.0*T), N//2)
	spectre = np.abs(librosa.stft(y))
	# import ipdb; ipdb.set_trace()

	vals = 2.0/N * spectre[0:N//2]

	plt.figure(figsize=(12, 4))
	plt.title('FFT of recording sampled with ' + str(sr) + ' Hz')
	plt.plot(xf, vals)
	plt.xlabel('Frequency')
	plt.grid()
	plt.show()
	return xf, vals

	# return spectre

def plot_Mel_spectrogram(y, sr):
	S = librosa.feature.melspectrogram(y, sr=sr, n_mels=128)
	log_S = librosa.power_to_db(S, ref=np.max)

	plt.figure(figsize=(12, 4))
	librosa.display.specshow(log_S, sr=sr, x_axis='time', y_axis='mel')
	plt.title('Mel power spectrogram ')
	plt.colorbar(format='%+02.0f dB')
	plt.tight_layout()
	plt.show()
	return

def plot_MFCC_coeffs(y, sr):
	S = librosa.feature.melspectrogram(y, sr=sr, n_mels=128) # shape = (128, ???)
	log_S = librosa.power_to_db(S, ref=np.max)

	mfcc = librosa.feature.mfcc(S=log_S, n_mfcc=13) # shape = (13, ???)

	delta2_mfcc = librosa.feature.delta(mfcc, order=2) # shape = (13, ???)

	# import ipdb; ipdb.set_trace()
	plt.figure(figsize=(12, 4))
	librosa.display.specshow(delta2_mfcc)
	plt.ylabel('MFCC coeffs')
	plt.xlabel('Time')
	plt.title('MFCC')
	plt.colorbar()
	plt.tight_layout()
	plt.show()
	return

def log_specgram(audio, sample_rate, window_size=20,
				 step_size=10, eps=1e-10):
	print("Warning... I don't understand this yet.")
	print("Warning... I don't understand this yet.")
	print("Warning... I don't understand this yet.")

	nperseg = int(round(window_size * sample_rate / 1e3))
	noverlap = int(round(step_size * sample_rate / 1e3))
	freqs, times, spec = signal.spectrogram(audio,
									fs=sample_rate,
									window='hann',
									nperseg=nperseg,
									noverlap=noverlap,
									detrend=False)
	return freqs, times, np.log(spec.T.astype(np.float32) + eps)

def load_audio_file(filename):
	# y, sr = librosa.load(filename, sr=22050, offset=0.0, duration=None)
	sr, y = wavfile.read(filename)
	# y, sr = librosa.core.load(filename, sr=22050, offset=15.0, duration=5.0)
	return y, sr


"""
* Assignment: Numpy Broadcasting Arithmetic
* Complexity: easy
* Lines of code: 4 lines
* Time: 3 min

English:
    1. Define `a: np.ndarray` with square root of each element in `A`
    2. Define `b: np.ndarray` with square root of each element in `B`
    3. Define `c: np.ndarray` with second power (square) of each element in `C`
    4. Add elements from `a` to `b`
    5. Multiply the result by `c`
    6. Run doctests - all must succeed

Polish:
    1. Zdefiniuj `a: np.ndarray` z pierwiastkiem kwadratowym każdego elementu `A`
    2. Zdefiniuj `b: np.ndarray` z pierwiastkiem kwadratowym każdego elementu `B`
    3. Zdefiniu `c: np.ndarray` z drugą potęgą (kwadratem) każdego z elementu w `C`
    4. Dodaj elementy z `a` do `b`
    5. Przemnóż wynik przez `c`
    6. Uruchom doctesty - wszystkie muszą się powieść

Tests:
    >>> import sys; sys.tracebacklimit = 0

    >>> assert result is not Ellipsis, \
    'Assign result to variable: `result`'
    >>> assert type(result) is np.ndarray, \
    'Variable `result` has invalid type, expected: np.ndarray'

    >>> result
    array([[ 1.41421356,  2.73205081],
           [45.254834  ,  0.        ]])
"""

import numpy as np

A = np.array([[0, 1], [2, 3]], float)
B = np.array([2, 3], float)
C = np.array([[1, 1], [4, 0]], float)

# np.ndarray: square root of each element in `A` use np.pow()
a = ...

# np.ndarray: square root of each element in `B` use `**` operator
b = ...

# np.ndarray: second power (square) of each element in `C` use `**` operator
c = ...

# np.ndarray: Add elements from `a` to `b` and then multiply by `c`
#             Remember about the operator precedence
result = ...


"""
This script demonstrates the implementation of the ReLU function.

It's a kind of activation function defined as the positive part of its argument in the
context of neural network.
The function takes a vector of K real numbers as input and then argmax(x, 0).
After through ReLU, the element of the vector always 0 or real number.

Script inspired from its corresponding Wikipedia article
https://en.wikipedia.org/wiki/Rectifier_(neural_networks)
"""
from typing import List

import numpy as np

def relu(vector: List[float]):
    """
    Implements the relu function

    Parameters:
        vector (np.array,list,tuple): A  numpy array of shape (1,n)
        consisting of real values or a similar list,tuple

    Returns:
        relu_vec (np.array): The input numpy array, after applying
        relu.

    >>> vec = np.array([-1, 0, 5])
    >>> relu(vec)
    array([0, 0, 5])
    """

    # compare two arrays and then return element-wise maxima.
    return np.maximum(0, vector)

if __name__ == "__main__":
    print(np.array(relu([-1, 0, 5])))  # --> [0, 0, 5]


import numpy as np
import matplotlib.pyplot as plt
from loadData import *
from scipy.special import expit

# logistic regression
def one_hot(data):
    shape = (data.shape[0], N_class)
    one_hot = np.zeros(shape)
    rows = np.arange(data.shape[0])
    one_hot[rows, data[:,0]] = 1
    return one_hot

def cross_entropy(p, y):
    m = y.shape[0]
    log_likelihood = -y * np.log(p+epsilon)
    loss = np.sum(log_likelihood) / m
    return loss

def predict(X, W, t = None):
    # X_new: Nsample x (d+1)
    # W: (d+1) x K

    Z = np.dot(X, W)
    y = expit(Z)
    t_hat = one_hot(t)
    loss = cross_entropy(y, t_hat)

    predicted = np.argmax(y, 1).reshape(X.shape[0],1)
    correct = (predicted == t).sum()
    acc = correct / X.shape[0]

    return y, t_hat, loss, acc

def train(X_train, y_train, X_val, y_val, alpha):
    N_train = X_train.shape[0]
    N_val   = X_val.shape[0]

    # init weight
    w = np.zeros([X_train.shape[1], 1])

    # init return val
    epoch_best = 0
    acc_best = 0
    W_best = None
    track_loss_train = []
    track_acc_val = []

    for epoch in range(MaxIter):
        loss_this_epoch = 0
        for b in range(int(np.ceil(N_train/batch_size)) ):
            X_batch = X_train[b*batch_size : (b+1)*batch_size]
            y_batch = y_train[b*batch_size : (b+1)*batch_size]

            y, t_hat, loss, _ = predict(X_batch, w, y_batch)

            loss_this_epoch += loss

            # calculate gradient
            gradient = np.dot(X_batch.transpose(), (y-t_hat))/batch_size + decay * w
            decay_alpha = np.power(0.99, epoch) * alpha # learning rate decay
            w = w - decay_alpha * gradient
        avg_loss_this_epoch = loss_this_epoch / (N_train/batch_size)
        _, _, loss, acc = predict(X_val, w, y_val)
        track_loss_train.append(avg_loss_this_epoch)
        track_acc_val.append(acc)
        if acc > acc_best:
            acc_best = acc
            epoch_best = epoch
            W_best = w

    return epoch_best, acc_best, W_best, track_loss_train, track_acc_val

#Main code starts here
X_train, t_train, X_val, t_val, X_test, t_test = loadSpamData()

print(X_train.shape, t_train.shape, X_val.shape, t_val.shape, X_test.shape, t_test.shape)

N_class = 2

batch_size   = 10    # batch size
MaxIter = 1000        # Maximum iteration
decay = 1e-2
epsilon = 1e-12

acc_val = []
alpha_set = np.arange(1e-4, 1e-3, 1e-4)
acc_best = 0
W_best = None
epoch_best = None
loss_curve = None
acc_curve = None
best_alpha = None
for alpha in alpha_set:
    print(alpha)
    ep, acc, w, track_loss_train, track_acc_val = train(X_train, t_train, X_val, t_val, alpha)
    acc_val.append(acc)
    if acc > acc_best:
        best_alpha = alpha
        acc_best = acc
        W_best = w
        epoch_best = ep
        loss_curve = track_loss_train
        acc_curve = track_acc_val

print('Best alpha:',best_alpha,'\nBest epoch:', epoch_best, 'with acc in val:',acc_best)
# plt.figure(1)
# plt.plot(loss_curve)
# plt.xlabel("epochs")
# plt.ylabel("training loss")
# plt.figure(2)
# plt.plot(acc_curve)
# plt.xlabel("epochs")
# plt.ylabel("validation acc")
# plt.show()


import numpy as np

# We define our Train Loss fuction (1/2)*np.transpose(X*w-y)*(X*w-y)
def train_loss(d_train, w):
    # We create the matrix X and the matriz y
    X = []
    y = []
    for data in d_train:
        X.append([1] + data[0]), y.append(data[1])
    X = np.array(X)
    y = np.array(y)
    w = np.array(w)
    return (1/2)*np.dot(np.transpose((np.dot(X, w) - y)), (np.dot(X, w) - y))

# We define our Train Loss Gradient fuction (ϕ(x) = x)
def train_loss_gradient(x, y, w):
    return 2 * np.dot((np.dot(w, x) - y), x)

# We define our Gradient descent algorithm
def stochastic_gradient_descent(d_train, w, n, iterations):
    print("Running stochastic gradient descent algorithm")
    iteration = 1
    j = 0
    while iteration <= iterations:
        y = d_train[j][1]
        # We add 1 as the first component in x
        x = np.array([1] + d_train[j][0])
        # We calculate the Train Loss Gradient
        train_loss_gradient_ = train_loss_gradient(x, y, w)
        # We update the vector w
        w = w - n * train_loss_gradient_
        # We calculate the Train Loss
        train_loss_ = train_loss(d_train, w)
        # We print the data
        print(f'In the iteration {iteration}, we get '
              f'w = {w} and TrainLoss(x, y, w) = {train_loss_}')
        iteration += 1
        if j == len(d_train) - 1:
            j = 0
        else:
            j += 1

    print('Algoritm finished')

# We define our DTrain, our weight vector, our n and our iteration number
d_train = [[[0, 0], 0], [[-1, 0], 1 / 2], [[1, 0], 1 / 2], [[0, 1], 1], [[1, 1], 3 / 2]]
w = np.array([0, 1, 1])
n = 0.5
iterations = 5

# We run our stochastic gradient descent algorithm
stochastic_gradient_descent(d_train, w, n, iterations)


# plot Rastrigin function
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import cm
from mpl_toolkits.mplot3d import Axes3D

X = np.linspace(-5.12, 5.12, 100)
Y = np.linspace(-5.12, 5.12, 100)
X, Y = np.meshgrid(X, Y)

Z =  2 * 10 + (X**2 - 10 * np.cos(2 * np.pi * X)) + (Y**2 - 10 * np.cos(2 * np.pi * Y))

# plot 3D
fig = plt.figure()
ax = fig.gca(projection='3d')
CS = ax.plot_surface(X, Y, Z, rstride=1, cstride=1, cmap=cm.nipy_spectral, linewidth=0.08, antialiased=True)
plt.xlabel("x")
plt.ylabel("y")
plt.title("Rastrigin Function", fontsize='small')
plt.grid(color='silver', linestyle='-', linewidth=0.2)
# add colorbar
cbar = fig.colorbar(CS)
cbar.ax.set_ylabel('Objective Value')
plt.tight_layout()
plt.show()

# plot the contour
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
ax.contour(X, Y, Z, 100, cmap=cm.nipy_spectral)

plt.xlabel("x")
plt.ylabel("y")
plt.title("Rastrigin Function", fontsize='small')
plt.grid(color='silver', linestyle='-', linewidth=0.2)
plt.tight_layout()
plt.show()


import numpy as np

weights = {
    'excellent': 3,
    'good': 2,
    'fair': 1
}

data = np.array([
    'excellent',
    'fair',
    'good',
    'excellent'
])

# Parsing the categories to weights
data = np.fromiter(map(lambda x: weights[x], data), dtype=np.int)

# Creating a zero matrix for the Proximity measure matrix
dmatrix  = np.zeros((data.shape[0], data.shape[0]))

# Get Pmeasure matrix dimensions
x, y = dmatrix.shape

# Max range in the array
M = data.max()

# Looping a lower triangular matrix
for j in range(0, y):
    for i in range(j+1, y):
        # d(i, j) = Z(i) - Z(j)
        # Z(i) = (r(i) - 1) / M - 1
        # Where Z(i) is the norm of "i" and r(i) is the weight of "i"
        # Carrying out the sum of fractions and simplifying,
        # the following equation remains
        dmatrix[i,j] = np.abs(data[i] - data[j])/(M - 1)

print(dmatrix)


# Start initialization
import numpy as np
import pylab as p

# Make an x,z grid
grid = np.linspace(-6,6,200)
x, z = np.meshgrid(grid,grid)
mu=4*np.pi  # dipole strength (in z-direction)
# and the 2D cross section of the potential (scalar field)
phi = mu * z / (4 * np.pi * (x**2+z**2)**(3/2))
# End initialization

# Start contour plot
from matplotlib import colors
p.figure(figsize=(6,6))
heights = np.linspace(-0.6,0.6,7)
CS = p.contour(x,z,phi,heights,colors='k')
p.clabel(CS, inline=1, fontsize=10,fmt='%3.1f')
p.xlabel(r'$x$')
p.ylabel(r'$z$')
# End contour plot

# Start fieldline plot
Fx = mu / (4 * np.pi * (x**2+z**2)**(5/2)) * 2*x*z
Fz = mu / (4 * np.pi * (x**2+z**2)**(5/2)) * (z**2-x**2)
p.streamplot(x,z,Fx,Fz)
# End fieldline plot
CS = p.contour(x,z,phi,heights,colors='k')
p.clabel(CS, inline=1, fontsize=10,fmt='%3.1f')
p.savefig('dipole_fieldplot.png')

#p.show()


from sympy import Range

def is_composite(a: int, d: int, n: int, s: int) -> bool:
    if pow(a, d, n) == 1:
        return False
    for i in Range(s):
        if pow(a, 2 ** i * d, n) == n - 1:
            return False
    # n  is definitely composite
    return True

def is_prime(number: int, rounds: int = 128) -> int:
    if number in primes:
        return True
    if any((number % p) == 0 for p in primes) or number in (0, 1):
        return False
    d, s = number - 1, 0
    while not d % 2:
        d, s = d >> 1, s + 1
    if number < 1373653:
        return not any(is_composite(a, d, number, s) for a in (2, 3))
    if number < 25326001:
        return not any(is_composite(a, d, number, s) for a in (2, 3, 5))
    if number < 118670087467:
        if number == 3215031751:
            return False
        return not any(is_composite(a, d, number, s) for a in (2, 3, 5, 7))
    if number < 2152302898747:
        return not any(is_composite(a, d, number, s) for a in (2, 3, 5, 7, 11))
    if number < 3474749660383:
        return not any(is_composite(a, d, number, s) for a in (2, 3, 5, 7, 11, 13))
    if number < 341550071728321:
        return not any(
            is_composite(a, d, number, s) for a in (2, 3, 5, 7, 11, 13, 17)
            )
    # otherwise
    return not any(is_composite(a, d, number, s) for a in primes[:rounds])

primes = [2, 3]
primes += [x for x in range(5, 1000, 2) if is_prime(x)]

def main() -> None:
    number = int(input("Enter the number to check for primality: "))
    if is_prime(number):
        print("The number", number, "is prime.")
    else:
        print("The number", number, "is composite.")

if __name__ == "__main__":
    main()


from abc import ABC, abstractmethod
import numpy as np

__all__ = [
    "ActivationFn",
    "Linear",
    "Sigmoid",
    "Softmax",
    "TanH",
    "ReLU",
    "LeakyReLU",
]

class ActivationFn(ABC):
    @abstractmethod
    def __call__(self, x):
        pass

    @abstractmethod
    def gradient(self, x):
        pass

# ==================================================================================================== #
# ==================================================================================================== #

class Linear(ActivationFn):
    """Computing the Linear (Identity) function and its derivative.

        g(x) = x
    """

    def __init__(self):
        pass

    def __call__(self, x):
        return x

    def gradient(self, x):
        return 1

# ==================================================================================================== #

class Sigmoid(ActivationFn):
    """Computing the Sigmoid function and its derivative.

        g(x) = 1 / (1 + e^-x)
    """

    def __init__(self):
        pass

    def __call__(self, x):
        return np.vectorize(lambda k: 1.0/(1 + np.exp(-k)))(x)

    def gradient(self, x):
        a = self(x)
        return a * (1 - a)

# ==================================================================================================== #

class Softmax(ActivationFn):
    """Computing the numerical stable Softmax function and its derivative.

        g(vec) = e^vec[j] / sum(e^vec[k])
    """

    def __init__(self, axis=-1):
        self.axis = axis

    def __call__(self, x):
        exps = np.exp(x - np.max(x, axis=self.axis, keepdims=True))
        return exps / np.sum(exps, axis=self.axis, keepdims=True)

    def gradient(self, x):
        a = self(x)
        return a * (1 - a)

# ==================================================================================================== #

class TanH(ActivationFn):
    """Computing the TanH function and its derivative.

        g(x) = (e^x - e^-x) / (e^x + e^-x)
    """

    def __init__(self):
        pass

    def __call__(self, x):
        return np.tanh(x)

    def gradient(self, x):
        return 1 - self(x)**2

# ==================================================================================================== #

class ReLU(ActivationFn):
    """Computing the ReLU function and its derivative.

        g(x) = max(x, 0)
    """

    def __init__(self):
        pass

    def __call__(self, x):
        return np.vectorize(max)(0, x)

    def gradient(self, x):
        return (np.array(x) >= 0).astype(int)

# ==================================================================================================== #

class LeakyReLU(ActivationFn):
    """Computing the ReLU function and its derivative.

        g(x) = max(x, eps*x)
    """

    def __init__(self, eps=0.01):
        self.eps = eps

    def __call__(self, x):
        return np.vectorize(max)(self.eps*x, x)

    def gradient(self, x):
        grad = (np.array(x) >= 0).astype(int)
        grad[grad == 0] = self.eps
        return grad

# ==================================================================================================== #


from math import sqrt, exp

from scipy.stats import norm

from opfu.bsm import N, bsm_price, d1, d2

def N_d(x):
    return norm.pdf(x)

def delta(S0, K, r=0.01, sigma=0.1, T=1, ds=0, is_call=True):
    if ds == 0:
        # the theortical result
        if is_call:
            return N(d1(S0, K, r, sigma, T))
        else:
            return N(d1(S0, K, r, sigma, T)) - 1
    # approach
    p = bsm_price(S0 + ds / 2, K, r, sigma, T, is_call)
    m = bsm_price(S0 - ds / 2, K, r, sigma, T, is_call)
    return (p - m) / ds

def gamma(S0, K, r=0.01, sigma=0.1, T=1, ds=0, is_call=True):
    if ds == 0:
        return N_d(d1(S0, K, r, sigma, T)) / (S0 * sigma * sqrt(T))
    # approach
    p = delta(S0 + ds / 2, K, r, sigma, T, ds, is_call)
    m = delta(S0 - ds / 2, K, r, sigma, T, ds, is_call)
    return (p - m) / ds

def theta(S0, K, r=0.01, sigma=0.1, T=1, dt=0, is_call=True):
    if dt == 0:
        return -S0 * N_d(d1(S0, K, r, sigma, T)) * sigma / (2 * sqrt(T)) - r * K * exp(-r * T) * N(
            d2(S0, K, r, sigma, T)) \
            if is_call else \
            -S0 * N_d(d1(S0, K, r, sigma, T)) * sigma / (2 * sqrt(T)) + r * K * exp(-r * T) * N(-d2(S0, K, r, sigma, T))
    # approach
    p = bsm_price(S0, K, r, sigma, T + dt / 2, is_call)
    m = bsm_price(S0, K, r, sigma, T - dt / 2, is_call)
    return -(p - m) / dt

def vega(S0, K, r=0.01, sigma=0.1, T=1, dsigma=0, is_call=True):
    if dsigma == 0:
        return S0 * sqrt(T) * N_d(d1(S0, K, r, sigma, T))
    # approach
    p = bsm_price(S0, K, r, sigma + dsigma / 2, T, is_call)
    m = bsm_price(S0, K, r, sigma - dsigma / 2, T, is_call)
    return (p - m) / dsigma

def rho(S0, K, r=0.01, sigma=0.1, T=1, dr=0, is_call=True):
    if dr == 0:
        return K * T * exp(-r * T) * N(d2(S0, K, r, sigma, T)) if is_call else -K * T * exp(-r * T) * N(
            -d2(S0, K, r, sigma, T))
    # approach
    p = bsm_price(S0, K, r + dr / 2, sigma, T, is_call)
    m = bsm_price(S0, K, r - dr / 2, sigma, T, is_call)
    return (p - m) / dr


import numpy as np

# multipool_matrix is a function defining multipool matrices based on the Standard Transversal Design if
# the pool size is a prime number n
# the population size is N = n**2
# the multiplicity k is a number between 1 and n+1

def multipool_matrix(n,k):
	# check constraint k <= n+1
	if k > n+1:
		print "Error! k must not be larger than n+1."
		return
	# check constraint n prime if k > 3
	if k > 3:
		for i in range(2,n):
			if (n % i) == 0:
 				print("Error! If k > 3 then n must be a prime number.")
 				return

	A = np.zeros((n*k,n**2))

	for i in range(n):
		for j in range(n):
			A[i][i*n+j] = 1
	for l in range(1,k):
		for i in range(n):
			for j in range(n):
				A[l*n+i][n*j+((i+(l-1)*j)%n)] = 1
	return A


# -*- coding: utf-8 -*-
import functools

import numpy as np
import matplotlib.pyplot as plt

def meshgrid3d(X,Y):
    """
        input:
            X : (m,d) tensor
            Y : (n,d) tensor
        returns:
            Xd : (m x n x d) tensor
            Yd : (m x n x d) tensor
    """
    Xd = X.reshape(X.shape[0],1,X.shape[1])
    Yd = Y.reshape(1,Y.shape[0],Y.shape[1])
    Xd = np.tile(Xd,[1,Y.shape[0],1])
    Yd = np.tile(Yd,[X.shape[0],1,1])
    return Xd,Yd

def kernel_matrix(X,Y,k,symmetrize=False):
    Xd,Yd = meshgrid3d(X,Y)
    K = k(Xd,Yd,keepdims=False)
    if symmetrize:
        K = 0.5*(K + K.transpose(1,0))
    return K

def rbf_kernel_f(x,y,l,theta,keepdims=True):
    return theta*np.exp(-0.5*np.sum((x-y)**2/(l**2),axis=-1,
                                          keepdims=keepdims))

#%% Priors
theta = 1.0
l = 1.0
X = np.linspace(-2,2)
kfunc = functools.partial(rbf_kernel_f,l=l,theta=theta)
K = kernel_matrix(X.reshape(-1,1),X.reshape(-1,1),kfunc)
ncurves = 200
y = np.random.multivariate_normal(np.zeros(len(X)),K,ncurves)
for yy in y:
    plt.plot(X.flatten(),yy,'b--',alpha=0.2)
plt.fill_between(X,- 2*np.sqrt(np.diag(K)),
                    2*np.sqrt(np.diag(K)),
                   color='red',alpha=0.2)

plt.savefig("/home/danilo/Danilo/Dissertação/Tex/figs/gprex1a")
plt.show()
#%% Posterior
def f(x):
    return np.sin(np.pi*x)
xpoints = np.linspace(-0.1,1.5,5)
ypoints = f(xpoints)
Kxx = kernel_matrix(xpoints.reshape(-1,1),xpoints.reshape(-1,1),kfunc)
Kxx_inv = np.linalg.inv(Kxx)
Kplot = kernel_matrix(X.reshape(-1,1),xpoints.reshape(-1,1),kfunc)
Kplot2 = kernel_matrix(X.reshape(-1,1),X.reshape(-1,1),kfunc)
pos_mean = Kplot@Kxx_inv@(ypoints.reshape(-1,1))
pos_cov = Kplot2 - Kplot@Kxx_inv@Kplot.T
y2 = np.random.multivariate_normal(pos_mean.flatten(),pos_cov,ncurves)
plt.fill_between(X,pos_mean.flatten() - 2*np.sqrt(np.diag(pos_cov)),
                   pos_mean.flatten() + 2*np.sqrt(np.diag(pos_cov)),
                   color='red',alpha=0.2)
#plt.plot(X,pos_mean,'red',alpha=0.5)
for yy2 in y2:
    plt.plot(X.flatten(),yy2,'b--',alpha=0.2)
plt.plot(X,f(X),'black',linestyle='--',alpha=0.8)
plt.plot(xpoints,ypoints,'go')
plt.savefig("../tex/figs/gprex1b")
plt.show()


import numpy as np
import matplotlib.pyplot as plt

def dbscan(D, eps,MinPts):
  '''the inputs has the same name as shown in the question
  @input
  D: a data set containing n objects,
  Eps: the radius parameter, and
  MinPts: the neighborhood density threshold.'''
  # STEP 1
  clusters = [0]*len(D) # this will contains the output cluster values of each datapoint
  # we will initialize with all zeros (means  mark all objects as unvisited)
  # the value -1 represent it is a noise

  currentClusterID = 0
  # STEP 2
  for P in range(0, len(D)):
    # STEP 3 : randomly select an unvisited object p;
    if not (clusters[P] == 0):
      continue
    # STEP : 4 :  Find all of P's neighboring points.
    neighbouringPoints = findNeighbouringPoints(D, P, eps)
    # STEP : 5 : If the Eps-neighborhood of p0 has at least MinPts points, add those points to N;
    if len(neighbouringPoints) < MinPts:
      clusters[P] = -1
    else:
      currentClusterID = currentClusterID + 1 #new cluster
      newCluster(D, clusters, P, neighbouringPoints, currentClusterID, eps, MinPts)

  return clusters

def findNeighbouringPoints(D, P, eps):
  neighbors = []
  # for all the points find the distance
  for point in range(0, len(D)):
    # If the distance is below the threshold, add it to the neighbors list.
    if np.linalg.norm(D[P] - D[point]) < eps:
      neighbors.append(point)
  return neighbors

def newCluster(D, clusters, P, neighbouringPoints, currentClusterID, eps, MinPts):
  # Assign the cluster label to the seed point.
  clusters[P] = currentClusterID
  i = 0
  while i < len(neighbouringPoints):
    # Get the next point from the queue.
    Pn = neighbouringPoints[i]
    if clusters[Pn] == -1:
      clusters[Pn] = currentClusterID
    elif clusters[Pn] == 0:
      # Add Pn to cluster C (Assign cluster label C).
      clusters[Pn] = currentClusterID

      # Find all the neighbors of Pn
      PnNeighborPts = findNeighbouringPoints(D, Pn, eps)

      # If Pn has at least MinPts neighbors, it's a branch point. Add all of its neighbors to the queue
      if len(PnNeighborPts) >= MinPts:
        neighbouringPoints = neighbouringPoints + PnNeighborPts
    i += 1

########################
# TO TEST THE CODE
#######################
# we use this to crease a sample datasets
from sklearn.datasets.samples_generator import make_blobs
from sklearn.preprocessing import StandardScaler

# Create three gaussian blobs to use as our clustering data.
centers = [[1, 1], [-1, -1], [1, -1]]
X, labels_true = make_blobs(n_samples=750, centers=centers, cluster_std=0.4, random_state=0) # create 750 values
X = StandardScaler().fit_transform(X)
theClustersForEachPoints = dbscan(X, eps=0.3, MinPts=10)
# I will display the first 10 cluster values
print(theClustersForEachPoints[:10])

centers = [[1, 1], [-1, -1], [1, -1]]
X, labels_true = make_blobs(n_samples=750, centers=centers, cluster_std=0.4, random_state=0) # create 750 values
X = StandardScaler().fit_transform(X)
theClustersForEachPoints = dbscan(X, eps=0.3, MinPts=10)

print(theClustersForEachPoints)

plt.plot(theClustersForEachPoints,'ro')
plt.xlabel('Deaths')
plt.ylabel('Confirmed')
plt.title('Clusters')
plt.grid(True)
plt.show()


import pytest
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from rolldecayestimators.measure import fft,fft_omega0
from numpy.testing import assert_almost_equal

omega0=2.0
T0 = 2*np.pi/omega0
A = 1.0

@pytest.fixture
def signal():
    t = np.arange(0,10.5*T0,0.01)
    y = A*np.sin(omega0*t)
    s = pd.Series(data=y, index=t)
    yield s

@pytest.fixture
def signal2():
    t = np.arange(0,5.0*T0,0.01)
    y = A*np.sin(omega0*t)
    s = pd.Series(data=y, index=t)
    yield s

def test_fft(signal):

    fig, ax = plt.subplots()
    signal.plot(ax=ax)
    plt.show()

    frequencies, dft = fft(signal)

    index = np.argmax(dft)
    natural_frequency = frequencies[index]
    dft_predict = dft[index]
    omega0_predict = 2 * np.pi * natural_frequency

    fig,ax = plt.subplots()
    ax.plot(2*np.pi*frequencies,dft, '.-', label='dft')
    ax.plot(omega0_predict, dft_predict, 'ro', label='omega_0=%f'%omega0_predict)
    ax.set_xlim(0,omega0_predict+5)
    ax.legend()

    plt.show()

#@pytest.mark.skip('Fix this one...')
def test_fft_omega0(signal):

    frequencies, dft = fft(signal)
    omega0_predict = fft_omega0(frequencies=frequencies, dft=dft)
    assert_almost_equal(omega0_predict, omega0, decimal=2)

def test_fft_omega0_even(signal2):

    frequencies, dft = fft(signal2)
    omega0_predict = fft_omega0(frequencies=frequencies, dft=dft)
    assert_almost_equal(omega0_predict, omega0, decimal=2)

def signal_generator(t,omega0):
    y = A*np.sin(omega0*t)
    s = pd.Series(data=y, index=t)
    return s

def test_fft_omega0_even_many():

    omega0s = np.linspace(0.1,6,10)

    for omega0_ in omega0s:
        T0_ = 2 * np.pi / omega0_
        t = np.arange(0, 10.5*T0_, 0.01)

        s = signal_generator(t=t, omega0=omega0_)

        frequencies, dft = fft(s)
        omega0_predict = fft_omega0(frequencies=frequencies, dft=dft)
        assert_almost_equal(omega0_predict, omega0_, decimal=2)

#@pytest.mark.skip('Fix this one...')
def test_fft_omega0_many():

    omega0s = np.linspace(0.1,6,10)

    for omega0_ in omega0s:
        T0_ = 2 * np.pi / omega0_
        t = np.arange(0, 10.5*T0_, 0.01)

        s = signal_generator(t=t, omega0=omega0_)

        frequencies, dft = fft(s)
        omega0_predict = fft_omega0(frequencies=frequencies, dft=dft)
        assert_almost_equal(omega0_predict, omega0_, decimal=2)


import shapely.geometry as geom
import shapely.affinity as affin

import numpy as np

from matplotlib import pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

from descartes import PolygonPatch

import math

from mayavi import mlab

# How many radians the inner gear rotates each step.
THETA_STEP = .2
# How thick the tubes of the 3d spirograph are.
TUBE_THICKNESS = 3

def show_2d_plot(poly, outer_radius, gear_name):
    fig = plt.figure()
    ax = fig.add_subplot(111)
    ax.add_patch(PolygonPatch(poly))

    axis_size = round(outer_radius)
    ax.axis((-axis_size, axis_size, -axis_size, axis_size))
    ax.set_title(gear_name)
    ax.set_aspect(1)

    plt.show()

def show_3d_plot(points, outer_radius, gear_name, display_3d, save_to_obj):
    black = (0,0,0)
    white = (1,1,1)
    scene = mlab.figure(figure=gear_name, bgcolor=white,fgcolor=black)
    xs = [p[0] for p in points]
    ys = [p[1] for p in points]
    zs = [p[2] for p in points]
    mlab.plot3d(xs,ys,zs, color=black, tube_radius=TUBE_THICKNESS)
    if save_to_obj:
        mlab.savefig(gear_name + ".obj")
    if display_3d:
        mlab.show()

def simulate(inner_teeth, outer_teeth, pencil_percent, offset_rad, display_2d=False, display_3d=False, save_to_obj=False, gear_name="Simulation"):
    points = []
    total_rads = 0
    pencil_radius = pencil_percent * inner_teeth
    r_diff = outer_teeth - inner_teeth

    lcm = np.lcm(inner_teeth, outer_teeth)
    rotations = lcm / inner_teeth

    while total_rads < rotations * 2 * math.pi:
        x = r_diff * math.cos(total_rads) + pencil_radius * math.cos(
            (r_diff / inner_teeth) * total_rads)
        y = r_diff * math.sin(total_rads) - pencil_radius * math.sin(
            (r_diff / inner_teeth) * total_rads)
        z = r_diff * math.cos(total_rads) - pencil_radius * math.cos(
            (r_diff / inner_teeth) * total_rads)
        points.append((x, y, z))
        total_rads += THETA_STEP

    line = geom.LineString(points).buffer(.01)
    line = affin.rotate(line, offset_rad, geom.Point(0,0), use_radians=True)

    should_handle_3d = display_3d or save_to_obj
    if display_2d:
        show_2d_plot(line, outer_teeth, gear_name)
    if should_handle_3d:
        show_3d_plot(points, outer_teeth, gear_name, display_3d, save_to_obj)

    return line


"""
equity.py

Created by Luca Camerani at 13/09/2020, University of Milano-Bicocca.
(l.camerani@campus.unimib.it)
All rights reserved.

This file is part of the EcoFin-Library (https://github.com/LucaCamerani/EcoFin-Library),
and is released under the "BSD Open Source License".
"""

import numpy as np

class Equity():
    def __init__(self, dataFrame, log=False, skipNaN=True):
        self.dataFrame = dataFrame
        self.log = log
        self.skipNaN = skipNaN

    def getReturns(self):
        if self.log:
            return self.dataFrame.pct_change()
        else:
            return self.getLogReturns()

    def getLogReturns(self):
        return np.log(1 + self.getReturns())

    def meanReturn(self, driver=None):
        # [!!!] driver
        return self.getReturns().mean(axis=0, skipna=self.skipNaN)

    def varReturn(self):
        return self.getReturns().var(axis=0, skipna=self.skipNaN)

    def stdReturn(self):
        return np.sqrt(self.varReturn())

    def covMatrix(self):
        return self.getReturns().cov()


from __future__ import annotations

import math

import numpy
import numpy as np

class UnivariateGaussian:
    """
    Class for univariate Gaussian Distribution Estimator
    """

    def __init__(self, biased_var: bool = False) -> UnivariateGaussian:
        """
        Estimator for univariate Gaussian mean and variance parameters

        Parameters
        ----------
        biased_var : bool, default=False
            Should fitted estimator of variance be a biased or unbiased estimator

        Attributes
        ----------
        fitted_ : bool
            Initialized as false indicating current estimator instance has not been fitted.
            To be set as True in `UnivariateGaussian.fit` function.

        mu_: float
            Estimated expectation initialized as None. To be set in `UnivariateGaussian.fit`
            function.

        var_: float
            Estimated variance initialized as None. To be set in `UnivariateGaussian.fit`
            function.
        """
        self.biased_ = biased_var
        self.fitted_, self.mu_, self.var_ = False, None, None

    def fit(self, X: np.ndarray) -> UnivariateGaussian:
        """
        Estimate Gaussian expectation and variance from given samples

        Parameters
        ----------
        X: ndarray of shape (n_samples, )
            Training data

        Returns
        -------
        self : returns an instance of self.

        Notes
        -----
        Sets `self.mu_`, `self.var_` attributes according to calculated estimation (where
        estimator is either biased or unbiased). Then sets `self.fitted_` attribute to `True`
        """
        self.mu_ = X.mean()
        if self.biased_:
            self.var_ = X.var(ddof=0)
        else:
            self.var_ = X.var(ddof=1)

        self.fitted_ = True
        return self

    def get_mean(self):
        return self.mu_

    def get_var(self):
        return self.var_

    def pdf(self, X: np.ndarray) -> np.ndarray:
        """
        Calculate PDF of observations under Gaussian model with fitted estimators

        Parameters
        ----------
        X: ndarray of shape (n_samples, )
            Samples to calculate PDF for

        Returns
        -------
        pdfs: ndarray of shape (n_samples, )
            Calculated values of given samples for PDF function of N(mu_, var_)

        Raises
        ------
        ValueError: In case function was called prior fitting the model
        """
        if not self.fitted_:
            raise ValueError("Estimator must first be fitted before calling `pdf` function")
        pdfs = []
        for x in X:
            pdfs.append(UnivariateGaussian._calculate_pdf(x, self.mu_, self.var_))
        return pdfs

    @staticmethod
    def log_likelihood(mu: float, sigma: float, X: np.ndarray) -> float:
        """
        Calculate the log-likelihood of the data under a specified Gaussian model

        Parameters
        ----------
        mu : float
            Expectation of Gaussian
        sigma : float
            Variance of Gaussian
        X : ndarray of shape (n_samples, )
            Samples to calculate log-likelihood with

        Returns
        -------
        log_likelihood: float
            log-likelihood calculated
        """
        var_sum = 0
        const = (-1.0) / (2 * sigma)
        for x in X:
            var_sum += pow(x - mu, 2)
        return (- len(X) / 2) * (numpy.log(2 * math.pi) + numpy.log(sigma)) + (const * var_sum)

    @staticmethod
    def _calculate_pdf(x, mu, var):
        return (1.0 / (var * math.sqrt(2 * math.pi))) * math.exp(-0.5 * pow((x - mu) / var, 2))

class MultivariateGaussian:
    """
    Class for multivariate Gaussian Distribution Estimator
    """

    def __init__(self):
        """
        Initialize an instance of multivariate Gaussian estimator

        Attributes
        ----------
        fitted_ : bool
            Initialized as false indicating current estimator instance has not been fitted.
            To be set as True in `MultivariateGaussian.fit` function.

        mu_: ndarray of shape (n_features,)
            Estimated expectation initialized as None. To be set in `MultivariateGaussian.fit`
            function.

        cov_: ndarray of shape (n_features, n_features)
            Estimated covariance initialized as None. To be set in `MultivariateGaussian.fit`
            function.
        """
        self.mu_, self.cov_ = None, None
        self.fitted_ = False

    def fit(self, X: np.ndarray) -> MultivariateGaussian:
        """
        Estimate Gaussian expectation and covariance from given samples

        Parameters
        ----------
        X: ndarray of shape (n_samples, n_features)
            Training data

        Returns
        -------
        self : returns an instance of self

        Notes
        -----
        Sets `self.mu_`, `self.cov_` attributes according to calculated estimation.
        Then sets `self.fitted_` attribute to `True`
        """
        self.mu_ = np.mean(X, 0)
        self.cov_ = np.cov(X.T)
        self.fitted_ = True
        return self

    def get_mean(self):
        return self.mu_

    def get_cov(self):
        return self.cov_

    def pdf(self, X: np.ndarray):
        """
        Calculate PDF of observations under Gaussian model with fitted estimators

        Parameters
        ----------
        X: ndarray of shape (n_samples, n_features)
            Samples to calculate PDF for

        Returns
        -------
        pdfs: ndarray of shape (n_samples, )
            Calculated values of given samples for PDF function of N(mu_, cov_)

        Raises
        ------
        ValueError: In case function was called prior fitting the model
        """
        if not self.fitted_:
            raise ValueError("Estimator must first be fitted before calling `pdf` function")
        pdfs = []
        for x in X:
            pdfs.append(MultivariateGaussian._calculate_pdf(x, self.mu_, self.cov_))
        return pdfs

    @staticmethod
    def _calculate_pdf(x, mu, cov):
        d = float(len(x))
        det = numpy.linalg.det(cov)
        if det == 0:
            raise ValueError("The covariance matrix has to be invertible")
        inverted = np.linalg.inv(cov)
        distance_x_mu = ((x - mu).T.dot(inverted)).dot(x - mu)
        return (1.0 / math.sqrt(pow(2 * math.pi, d) * det)) * math.exp(-0.5 * distance_x_mu)

    @staticmethod
    def _calculate_likelihood(x, cov_det, k, inverted):
        return -0.5 * (np.log(cov_det) + x.T @ inverted @ x + k * np.log(2 * math.pi))

    @staticmethod
    def log_likelihood(mu: np.ndarray, cov: np.ndarray, X: np.ndarray) -> float:
        """
        Calculate the log-likelihood of the data under a specified Gaussian model

        Parameters
        ----------
        mu : ndarray of shape (n_features,)
            Expectation of Gaussian
        cov : ndarray of shape (n_features, n_features)
            covariance matrix of Gaussian
        X : ndarray of shape (n_samples, n_features)
            Samples to calculate log-likelihood with

        Returns
        -------
        log_likelihood: float
            log-likelihood calculated over all input data and under given parameters of Gaussian
        """
        v = X - mu
        inverted = np.linalg.inv(cov)
        k = mu.shape[0]
        cov_det = np.linalg.det(cov)
        log_likelihood = np.apply_along_axis(
            lambda x: MultivariateGaussian._calculate_likelihood(x, cov_det, k, inverted), 1, v)
        return log_likelihood.sum()


#
#
# Regression and Classification programming exercises
#
#

#
#	In this exercise we will be taking a small data set and computing a linear function
#	that fits it, by hand.
#

#	the data set

import numpy as np

sleep = [5, 6, 7, 8, 10]
scores = [65, 51, 75, 75, 86]

def compute_regression(sleep, scores):

    #	First, compute the average amount of each list
    avg_sleep = np.mean(sleep)
    avg_scores = np.mean(scores)
    print avg_sleep, avg_scores

    #	Then normalize the lists by subtracting the mean value from each entry
    normalized_sleep = np.array(sleep) - avg_sleep
    normalized_scores = np.array(scores) - avg_scores
    print normalized_sleep
    print normalized_scores

    #	Compute the slope of the line by taking the sum over each student
    #	of the product of their normalized sleep times their normalized test score.
    #	Then divide this by the sum of squares of the normalized sleep times.

    slope = np.sum(normalized_sleep * normalized_scores) / np.sum(np.square(normalized_sleep))
    print slope

    #	Finally, We have a linear function of the form
    #	y - avg_y = slope * ( x - avg_x )
    #	Rewrite this function in the form
    #	y = m * x + b
    #	Then return the values m, b

    return slope, avg_scores - (avg_sleep * slope)

#
#	Polynomial Regression
#
#	In this exercise we will examine more complex models of test grades as a function of
#	sleep using numpy.polyfit to determine a good relationship and incorporating more data.
#
#
#   at the end, store the coefficients of the polynomial you found in coeffs
#

def polyRegression():
    sleep = [5,6,7,8,10,12,16]
    scores = [65,51,75,75,86,80,0]
    coeffs = np.polyfit(sleep, scores, 2)
    return coeffs

if __name__ == "__main__":
    m, b = compute_regression(sleep, scores)
    print "Your linear model is y={}*x+{}".format(m, b)
    print "Polynomial regression: ", polyRegression()


import numpy

def feature_normalize(x_data, y_data, num_examples):

    x_mean = numpy.mean(x_data, axis=0)
    x_std = numpy.std(x_data, axis=0)
    x_norm = (x_data - x_mean)/x_std

    y_mean = numpy.mean(y_data)
    y_std = numpy.std(y_data)
    y_norm = (y_data - y_mean)/y_std

    return x_norm, y_norm, x_mean, y_mean, x_std, y_std


# -*- coding: utf-8 -*-
"""
Gaussian Error Linear Unit (GELU)
This module is implementation of "GAUSSIAN ERROR LINEAR UNITS (GELUS)"
(https://arxiv.org/abs/1606.08415)
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import numpy as np
import keras.backend as K

def gelu(x, approximate='tanh'):
    """Gaussian Error Linear Unit.
    (implementation of https://arxiv.org/abs/1606.08415)

    # Arguments
        x: Input tensor.
        approximate: Approximation method of GELU.
            "tanh" or "sigmoid" (defaults to "tanh".)

    # Returns
        The gaussian error unit activation: x * Phi(x)
            where Phi(x) = P(X <= x), X ~ Normal(0, 1)
            (approximated by "tanh" or "sigmoid")
    """
    if not approximate in ('tanh', 'sigmoid'):
        raise ValueError('approximate must be in ("tanh", "sigmoid")')
    if approximate == 'tanh':
        return 0.5*x * (1 + K.tanh(np.sqrt(2/np.pi) * \
                                   (x + 0.044715*K.pow(x, 3))))
    else:
        return x * K.sigmoid(1.702*x)


#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Tue Jan 29 19:41:58 2019

@author: alankar
"""
import numpy as np

def func(z):
    z1, z2 = z[0], z[1]
    equations = []
    equations.append( z2**2*(1-z1) - z1**3 )
    equations.append( z1**2 + z2**2 -1 )
    return np.array(equations).T

def jac(z):
    z1, z2 = z[0], z[1]
    J = np.array( [
                  [-z2**2-3*z1**2,    2*z2*(1-z1)],
                  [ 2*z1,                2*z2] ])
    return J

tol = 1e-6

z_new = np.array([-1.6+0.j,0.8+0.j])
z_old = np.array([0.+0.j,0.+0.j])

while( np.abs(z_old[0]-z_new[0])>tol or np.abs(z_old[1]-z_new[1])>tol ):
    z_old = z_new
    equations = func(z_old)
    delz = np.linalg.solve(jac(z_old),-equations)
    z_new = z_old + delz.T

print('Roots are: ')
for i in range(len(z_new)):
    print('z%d = '%(i+1),end='')
    if (np.abs(z_new[i].real) >=tol): print(" %.2f"%z_new[i].real,end='')
    if((np.abs(z_new[i].real) >=tol) and (np.abs(z_new[i].imag) >=tol)): print(' + ',end='')
    if (np.abs(z_new[i].imag) >=tol): print("%.2f i"%z_new[i].imag)
    print('')

"""
Output

Roots are:
z1 =  -1.62
z2 = 1.27 i
"""


import math
import cmath
import numpy as np
import matplotlib.pyplot as plt
import datetime
import time

def main():

    sair = False
    while(not sair):

        menu = int(input("\n\nSelecione vizualizar testes de transformada(1) Testes de dados(2) ou Resolução do problema proposto (3): "))

        if menu == 1:

            menu_teste = int(input("\n\nSelecione teste a) (1), b) (2) ou c) (3): "))

            if menu_teste == 1:

                print("\n\nTeste a) f = [5,-1,3,1]",end="\n\n")
                f = [5,-1,3,1]

                print("Implementação ineficiente da Transformada de Fourier direta",end="\n")
                c1 = fft_direta(f)
                print_vector(c1)

                print("Implementação recursiva da FFT direta",end="\n")
                c = [0,0,0,0]
                fftrec(c,f,2,True)
                print_vector(np.divide(c,4))

                print("\n\nFazendo c = (2, 1/2 + i/2, 2, 1/2 − i/2)",end="\n\n")
                c = [complex(2,0),complex(0.5,0.5),complex(2,0),complex(0.5,-0.5)]

                print("Implementação ineficiente da Transformada de Fourier inversa",end="\n")
                f1 = fft_inversa(c)
                print_vector(f1)

                print("Implementação recursiva da FFT inversa",end="\n")
                f = [0,0,0,0]
                fftrec(f,c,2,False)
                print_vector(f)

            if menu_teste == 2:

                print("\n\nTeste b) f = [6, 2, 5, 2, 11, 2, 8, 8]",end="\n\n")
                f = [6, 2, 5, 2, 11, 2, 8, 8]

                print("Implementação ineficiente da Transformada de Fourier direta",end="\n")
                c1 = fft_direta(f)
                print_vector(c1)

                print("Implementação recursiva da FFT direta",end="\n")
                c = [0,0,0,0,0,0,0,0]
                fftrec(c,f,4,True)
                print_vector(np.divide(c,8))

                print("Implementação recursiva da FFT direta pelo numpy (rfft)",end="\n")
                print("Note que são mostrados apenas os coeficientes de frequencias positivas pois o input é real")
                c2 = np.fft.rfft(f)
                print_vector(np.divide(c2,8))

                print("Implementação ineficiente da Transformada de Fourier inversa",end="\n")
                f1 = fft_inversa(c1)
                print_vector(f1)

                print("Implementação recursiva da FFT inversa",end="\n")
                f3 = [0,0,0,0,0,0,0,0]
                fftrec(f3,np.divide(c,8),4,False)
                print_vector(f3)

                print("Implementação recursiva da FFT inversa pelo numpy (rfft)",end="\n")
                f2 = np.fft.irfft(c2)
                print_vector(f2)

            if menu_teste == 3:

                print("\n\nTeste b) F(x) = 10sin(x) + 7cos(30x) + 11sin(352x) − 8cos(711x) com xj = jπ/512, j = 0, 1023",end="\n\n")
                dois_n = 1024
                f = [ 0 for j in range(dois_n)]
                x = [j*math.pi/(dois_n/2) for j in range(dois_n)]
                f = [10*math.sin(x[j])+7*math.cos(30*x[j])+11*math.sin(352*x[j])-8*math.cos(711*x[j]) for j in range(dois_n)]

                print("Implementação ineficiente da Transformada de Fourier direta",end="\n")
                c1 = fft_direta(f)

                print("Implementação recursiva da FFT direta",end="\n")
                c = [0 for i in range(dois_n)]
                fftrec(c,f,int(dois_n/2),True)

                print("Implementação recursiva da FFT direta pelo numpy (rfft)",end="\n")
                print("Note que são mostrados apenas os coeficientes de frequencias positivas pois o input é real")
                c2 = np.fft.rfft(f)

                print("Implementação ineficiente da Transformada de Fourier inversa",end="\n")
                f1 = fft_inversa(c1)

                print("Implementação recursiva da FFT inversa",end="\n")
                f3 = [0 for i in range(dois_n)]
                fftrec(f3,np.divide(c,dois_n),int(dois_n/2),False)

                print("Implementação recursiva da FFT inversa pelo numpy (rfft)",end="\n")
                f2 = np.fft.irfft(c2)

                plt.figure(1)
                ax1 = plt.subplot(231)
                t = [j for j in range(int(dois_n))]
                plt.plot(t, np.absolute(c1), 'bo')
                ax1.set_title('TF Ineficiente')
                k = [j for j in range(dois_n)]
                plt.subplot(234)
                plt.plot(k, f1, 'r-')

                ax2 = plt.subplot(232)
                t = [j for j in range(int(dois_n))]
                plt.plot(t, np.absolute(np.divide(c,dois_n)), 'bo')
                ax2.set_title('FFT recursiva')
                k = [j for j in range(dois_n)]
                plt.subplot(235)
                plt.plot(k, f3, 'r-')

                ax3 = plt.subplot(233)
                t = [j for j in range(int(dois_n/2+1))]
                plt.plot(t, np.absolute(np.divide(c2,dois_n)), 'bo')
                ax3.set_title('FFT numpy')
                k = [j for j in range(dois_n)]
                plt.subplot(236)
                plt.plot(k, f2, 'r-')

                plt.show()

        if menu == 2:

            Alum = read_file("Alum.csv")
            nome = 'Aluminio'
            print("===============================")
            tipo_tf = int(input("Selecione transformada Lenta N2(0), recursiva implementada(1) ou numpy(2): "))
            tipo_filtro = int(input("Selecione filtro passa baixas(0) ou passa altas(1): "))
            K = int(input("Digite o indice K de corte do filtro: "))
            print("===============================")

            Alum_preco = [x[2] for x in Alum]

            if tipo_tf == 0:
                timedir,timeinv = plotComTF_Lenta(Alum_preco, nome, K, tipo_filtro)
            if tipo_tf == 1:
                timedir,timeinv = plotComTF_rec(Alum_preco, nome, K, tipo_filtro)
            if tipo_tf == 2:
                timedir,timeinv = plotComTF_numpy(Alum_preco, nome, K, tipo_filtro)

            print("===============================")
            print("\n\nTempo decorrido para transformada direta: %f ms" %(timedir))
            print("\nTempo decorrido para transformada inversa: %f ms" %(timeinv))
            print("\nDesvio Padrao da serie temporal: %f" %(desvPad(Alum)))
            print("===============================")

        if menu == 3:

            Cattle = read_file("Cattle.csv")

            for i in range(30):
                print(Cattle[i])

        sair = bool(int(input("Sair: 1 Continuar: 0\n")))

def fft_inversa(c):
    n = int(len(c)/2)
    f = [0 for i in range(2*n)]

    for j in range(0,2*n):

        for k in range(0,2*n):

            f[j] += c[k]*cmath.exp(complex(0,k*j*math.pi/n))

    return f

def fft_direta(f):
    n = int(len(f)/2)
    c = [0 for i in range(2*n)]

    for k in range(0,2*n):

        for j in range(0,2*n):

            c[k] += (f[j]*cmath.exp(complex(0,-k*j*math.pi/n)))/(2*n)

    return c

def fftrec(c,f,n,direto):
    fe = [0 for i in range(2*n)]
    fo = [0 for i in range(2*n)]
    even = [c[2*j] for j in range(n)]
    odd = [c[2*j+1] for j in range(n)]

    if n == 1:

        c[0] = f[0]+f[1]
        c[1] = f[0]-f[1]

    else:

        for j in range(n):

            fe[j] = f[2*j]
            fo[j] = f[2*j+1]

        fftrec(even,fe,int(n/2),direto)
        fftrec(odd,fo,int(n/2),direto)

        for j in range(n):

            if direto:
                eij = cmath.exp(complex(0,-j*math.pi/n))

            else:
                eij = cmath.exp(complex(0,j*math.pi/n))

            c[j] = even[j] + eij*odd[j]
            c[j+n] = even[j] - eij*odd[j]

#=======================================================================

def print_vector(vetor):
    ''' matriz -> null '''
    for k in range(45):
        print("=", end="")
    print("",end="\n")
    for n in range (0,len(vetor),1):
        print ("| (%.5f, %.5fi)" %(vetor[n].real, vetor[n].imag), end="")
    print("|")
    for k in range(45):
        print("=", end="")
    print("", end="\n\n")
    return

#=======================================================================

def read_file(file):
    '''
        Lê arquivo
    '''
    f = open(file)
    arquivo = [x.strip('\n') for x in f.readlines()]
    arquivo = [x.split(', ') for x in arquivo]

    for l in range(len(arquivo)):
        for i in range(3):
            if i == 0:
                arquivo[l][i] = datetime.date(int(arquivo[l][i][0:4]),
                                              int(arquivo[l][i][4:6]),
                                              int(arquivo[l][i][6:9]))
            if i == 1:
                arquivo[l][i] = int(arquivo[l][i])

            if i == 2:
                arquivo[l][i] = float(arquivo[l][i])

    return arquivo

#=======================================================================

def mean(numbers):
    '''
        Calcula media de vetor
    '''
    return float(sum(numbers)) / max(len(numbers), 1)

#=======================================================================

def desvPad(file):
    '''
        Calcula desvio padrao
    '''

    valores = [x[2] for x in file]
    media = mean(valores)
    argsum = np.divide(np.power(np.add(valores,-media),2),len(valores))
    desvPad = math.sqrt( sum( argsum ) )

    return float(desvPad)

#=======================================================================

def passa_baixa(c,K):
    c2 = [c[j] if j < K-1 else 0 for j in range(len(c))]
    return c2

def passa_alta(c,K):
    c2 = [c[j] if j >= K-1 else 0 for j in range(len(c))]
    return c2

def passa_baixa2(c,K):
    tam = len(c)
    c2 = [0 for j in range(tam)]
    for j in range(tam):
        if (j < K-1) or (j > tam-K-1 ):
            c2[j] = c[j]
    return c2

def passa_alta2(c,K):
    tam = len(c)
    c2 = [0 for j in range(tam)]
    for j in range(tam):
        if (j >= K-1) and (j <= tam-K-1 ):
            c2[j] = c[j]
    return c2

#=======================================================================

def plotComTF_Lenta(valor, nome, K, tipo_filtro):
    '''
        valor: vetor de floats
        nome: string
        K: inteiro
        tipo_filtro: inteiro
    '''
    print("\n\nNote que a frequencia de indice 0 está omitida do plot")
    time_dir_st = time.time()
    c = fft_direta(valor)
    time_dir_end = time.time()
    time_dir = (time_dir_end - time_dir_st)*1000

    plt.figure(1)
    ax1 = plt.subplot(211)
    t = [j for j in range(1,len(valor))]
    plt.plot(t, np.absolute(c)[1:len(c)], 'bo')
    ax1.set_title(nome)
    plt.subplot(212)
    k = [j for j in range(len(valor))]
    plt.plot(k, valor, 'b-')

    if tipo_filtro == 0:
        c_filtrado = passa_baixa2(c,K)

    if tipo_filtro == 1:
        c_filtrado = passa_alta2(c,K)

    time_inv_st = time.time()
    f_filtrado = fft_inversa(c_filtrado)
    time_inv_end = time.time()
    time_inv = (time_inv_end - time_inv_st)*1000

    plt.subplot(212)
    plt.plot(k, np.real(f_filtrado), 'r-')
    plt.show()

    return (time_dir,time_inv)

#=======================================================================

def plotComTF_rec(valor, nome, K, tipo_filtro):
    '''
        valor: vetor de floats
        nome: string
        K: inteiro
        tipo_filtro: inteiro
    '''
    print("\n\nNote que a frequencia de indice 0 está omitida do plot")
    c = [0 for i in range(len(valor))]
    time_dir_st = time.time()
    fftrec(c,valor,int(len(valor)/2),True)
    time_dir_end = time.time()
    time_dir = (time_dir_end - time_dir_st)*1000

    plt.figure(1)
    ax1 = plt.subplot(211)
    t = [j for j in range(1,len(valor))]
    plt.plot(t, np.absolute(c)[1:len(c)], 'bo')
    ax1.set_title(nome)
    plt.subplot(212)
    k = [j for j in range(len(valor))]
    plt.plot(k, valor, 'b-')

    if tipo_filtro == 0:
        c_filtrado = passa_baixa2(c,K)

    if tipo_filtro == 1:
        c_filtrado = passa_alta2(c,K)

    valor_filtrado = [0 for i in range(len(valor))]
    time_inv_st = time.time()
    fftrec(valor_filtrado,np.divide(c_filtrado,len(c_filtrado)),int(len(valor)/2),False)
    time_inv_end = time.time()
    time_inv = (time_inv_end - time_inv_st)*1000

    plt.subplot(212)
    plt.plot(k, np.real(valor_filtrado), 'r-')
    plt.show()

    return (time_dir,time_inv)

#=======================================================================

def plotComTF_numpy(valor, nome, K, tipo_filtro):
    '''
        valor: vetor de floats
        nome: string
        K: inteiro
        tipo_filtro: inteiro
    '''
    print("\n\nNote que a frequencia de indice 0 está omitida do plot")
    time_dir_st = time.time()
    c = np.fft.rfft(valor)
    time_dir_end = time.time()
    time_dir = (time_dir_end - time_dir_st)*1000

    plt.figure(1)
    ax1 = plt.subplot(211)
    t = [j for j in range(1,int(len(valor)/2)+1)]
    plt.plot(t, np.absolute(c)[1:], 'bo')
    ax1.set_title(nome)
    k = [j for j in range(len(valor))]
    plt.subplot(212)
    plt.plot(k, valor, 'b-')

    if tipo_filtro == 0:
        c_filtrado = passa_baixa(c,K)

    if tipo_filtro == 1:
        c_filtrado = passa_alta(c,K)

    time_inv_st = time.time()
    valor_filtrado = np.fft.irfft(c_filtrado)
    time_inv_end = time.time()
    time_inv = (time_inv_end - time_inv_st)*1000

    plt.subplot(212)
    plt.plot(k, valor_filtrado, 'r-')
    plt.show()

    return (time_dir,time_inv)


import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import scipy

def PDF(n,t,m):
    from scipy.stats import kde
    density = kde.gaussian_kde(n)
    xgrid = np.linspace(n.min(), n.max(), 100)
    plt.title('PDF')
    plt.hist(n, bins=m, density=True)
    plt.plot(xgrid, density(xgrid), 'r-')
    plt.xlabel(f'Distribution of ' + t)
    plt.show()

def BoxWithWhiskers (n,t):
    median = n.quantile(0.5)
    q1 = n.quantile(0.25)
    q3 = n.quantile(0.75)
    iqr = q3 - q1
    print('Median =',median,'; First quartile =',q1,'; Third quartile =',q3,'; Interquartile range =',iqr)
    plt.title('Box With Whiskers')
    plt.xlabel(f'Distribution of ' + t)
    plt.boxplot(n)
    plt.show()

def MaximumLikelihood (n,t,m,lim):
    from scipy.stats import lognorm
    x = np.linspace(n.min(), n.max(), 100)
    parameters = scipy.stats.lognorm.fit(n)
    plt.hist(n, bins=m, density=True)
    pdf = scipy.stats.lognorm.pdf(x, parameters[0], parameters[1], parameters[2])
    plt.xlabel(f'Distribution of ' + t)
    axes = plt.gca()
    axes.set_ylim([0, lim])
    plt.title('Maximum Likelihood Method. Lognormal')
    plt.plot(x, pdf, 'r--')
    plt.show()

def MaximumLikelihoodNormal (n,t,m,lim):
    from scipy.stats import norm
    x = np.linspace(n.min(), n.max(), 100)
    parameters = scipy.stats.norm.fit(n)
    plt.hist(n, bins=m, density=True)
    pdf = scipy.stats.norm.pdf(x, parameters[0], parameters[1])
    plt.xlabel(f'Distribution of ' + t)
    axes = plt.gca()
    axes.set_ylim([0, lim])
    plt.title('Maximum Likelihood Method. Normal')
    plt.plot(x, pdf, 'r--')
    plt.show()

def log(x, m, s):
    return np.exp(-((np.log(x)-m)**2)/(2*s**2))/(x*s*(2*np.pi)**0.5)

def LeastSquaresMethod (n,t,m,m1,lim):
    from scipy import optimize
    value, binsedge = np.histogram(n, bins=m1, density=True)
    ydata = [value[i] for i in range(len(value)-1)]
    xdata = [(binsedge[i] + binsedge[i + 1]) / 2 for i in range(len(value) - 1)]
    popt, pcov = scipy.optimize.curve_fit(log, xdata, ydata)
    mu, sigma = popt
    x = np.linspace(n.min(), n.max(), 100)
    x1 = x.tolist()
    x2 = [log(x2,mu,sigma) for x2 in x1]
    plt.xlabel(f'Distribution of ' + t)
    axes = plt.gca()
    axes.set_ylim([0, lim])
    plt.title('Least Squares Method. Lognormal')
    plt.plot(x, x2, 'r--')
    plt.hist(n, bins=m, density=True)
    plt.show()

def normal(x, m, s):
    return np.exp(-(((x-m)**2)/(2*(s**2)))/(s*(2*np.pi)**0.5))

def LeastSquaresMethodNormal (n,t,m,m1,lim):
    from scipy import optimize
    value, binsedge = np.histogram(n, bins=m1, density=True)
    ydata = [value[i] for i in range(len(value)-1)]
    xdata = [(binsedge[i] + binsedge[i + 1]) / 2 for i in range(len(value) - 1)]
    popt, pcov = scipy.optimize.curve_fit(normal, xdata, ydata)
    mu, sigma = popt
    x = np.linspace(n.min(), n.max(), 100)
    x1 = x.tolist()
    x2 = [normal(x2,mu,sigma) for x2 in x1]
    plt.xlabel(f'Distribution of ' + t)
    axes = plt.gca()
    axes.set_ylim([0, lim])
    plt.title('Least Squares Method. Normal')
    plt.plot(x, x2, 'r--')
    plt.hist(n, bins=m, density=True)
    plt.show()

def QQPlotLognorm (n, m, a, b,t):
    from scipy.stats import lognorm
    parameters = scipy.stats.lognorm.fit(n)
    percs = np.linspace(0, 100, m)
    qn_first = np.percentile(n, percs)
    qn_lognorm = scipy.stats.lognorm.ppf(percs / 100.0, *parameters)

    plt.figure(figsize=(10, 10))
    plt.plot(qn_first, qn_lognorm, ls="", marker="o", markersize=6)
    x = np.linspace(np.min(n), np.max(n))
    plt.plot(x, x, color="k", ls="--")
    plt.xlim(a, b)
    plt.ylim(a, b)
    plt.title(t)
    plt.xlabel(f'Empirical distribution')
    plt.ylabel('Theoretical (lognormal) distribution')
    plt.show()

def QQPlotNorm (n, m, a, b,t):
    from scipy.stats import norm
    parameters = scipy.stats.norm.fit(n)
    percs = np.linspace(0, 100, m)
    qn_first = np.percentile(n, percs)
    qn_norm = scipy.stats.norm.ppf(percs / 100.0, *parameters)

    plt.figure(figsize=(10, 10))
    plt.plot(qn_first, qn_norm, ls="", marker="o", markersize=6)
    x = np.linspace(np.min(n), np.max(n))
    plt.plot(x, x, color="k", ls="--")
    plt.xlim(a, b)
    plt.ylim(a, b)
    plt.title(t)
    plt.xlabel(f'Empirical distribution')
    plt.ylabel('Theoretical (lognormal) distribution')
    plt.show()

def TestLognorm (n):
    from scipy.stats import lognorm
    parameters = scipy.stats.lognorm.fit(n)
    ks = scipy.stats.kstest(n, 'lognorm', parameters, N=100)
    cm = scipy.stats.cramervonmises(n, 'lognorm', parameters)
    print(ks)
    print(cm)

def TestNorm (n):
    from scipy.stats import norm
    parameters = scipy.stats.norm.fit(n)
    ks = scipy.stats.kstest(n, 'norm', parameters, N=100)
    cm = scipy.stats.cramervonmises(n, 'norm', parameters)
    print(ks)
    print(cm)

def LSM_lognormal_for_LSM_and_MLE (n,m1):
    from scipy import optimize
    value, binsedge = np.histogram(n, bins=m1, density=True)
    ydata = [value[i] for i in range(len(value)-1)]
    xdata = [(binsedge[i] + binsedge[i + 1]) / 2 for i in range(len(value) - 1)]
    popt, pcov = scipy.optimize.curve_fit(log, xdata, ydata)
    mu, sigma = popt
    x = np.linspace(n.min(), n.max(), 100)
    x1 = x.tolist()
    x2 = [log(x2,mu,sigma) for x2 in x1]
    return x2

def MLE_lognormal_for_LSM_and_MLE (n):
    from scipy.stats import lognorm
    x = np.linspace(n.min(), n.max(), 100)
    parameters = scipy.stats.lognorm.fit(n)
    pdf = scipy.stats.lognorm.pdf(x, parameters[0], parameters[1], parameters[2])
    return pdf

def LSM_normal_for_LSM_and_MLE (n,m1):
    from scipy import optimize
    value, binsedge = np.histogram(n, bins=m1, density=True)
    ydata = [value[i] for i in range(len(value)-1)]
    xdata = [(binsedge[i] + binsedge[i + 1]) / 2 for i in range(len(value) - 1)]
    popt, pcov = scipy.optimize.curve_fit(normal, xdata, ydata)
    mu, sigma = popt
    x = np.linspace(n.min(), n.max(), 100)
    x1 = x.tolist()
    x2 = [normal(x2,mu,sigma) for x2 in x1]
    return x2

def MLE_normal_for_LSM_and_MLE (n):
    from scipy.stats import norm
    x = np.linspace(n.min(), n.max(), 100)
    parameters = scipy.stats.norm.fit(n)
    pdf = scipy.stats.norm.pdf(x, parameters[0], parameters[1])
    return pdf

def MLE_and_LSM_Lognormal(n, t, m, m1, lim):
    x = np.linspace(n.min(), n.max(), 100)

    plt.hist(n, bins=m, density=True)
    plt.plot(x, MLE_lognormal_for_LSM_and_MLE(n), 'r--')
    plt.plot(x, LSM_lognormal_for_LSM_and_MLE(n, m1), 'b--')
    plt.legend(['Maximum Likelihood Method', 'Least Squares Method'])

    plt.xlabel(f'Distribution of ' + t)
    axes = plt.gca()
    axes.set_ylim([0, lim])
    plt.title('Maximum Likelihood Method and Least Squares Method. Lognormal')
    plt.show()

def MLE_and_LSM_Normal(n, t, m, m1, lim):
    x = np.linspace(n.min(), n.max(), 100)

    plt.hist(n, bins=m, density=True)
    plt.plot(x, MLE_normal_for_LSM_and_MLE(n), 'r--')
    plt.plot(x, LSM_normal_for_LSM_and_MLE(n, m1), 'b--')
    plt.legend(['Maximum Likelihood Method', 'Least Squares Method'])

    plt.xlabel(f'Distribution of ' + t)
    axes = plt.gca()
    axes.set_ylim([0, lim])
    plt.title('Maximum Likelihood Method and Least Squares Method. Normal')
    plt.show()

path_to_file = 'avocado.csv'
data = pd.read_csv(r'C:\Users\Ксения\Documents\Учеба\ИТМО\1 семестр\Методы\Лаб 1\Код Лаб 1\avocado.csv')
df = pd.DataFrame(data,columns= ['AveragePrice', 'Total Volume', '4046'])

PDF(df['AveragePrice'],'Average Price', 10)
PDF(df['Total Volume'],'Total Volume', 50)
PDF(df['4046'],'Total number of avocados sold with PLU 4046', 50)

BoxWithWhiskers(df['AveragePrice'],'Average Price')
BoxWithWhiskers(df['Total Volume'],'Total Volume')
BoxWithWhiskers(df['4046'],'Total number of avocados sold with PLU 4046')

MaximumLikelihood(df['AveragePrice'],'Average Price', 10, 1.1)
MaximumLikelihood(df['Total Volume'], 'Total Volume', 50, 0.0000008)
MaximumLikelihood(df['4046'], 'Total number of avocados sold with PLU 4046', 50, 0.0000008)

LeastSquaresMethod(df['AveragePrice'],'Average Price', 10, 10, 1.1)
LeastSquaresMethod(df['Total Volume'], 'Total Volume', 50, 100000, 0.0000008)
LeastSquaresMethod(df['4046'], 'Total number of avocados sold with PLU 4046', 50, 100000, 0.0000008)

QQPlotLognorm(df['AveragePrice'], 21, 0, 4,'Average Price. Lognorm')
QQPlotLognorm(df['Total Volume'], 1000, 0, 10000000, 'Total Volume. Lognorm')
QQPlotLognorm(df['4046'], 100, 0, 50000, 'Total number of avocados sold with PLU 4046. Lognorm')

print('Test for Average Price Lognorm')
TestLognorm(df['AveragePrice'])
print('---------------------------------------------------------------------------')
print('Test for Total Volume Lognorm')
TestLognorm(df['Total Volume'])
print('---------------------------------------------------------------------------')
print('Test for 4046 Lognorm')
TestLognorm(df['4046'])

MaximumLikelihoodNormal(df['AveragePrice'],'Average Price', 10, 1.1)
LeastSquaresMethodNormal(df['AveragePrice'],'Average Price', 10, 10, 1.1)
QQPlotNorm(df['AveragePrice'], 21, 0, 4,'Average Price. Normal')
print('Test for Average Price Normal')
TestNorm(df['AveragePrice'])

MLE_and_LSM_Lognormal(df['Average Price'], 'AveragePrice', 10, 100, 1.1)
MLE_and_LSM_Lognormal(df['Total Volume'], 'Total Volume', 50, 100000, 0.0000008)
MLE_and_LSM_Lognormal(df['4046'], 'Total number of avocados sold with PLU 4046', 50, 100000, 0.0000008)

MLE_and_LSM_Normal(df['Average Price'], 'AveragePrice', 10, 100, 1.1)


"""
Linear Regression Algorithm
David Felipe Alvear Goyes
Artificial Intelligence Course 2020 - I
Columbia University
04/2020
"""

import sys
import pandas as pd
import numpy as np
import sklearn
import matplotlib.pyplot as plt

class linear_regression(object):
    def __init__(self, data, labels, lr, iterations):
        self.data = data
        self.labels = labels
        self.lr = lr
        self.iterations = iterations

    def run(self):
        betas = np.zeros(self.data.shape[1])
        log_betas = []

        # iterate over all the learning rates
        for lrate in self.lr:
            counter = 0
            betas = np.zeros(self.data.shape[1]) # reset betas

            # Gradient descent for the given learning rate
            for counter in range(self.iterations+1):
                betas = self.gradient_descent(self.data, self.labels, betas, lrate)

            log_betas.append([lrate, counter, betas[0], betas[1], betas[2]])

        return log_betas

    def gradient_descent(self, data, labels, betas, lr):
        # Calculate the derivative of the risk function
        functionx = np.dot(data, betas)
        temp1 = (functionx[:,None]-labels).flatten()
        loss = (data*temp1[:,None]).sum(axis=0)
        dRisk = lr * loss / (data.shape[0])

        # Update betas
        betas = betas - dRisk
        return betas

class data_utils(object):
    def __init__(self, input, output):
        self.input_string = input
        self.output = output

    def load_data(self):
        """
            load_data - Read csv file with the linear dataset
            - input_string : name of the csv file to read
            - return array with the data loaded
        """
        input_data = pd.read_csv(self.input_string, sep=',',header=None)

        return input_data.values

    def normalization(self, data):
        """
        Data - age, weight, height
        return - normalized data
        """
        # Nomrmalize the data
        #mean = np.mean(data, axis=0) #[age_m, we_m, hei_m]
        mean = data.mean(axis=0)
        # Standard deviation
        #std = np.std(data, axis=0) #[age_std, we_std, hei_std]
        std = data.std(axis=0)
        data = (data - mean[None,:]) / std[None,:]
        return data

    def output_file(self, list_betas):
        pd.DataFrame(list_betas).to_csv(self.output, header=None, index=None)

def main():
    ## First - load the input1.csv
    sm = sys.argv[1].lower()
    input_name = sys.argv[1].split(" ")[0]
    output_name = sys.argv[2].split(" ")[0]

    lrList = [0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10, 0.67]
    #iterationMaxList = [100, 100, 100, 100, 100, 100, 100, 100, 100, 58]

    data_manager = data_utils(input_name, output_name)
    raw_data = data_manager.load_data()
    data = data_manager.normalization(raw_data[:,:-1]) # Normalize data

    #data = data[:,:-1] # extract X inputs from the data
    labels = raw_data[:,-1:] # extract labels from the data
    datax = np.c_[np.ones(raw_data.shape[0]), data] # add one column at end for w0

    # linearRegression
    linearRegression = linear_regression(datax, labels, lrList, 100)
    log_betas = linearRegression.run()

    ## save results
    data_manager.output_file(log_betas)
    #data_maneger.graph(raw_data, final_weights)

if __name__ == '__main__':
    main()


import math
import numpy as np

def cal_quartile(arr):
    # 计算四分位的各个数值
    sortedArr = np.sort(arr)  # small --> big
    num = arr.size

    Q1_index = (num + 1) / 4
    if (num + 1) % 4 == 0:
        Q1 = sortedArr[int(Q1_index)]
    else:
        decimal_part = Q1_index - math.floor(Q1_index)
        if decimal_part < 0.5:
            Q1 = 0.75 * sortedArr[int(Q1_index)] + 0.25 * sortedArr[int(Q1_index) + 1]
        else:
            Q1 = 0.25 * sortedArr[int(Q1_index)] + 0.75 * sortedArr[int(Q1_index) + 1]

    Q2_index = (num + 1) / 2
    if (num + 1) % 2 == 0:
        Q2 = sortedArr[int(Q2_index)]
    else:
        decimal_part = Q2_index - math.floor(Q2_index)
        if decimal_part < 0.5:
            Q2 = 0.75 * sortedArr[int(Q2_index)] + 0.25 * sortedArr[int(Q2_index) + 1]
        else:
            Q2 = 0.25 * sortedArr[int(Q2_index)] + 0.75 * sortedArr[int(Q2_index) + 1]

    Q3_index = (num + 1) * 3 / 4
    if (num + 1) * 3 % 4 == 0:
        Q3 = sortedArr[int(Q3_index)]
    else:
        decimal_part = Q3_index - math.floor(Q3_index)
        if decimal_part < 0.5:
            Q3 = 0.75 * sortedArr[int(Q3_index)] + 0.25 * sortedArr[int(Q3_index) + 1]
        else:
            Q3 = 0.25 * sortedArr[int(Q3_index)] + 0.75 * sortedArr[int(Q3_index) + 1]

    # IQR: (Inter-Quartile Range) 在统计中叫内距.内距又称为四分位差
    IQR = Q3 - Q1

    up_boundary = Q3 + 1.5 * IQR
    low_boundary = Q1 - 1.5 * IQR
    return [low_boundary, Q1, Q2, Q3, up_boundary]