# Bite Size Bayes

Copyright 2020 Allen B. Downey

License: [Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0)](https://creativecommons.org/licenses/by-nc-sa/4.0/)

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

## Fixing GPS

The Global Positioning System (GPS) is widely used satellite-based navigation system.  If you have a GPS receiver, you can use it to estimate your position on earth in three dimensions: latitude, longitude, and altitude.

Although GPS is accurate enough for applications like getting driving directions, is is subject to errors from several sources, [described in this Wikipedia article](https://en.wikipedia.org/wiki/Error_analysis_for_the_Global_Positioning_System).

For purposes of this problem, I'll assume that the distribution of errors in the horizontal directions is a normal distribution with mean 0 and standard deviation 5 m.  I'll also assume that errors in the $x$ and $y$ directions are independent.

In other words, if you stand on a survey marker placed by the [National Geodetic Survey (NGS)](https://en.wikipedia.org/wiki/U.S._National_Geodetic_Survey), so that your position is known with high accuracy, and take a single reading from a GPS receiver, the difference between the reading and the actual position would be well-modeled by a normal distribution.

SciPy provides an object called `norm` we can use to represent a normal distribution with a given mean and standard deviation.

from scipy.stats import norm

error_dist = norm(0, 5)

`norm` provides a function called `pdf` that compute the Probability Density Function (PDF) of the distribution.

xs = np.linspace(-15, 15)
ps = error_dist.pdf(xs)

error_series = pd.Series(ps, index=xs)

error_series.plot()
plt.xlabel('Error (m)')
plt.ylabel('Probability density')
plt.title('Error distribution');

def outer_product(s1, s2):
    """Compute the outer product of two Series.

    First Series goes down the rows;
    second goes across the columns.

    s1: Series
    s2: Series

    return: DataFrame
    """
    a = np.multiply.outer(s1.to_numpy(), s2.to_numpy())
    return pd.DataFrame(a, index=s1.index, columns=s2.index)

Here's what the distributions of width look like.

joint_dist = outer_product(error_series, error_series)

def plot_contour(joint):
    """Plot a joint distribution.

    joint: DataFrame representing a joint PMF
    """
    plt.contour(joint.index, joint.index, joint)
    plt.ylabel('y position in m')
    plt.xlabel('x position in m')

plot_contour(joint_dist)

## Inferring position

Now suppose I am standing in a field with a survey marker at the center.  I'll define the position of the marker to be the origin, where `x` and `y` positions are `0`, and I'll define east to be the positive `x` axis and north to be the positive `y` axis.

I don't know my actual location, but based on a single GPS reading, I compute that I am 10 m east and 20 m north of the marker.

Based on this reading, what is the posterior distribution of my actual location?

Well, it depends on how I got there.

To get started, let's assume that I am equally likely to be at any location in a large, square field.  In that case, `x` and `y` are independent and their distributions are uniform.

def make_pmf(xs, ps, **options):
    """Make a Series that represents a PMF.

    xs: sequence of values
    ps: sequence of probabilities
    options: keyword arguments passed to Series constructor

    returns: Pandas Series
    """
    pmf = pd.Series(ps, index=xs, **options)
    pmf /= pmf.sum()
    return pmf

xs = np.linspace(-50, 50, 101)
prior_x = make_pmf(xs, 1)

ys = np.linspace(-50, 50, 101)
prior_y = make_pmf(ys, 1)

I'll group the data by species and compute summary statistics.

To compute the likelihood of the `x` data, we can ask, "If my actual position were `x`, what is the probability my GPS reading would be 10 m?"

To answer that question, we can compute the hypothetical error for each value of `x`:

error_x = 10 - xs

Then we can use `error_dist` to compute the probability of the hypothetical errors.

likelihood = error_dist.pdf(error_x)

And the posterior distribution.

posterior_x = prior_x * likelihood
posterior_x /= posterior_x.sum()

prior_x.plot(label='prior')
posterior_x.plot(label='posterior')

plt.xlabel('Position (m)')
plt.ylabel('Probability')
plt.title('Prior distribution of position');

error_y = 20 - ys

likelihood = error_dist.pdf(error_y)

posterior_y = prior_y * likelihood
posterior_y /= posterior_y.sum()

joint_posterior1 = outer_product(posterior_y, posterior_x)
plot_contour(joint_posterior1)

This result might not come as a surprise to you.  With a uniform prior, the posterior is just a normalized version of the likelihood function.

Since the likelihood function is the PDF of a normal distribution, the posterior is a normal distribution with mean at the measured location, (10, 20), and the same standard deviation as the error distribution.

So we could have figured that out in our heads.  But now that we have done it the hard way, we are ready to solve harder problems.

## Parachuting in

Suppose I parachute into the field and try to land on the marker, and suppose my actualy landing position is well modeled by a normal distribution, along both axes, with mean 0 and standard deviation 15 m.

We can construct the marginal distributions for my positions along the `x` and `y` axes, like this:

landing_dist = norm(0, 15)

xs = np.linspace(-50, 50, 101)
ps = landing_dist.pdf(xs)
prior_x = make_pmf(xs, ps)

ys = np.linspace(-50, 50, 101)
ps = landing_dist.pdf(ys)
prior_y = make_pmf(ys, ps)

Assuming that my errors along the two axes are independent, we can use `outer_product` to construct the joint distribution for my landing spot.

joint_posterior = outer_product(prior_y, prior_x)

plot_contour(joint_posterior)

After landing, suppose I take a single GPS reading that indicates I am 10 m east and 20 m north of the marker.

I can use the GPS reading to update the distribution of my `x` coordinate, exactly as in the previous problem.

error_x = 10 - xs

likelihood = error_dist.pdf(error_x)

posterior_x = prior_x * likelihood
posterior_x /= posterior_x.sum()

Here's what the prior and posterior distributions look like for my `x` coordinate:

prior_x.plot(label='prior')
posterior_x.plot(label='posterior')

plt.xlabel('Position (m)')
plt.ylabel('Probability')
plt.title('Prior distribution of position')
plt.legend();

To compute the posterior mean, I'll use this function, which we've seen in a previous notebook.

def pmf_mean(pmf):
    """Compute the mean of a PMF.

    pmf: Series representing a PMF

    return: float
    """
    return np.sum(pmf.index * pmf)

The posterior mean is 9, a little less than the GPS reading, because in the prior distribution, values closer to 0 are more likely.

pmf_mean(posterior_x)

**Exercise:**  Update the `y` coordinate with the GPS read (20 m north) and compute the posterior mean.  We expect it to be a bit less than 20 m.

# Solution

error_y = 20 - ys

likelihood = error_dist.pdf(error_y)

posterior_y = prior_y * likelihood
posterior_y /= posterior_y.sum()

# Solution

pmf_mean(posterior_y)

In this version of the problem, the `x` and `y` coordinates are independent, so we can use `outer_product` to construct the joint posterior.

joint_posterior2 = outer_product(posterior_y, posterior_x)
plot_contour(joint_posterior2)

If you are not convinced that the coordinates are independent, just because I say say, you could update the joint prior distribution rather than the marginal distributions.  In the next example, we'll see how.

## Running away

Suppose I start out standing on the marker, spin until I am facing in a random direction, and then walk about 30 m, where my margin of error is 3 m.

So my distance from the origin is well modeled by a normal distribution with mean 30 and standard deviation 3.

Before I get a GPS reading, what is the prior distribution of my position?

In this example, `x` and `y` are not independent.  For example, if I know that `x` is near 30 m, that implies that `y` is near 0.  And if I know `y` is near 0, that implies that `x` is close to 30 m or -30 m.

To construct the prior distribution, I'll start with a joint distribution for `x` and `y` that is uniform and independent.  Then I'll do a Bayesian update with the data that I am 15 from the origin.

Here are the uniform marginal priors we've used before.

xs = np.linspace(-50, 50, 101)
prior_x = make_pmf(xs, 1)

ys = np.linspace(-50, 50, 101)
prior_y = make_pmf(ys, 1)

And here is the uniform joint prior.

joint_prior = outer_product(prior_x, prior_y)

To compute the distances from the center, I'll use the `outer` version of the NumPy function `hypot`, which computes the hypotenuse of a triangle with the given sides.

See how useful outer function are?

distances = np.hypot.outer(xs, ys)
distances.shape

Now, to compute a likelihood function, we can compute `error`, which is the difference between each hypothetical distance and the data, which is 30 m.

error = distances - 30
error.shape

The distribution of these errors is normal with standard deviation 3 m.

error_dist = norm(0, 3)

To get the likelihoods, we evaluate the normal PDF for each hypothetical error.

likelihood = error_dist.pdf(error)
likelihood.shape

As usual, the posterior distribution is the normalize product of the prior and the likelihood.

joint_posterior3 = joint_prior * likelihood
joint_posterior3 /= joint_posterior3.to_numpy().sum()

And here's what the posterior joint distribution looks like.

plot_contour(joint_posterior3)

plt.title('Joint distribution');

Remember that we did all that to compute the prior before we get a GPS reading.

Now suppose we take a reading and it indicates that we are 10 m east and 20 m north of the marker (as in the previous versions).

We'll use the posterior we just computed as the prior for the next update.

donut_prior = joint_posterior3

To compute the likelihoods, I'm going to cheat a little.

Remember that in the first version of the problem we started with a uniform prior and did an update with the same data.  The posterior we computed is just the normalized likelihood of the data, so we can use it as the likelihood for this update.

likelihood = joint_posterior1

As usual, the posterior is the normalized product of the prior and the likelihood.

donut_posterior = donut_prior * likelihood
donut_posterior /= donut_posterior.to_numpy().sum()

And here's what it looks like.

plot_contour(donut_posterior)

plt.title('Joint posterior distribution');

It reminds me of [The Bean](https://en.wikipedia.org/wiki/Cloud_Gate).

## Marginal posterior distributions

From the joint posterior distribution we can extract the marginal distributions for the coordinates.

def marginal(joint, axis):
    """Compute a marginal distribution.

    axis=0 returns the marginal distribution of the second variable
    axis=1 returns the marginal distribution of the first variable

    joint: DataFrame representing a joint PMF
    axis: int axis to sum along

    returns: Series representing a marginal PMF
    """
    return joint.sum(axis=axis)

donut_posterior_x = marginal(donut_posterior, 0)
donut_posterior_y = marginal(donut_posterior, 1)

Here's what they look like.

donut_posterior_x.plot(label='x')
donut_posterior_y.plot(label='y')

plt.xlabel('Position (m)')
plt.ylabel('Probability')
plt.title('Marginal posterior distributions')
plt.legend();

And here are the posterior means.

mean_x = pmf_mean(donut_posterior_x)
mean_x

mean_y = pmf_mean(donut_posterior_y)
mean_y

Here's the distance from the marker to the location of the posterior means:

np.hypot(mean_x, mean_y)

This distance represents a compromise between two sources of information:

* I started with the prior belief that I was 30 m from the marker, but

* My GPS reading suggests that I am only about 22 m from the marker.

np.hypot(10, 20)

The posterior mean distance is close to 30 m than 22 m because the margin of error in the prior, 3 m, is less than the margin of error of GPS, which is 5 m in each direction.

np.hypot(5, 5)

In this case, Bayes's theorem tells us that we should give more weight to out prior belief about the distance.

Actually, the posterior mean distance we just computed is not quite right because we computed the means of the marginal distributions independently.

To compute the posterior mean correctly, we have to compute the weighted sum of the hypothetical distances, using the posterior probabilities as weights.

qs = distances
ps = donut_posterior.values

np.sum(qs * ps)

## Summary


from numpy import random

class DistributionGenerator:
    """
    Generate random data within a given distribution
    """

    def __init__(self, num_samples, mean=0, sd=1, interval=1, n=1, p=0.5):
        """

        :param num_samples: Int The number of samples to be returned
        :param mean: The Mean value for a normal distribution
        :param sd: The Standard Deviation for a normal distribution
        :param interval: The Interval for a poisson distribution
        :param n: The number of samples for a binomial distribution
        :param p: The probability for a binomial distribution
        """
        self.num_samples = num_samples
        self.mean = mean
        self.sd = sd
        self.interval = interval
        self.n = n
        self.p = p

        self.data = None

    def normal(self):
        """ Produce an array of values in a normal distribution

        :return: ndarray
        """
        self.data = random.normal(size=self.num_samples, loc=self.mean, scale=self.sd)
        return self.data

    def poisson(self):
        """ Produce an array of values in a poisson distribution

        :return: ndarray
        """
        self.data = random.poisson(size=self.num_samples, lam=self.interval)
        return self.data

    def binomial(self):
        """ Produce an array of values in a binomial distribution

        :return: ndarray
        """
        self.data = random.binomial(size=self.num_samples, n=self.n, p=self.p)
        return self.data

    def summarise(self):
        """ Print summary statistics about the last dataset created

        :return:
        """
        if self.data is None:
            print("There is no current data")
        else:
            print("Min: " + str(self.data.min()))
            print("Max: " + str(self.data.max()))
            print("Mean: " + str(self.data.mean()))
            print("Standard deviation: " + str(self.data.std()))


# logisitc regression classifier for the XOR problem.
#
# the notes for this class can be found at:
# https://deeplearningcourses.com/c/data-science-logistic-regression-in-python
# https://www.udemy.com/data-science-logistic-regression-in-python

import numpy as np
import matplotlib.pyplot as plt

N = 4
D = 2

# XOR
X = np.array([
    [0, 0],
    [0, 1],
    [1, 0],
    [1, 1],
])
T = np.array([0, 1, 1, 0])

# add a column of ones
# ones = np.array([[1]*N]).T
ones = np.ones((N, 1))

# add a column of xy = x*y
xy = np.matrix(X[:,0] * X[:,1]).T
Xb = np.array(np.concatenate((ones, xy, X), axis=1))

# randomly initialize the weights
w = np.random.randn(D + 2)

# calculate the model output
z = Xb.dot(w)

def sigmoid(z):
    return 1/(1 + np.exp(-z))

Y = sigmoid(z)

# calculate the cross-entropy error
def cross_entropy(T, Y):
    E = 0
    for i in xrange(N):
        if T[i] == 1:
            E -= np.log(Y[i])
        else:
            E -= np.log(1 - Y[i])
    return E

# let's do gradient descent 100 times
learning_rate = 0.001
error = []
for i in xrange(10000):
    e = cross_entropy(T, Y)
    error.append(e)
    if i % 100 == 0:
        print e

    # gradient descent weight udpate with regularization
    # w += learning_rate * ( np.dot((T - Y).T, Xb) - 0.01*w )
    w += learning_rate * ( Xb.T.dot(T - Y) - 0.01*w )

    # recalculate Y
    Y = sigmoid(Xb.dot(w))

plt.plot(error)
plt.title("Cross-entropy per iteration")
plt.show()

print "Final w:", w
print "Final classification rate:", 1 - np.abs(T - np.round(Y)).sum() / N


# IMPORTS
import numpy as np
import scipy.stats as stats
import matplotlib.pyplot as plt
import matplotlib.style as style
from IPython.core.display import HTML

# PLOTTING CONFIG
%matplotlib inline
style.use('fivethirtyeight')
plt.rcParams["figure.figsize"] = (14, 7)
HTML("""
<style>
.output_png {
    display: table-cell;
    text-align: center;
    vertical-align: center;
}
</style>
""")
plt.figure(dpi=100)

# PDF k = 1
plt.plot(np.linspace(0, 15, 500),
         stats.chi2.pdf(np.linspace(0, 15, 500), df=1),
        )
plt.fill_between(np.linspace(0, 15, 500),
                 stats.chi2.pdf(np.linspace(0, 15, 500), df=1),
                 alpha=.15,
                )

# PDF k = 3
plt.plot(np.linspace(0, 15, 100),
         stats.chi2.pdf(np.linspace(0, 15, 100), df=3),
        )
plt.fill_between(np.linspace(0, 15, 100),
                 stats.chi2.pdf(np.linspace(0, 15, 100), df=3),
                 alpha=.15,
                )

# PDF k = 6
plt.plot(np.linspace(0, 15, 100),
         stats.chi2.pdf(np.linspace(0, 15, 100), df=6),
        )
plt.fill_between(np.linspace(0, 15, 100),
                 stats.chi2.pdf(np.linspace(0, 15, 100), df=6),
                 alpha=.15,
                )

# LEGEND
plt.text(x=.5, y=.7, s="$ k = 1$", rotation=-65, alpha=.75, weight="bold", color="#008fd5")
plt.text(x=1.5, y=.35, s="$ k = 3$", alpha=.75, weight="bold", color="#fc4f30")
plt.text(x=5, y=.2, s="$ k = 6$", alpha=.75, weight="bold", color="#e5ae38")

# TICKS
plt.tick_params(axis = 'both', which = 'major', labelsize = 18)
plt.axhline(y = 0, color = 'black', linewidth = 1.3, alpha = .7)

# TITLE, SUBTITLE & FOOTER
plt.text(x = -1.5, y = 2.8, s = "Chi-Squared Distribution - $ k $",
               fontsize = 26, weight = 'bold', alpha = .75)
plt.text(x = -1.5, y = 2.5,
         s = 'Depicted below are three Chi-Squared distributed random variables with varying $ k $. As one can\nsee the parameter $k$ smoothens the distribution and softens the skewness.',
         fontsize = 19, alpha = .85)
plt.text(x = -1.5,y = -0.4,
         s = '   ©Joshua Görner                                                                                github.com/jgoerner   ',
         fontsize = 14, color = '#f0f0f0', backgroundcolor = 'grey');


"""
* Assignment: Numpy Broadcasting Arithmetic
* Complexity: easy
* Lines of code: 4 lines
* Time: 3 min

English:
    1. Define `a: np.ndarray` with square root of each element in `A`
    2. Define `b: np.ndarray` with square root of each element in `B`
    3. Define `c: np.ndarray` with second power (square) of each element in `C`
    4. Add elements from `a` to `b`
    5. Multiply the result by `c`
    6. Run doctests - all must succeed

Polish:
    1. Zdefiniuj `a: np.ndarray` z pierwiastkiem kwadratowym każdego elementu `A`
    2. Zdefiniuj `b: np.ndarray` z pierwiastkiem kwadratowym każdego elementu `B`
    3. Zdefiniu `c: np.ndarray` z drugą potęgą (kwadratem) każdego z elementu w `C`
    4. Dodaj elementy z `a` do `b`
    5. Przemnóż wynik przez `c`
    6. Uruchom doctesty - wszystkie muszą się powieść

Tests:
    >>> import sys; sys.tracebacklimit = 0

    >>> assert result is not Ellipsis, \
    'Assign result to variable: `result`'
    >>> assert type(result) is np.ndarray, \
    'Variable `result` has invalid type, expected: np.ndarray'

    >>> result
    array([[ 1.41421356,  2.73205081],
           [45.254834  ,  0.        ]])
"""

import numpy as np

A = np.array([[0, 1], [2, 3]], float)
B = np.array([2, 3], float)
C = np.array([[1, 1], [4, 0]], float)

# np.ndarray: square root of each element in `A` use np.pow()
a = ...

# np.ndarray: square root of each element in `B` use `**` operator
b = ...

# np.ndarray: second power (square) of each element in `C` use `**` operator
c = ...

# np.ndarray: Add elements from `a` to `b` and then multiply by `c`
#             Remember about the operator precedence
result = ...


"""
This script demonstrates the implementation of the ReLU function.

It's a kind of activation function defined as the positive part of its argument in the
context of neural network.
The function takes a vector of K real numbers as input and then argmax(x, 0).
After through ReLU, the element of the vector always 0 or real number.

Script inspired from its corresponding Wikipedia article
https://en.wikipedia.org/wiki/Rectifier_(neural_networks)
"""
from typing import List

import numpy as np

def relu(vector: List[float]):
    """
    Implements the relu function

    Parameters:
        vector (np.array,list,tuple): A  numpy array of shape (1,n)
        consisting of real values or a similar list,tuple

    Returns:
        relu_vec (np.array): The input numpy array, after applying
        relu.

    >>> vec = np.array([-1, 0, 5])
    >>> relu(vec)
    array([0, 0, 5])
    """

    # compare two arrays and then return element-wise maxima.
    return np.maximum(0, vector)

if __name__ == "__main__":
    print(np.array(relu([-1, 0, 5])))  # --> [0, 0, 5]


import numpy as np

# We define our Train Loss fuction (1/2)*np.transpose(X*w-y)*(X*w-y)
def train_loss(d_train, w):
    # We create the matrix X and the matriz y
    X = []
    y = []
    for data in d_train:
        X.append([1] + data[0]), y.append(data[1])
    X = np.array(X)
    y = np.array(y)
    w = np.array(w)
    return (1/2)*np.dot(np.transpose((np.dot(X, w) - y)), (np.dot(X, w) - y))

# We define our Train Loss Gradient fuction (ϕ(x) = x)
def train_loss_gradient(x, y, w):
    return 2 * np.dot((np.dot(w, x) - y), x)

# We define our Gradient descent algorithm
def stochastic_gradient_descent(d_train, w, n, iterations):
    print("Running stochastic gradient descent algorithm")
    iteration = 1
    j = 0
    while iteration <= iterations:
        y = d_train[j][1]
        # We add 1 as the first component in x
        x = np.array([1] + d_train[j][0])
        # We calculate the Train Loss Gradient
        train_loss_gradient_ = train_loss_gradient(x, y, w)
        # We update the vector w
        w = w - n * train_loss_gradient_
        # We calculate the Train Loss
        train_loss_ = train_loss(d_train, w)
        # We print the data
        print(f'In the iteration {iteration}, we get '
              f'w = {w} and TrainLoss(x, y, w) = {train_loss_}')
        iteration += 1
        if j == len(d_train) - 1:
            j = 0
        else:
            j += 1

    print('Algoritm finished')

# We define our DTrain, our weight vector, our n and our iteration number
d_train = [[[0, 0], 0], [[-1, 0], 1 / 2], [[1, 0], 1 / 2], [[0, 1], 1], [[1, 1], 3 / 2]]
w = np.array([0, 1, 1])
n = 0.5
iterations = 5

# We run our stochastic gradient descent algorithm
stochastic_gradient_descent(d_train, w, n, iterations)


# plot Rastrigin function
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import cm
from mpl_toolkits.mplot3d import Axes3D

X = np.linspace(-5.12, 5.12, 100)
Y = np.linspace(-5.12, 5.12, 100)
X, Y = np.meshgrid(X, Y)

Z =  2 * 10 + (X**2 - 10 * np.cos(2 * np.pi * X)) + (Y**2 - 10 * np.cos(2 * np.pi * Y))

# plot 3D
fig = plt.figure()
ax = fig.gca(projection='3d')
CS = ax.plot_surface(X, Y, Z, rstride=1, cstride=1, cmap=cm.nipy_spectral, linewidth=0.08, antialiased=True)
plt.xlabel("x")
plt.ylabel("y")
plt.title("Rastrigin Function", fontsize='small')
plt.grid(color='silver', linestyle='-', linewidth=0.2)
# add colorbar
cbar = fig.colorbar(CS)
cbar.ax.set_ylabel('Objective Value')
plt.tight_layout()
plt.show()

# plot the contour
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
ax.contour(X, Y, Z, 100, cmap=cm.nipy_spectral)

plt.xlabel("x")
plt.ylabel("y")
plt.title("Rastrigin Function", fontsize='small')
plt.grid(color='silver', linestyle='-', linewidth=0.2)
plt.tight_layout()
plt.show()


import numpy as np

weights = {
    'excellent': 3,
    'good': 2,
    'fair': 1
}

data = np.array([
    'excellent',
    'fair',
    'good',
    'excellent'
])

# Parsing the categories to weights
data = np.fromiter(map(lambda x: weights[x], data), dtype=np.int)

# Creating a zero matrix for the Proximity measure matrix
dmatrix  = np.zeros((data.shape[0], data.shape[0]))

# Get Pmeasure matrix dimensions
x, y = dmatrix.shape

# Max range in the array
M = data.max()

# Looping a lower triangular matrix
for j in range(0, y):
    for i in range(j+1, y):
        # d(i, j) = Z(i) - Z(j)
        # Z(i) = (r(i) - 1) / M - 1
        # Where Z(i) is the norm of "i" and r(i) is the weight of "i"
        # Carrying out the sum of fractions and simplifying,
        # the following equation remains
        dmatrix[i,j] = np.abs(data[i] - data[j])/(M - 1)

print(dmatrix)


# Start initialization
import numpy as np
import pylab as p

# Make an x,z grid
grid = np.linspace(-6,6,200)
x, z = np.meshgrid(grid,grid)
mu=4*np.pi  # dipole strength (in z-direction)
# and the 2D cross section of the potential (scalar field)
phi = mu * z / (4 * np.pi * (x**2+z**2)**(3/2))
# End initialization

# Start contour plot
from matplotlib import colors
p.figure(figsize=(6,6))
heights = np.linspace(-0.6,0.6,7)
CS = p.contour(x,z,phi,heights,colors='k')
p.clabel(CS, inline=1, fontsize=10,fmt='%3.1f')
p.xlabel(r'$x$')
p.ylabel(r'$z$')
# End contour plot

# Start fieldline plot
Fx = mu / (4 * np.pi * (x**2+z**2)**(5/2)) * 2*x*z
Fz = mu / (4 * np.pi * (x**2+z**2)**(5/2)) * (z**2-x**2)
p.streamplot(x,z,Fx,Fz)
# End fieldline plot
CS = p.contour(x,z,phi,heights,colors='k')
p.clabel(CS, inline=1, fontsize=10,fmt='%3.1f')
p.savefig('dipole_fieldplot.png')

#p.show()


from sympy import Range

def is_composite(a: int, d: int, n: int, s: int) -> bool:
    if pow(a, d, n) == 1:
        return False
    for i in Range(s):
        if pow(a, 2 ** i * d, n) == n - 1:
            return False
    # n  is definitely composite
    return True

def is_prime(number: int, rounds: int = 128) -> int:
    if number in primes:
        return True
    if any((number % p) == 0 for p in primes) or number in (0, 1):
        return False
    d, s = number - 1, 0
    while not d % 2:
        d, s = d >> 1, s + 1
    if number < 1373653:
        return not any(is_composite(a, d, number, s) for a in (2, 3))
    if number < 25326001:
        return not any(is_composite(a, d, number, s) for a in (2, 3, 5))
    if number < 118670087467:
        if number == 3215031751:
            return False
        return not any(is_composite(a, d, number, s) for a in (2, 3, 5, 7))
    if number < 2152302898747:
        return not any(is_composite(a, d, number, s) for a in (2, 3, 5, 7, 11))
    if number < 3474749660383:
        return not any(is_composite(a, d, number, s) for a in (2, 3, 5, 7, 11, 13))
    if number < 341550071728321:
        return not any(
            is_composite(a, d, number, s) for a in (2, 3, 5, 7, 11, 13, 17)
            )
    # otherwise
    return not any(is_composite(a, d, number, s) for a in primes[:rounds])

primes = [2, 3]
primes += [x for x in range(5, 1000, 2) if is_prime(x)]

def main() -> None:
    number = int(input("Enter the number to check for primality: "))
    if is_prime(number):
        print("The number", number, "is prime.")
    else:
        print("The number", number, "is composite.")

if __name__ == "__main__":
    main()


from abc import ABC, abstractmethod
import numpy as np

__all__ = [
    "ActivationFn",
    "Linear",
    "Sigmoid",
    "Softmax",
    "TanH",
    "ReLU",
    "LeakyReLU",
]

class ActivationFn(ABC):
    @abstractmethod
    def __call__(self, x):
        pass

    @abstractmethod
    def gradient(self, x):
        pass

# ==================================================================================================== #
# ==================================================================================================== #

class Linear(ActivationFn):
    """Computing the Linear (Identity) function and its derivative.

        g(x) = x
    """

    def __init__(self):
        pass

    def __call__(self, x):
        return x

    def gradient(self, x):
        return 1

# ==================================================================================================== #

class Sigmoid(ActivationFn):
    """Computing the Sigmoid function and its derivative.

        g(x) = 1 / (1 + e^-x)
    """

    def __init__(self):
        pass

    def __call__(self, x):
        return np.vectorize(lambda k: 1.0/(1 + np.exp(-k)))(x)

    def gradient(self, x):
        a = self(x)
        return a * (1 - a)

# ==================================================================================================== #

class Softmax(ActivationFn):
    """Computing the numerical stable Softmax function and its derivative.

        g(vec) = e^vec[j] / sum(e^vec[k])
    """

    def __init__(self, axis=-1):
        self.axis = axis

    def __call__(self, x):
        exps = np.exp(x - np.max(x, axis=self.axis, keepdims=True))
        return exps / np.sum(exps, axis=self.axis, keepdims=True)

    def gradient(self, x):
        a = self(x)
        return a * (1 - a)

# ==================================================================================================== #

class TanH(ActivationFn):
    """Computing the TanH function and its derivative.

        g(x) = (e^x - e^-x) / (e^x + e^-x)
    """

    def __init__(self):
        pass

    def __call__(self, x):
        return np.tanh(x)

    def gradient(self, x):
        return 1 - self(x)**2

# ==================================================================================================== #

class ReLU(ActivationFn):
    """Computing the ReLU function and its derivative.

        g(x) = max(x, 0)
    """

    def __init__(self):
        pass

    def __call__(self, x):
        return np.vectorize(max)(0, x)

    def gradient(self, x):
        return (np.array(x) >= 0).astype(int)

# ==================================================================================================== #

class LeakyReLU(ActivationFn):
    """Computing the ReLU function and its derivative.

        g(x) = max(x, eps*x)
    """

    def __init__(self, eps=0.01):
        self.eps = eps

    def __call__(self, x):
        return np.vectorize(max)(self.eps*x, x)

    def gradient(self, x):
        grad = (np.array(x) >= 0).astype(int)
        grad[grad == 0] = self.eps
        return grad

# ==================================================================================================== #


from math import sqrt, exp

from scipy.stats import norm

from opfu.bsm import N, bsm_price, d1, d2

def N_d(x):
    return norm.pdf(x)

def delta(S0, K, r=0.01, sigma=0.1, T=1, ds=0, is_call=True):
    if ds == 0:
        # the theortical result
        if is_call:
            return N(d1(S0, K, r, sigma, T))
        else:
            return N(d1(S0, K, r, sigma, T)) - 1
    # approach
    p = bsm_price(S0 + ds / 2, K, r, sigma, T, is_call)
    m = bsm_price(S0 - ds / 2, K, r, sigma, T, is_call)
    return (p - m) / ds

def gamma(S0, K, r=0.01, sigma=0.1, T=1, ds=0, is_call=True):
    if ds == 0:
        return N_d(d1(S0, K, r, sigma, T)) / (S0 * sigma * sqrt(T))
    # approach
    p = delta(S0 + ds / 2, K, r, sigma, T, ds, is_call)
    m = delta(S0 - ds / 2, K, r, sigma, T, ds, is_call)
    return (p - m) / ds

def theta(S0, K, r=0.01, sigma=0.1, T=1, dt=0, is_call=True):
    if dt == 0:
        return -S0 * N_d(d1(S0, K, r, sigma, T)) * sigma / (2 * sqrt(T)) - r * K * exp(-r * T) * N(
            d2(S0, K, r, sigma, T)) \
            if is_call else \
            -S0 * N_d(d1(S0, K, r, sigma, T)) * sigma / (2 * sqrt(T)) + r * K * exp(-r * T) * N(-d2(S0, K, r, sigma, T))
    # approach
    p = bsm_price(S0, K, r, sigma, T + dt / 2, is_call)
    m = bsm_price(S0, K, r, sigma, T - dt / 2, is_call)
    return -(p - m) / dt

def vega(S0, K, r=0.01, sigma=0.1, T=1, dsigma=0, is_call=True):
    if dsigma == 0:
        return S0 * sqrt(T) * N_d(d1(S0, K, r, sigma, T))
    # approach
    p = bsm_price(S0, K, r, sigma + dsigma / 2, T, is_call)
    m = bsm_price(S0, K, r, sigma - dsigma / 2, T, is_call)
    return (p - m) / dsigma

def rho(S0, K, r=0.01, sigma=0.1, T=1, dr=0, is_call=True):
    if dr == 0:
        return K * T * exp(-r * T) * N(d2(S0, K, r, sigma, T)) if is_call else -K * T * exp(-r * T) * N(
            -d2(S0, K, r, sigma, T))
    # approach
    p = bsm_price(S0, K, r + dr / 2, sigma, T, is_call)
    m = bsm_price(S0, K, r - dr / 2, sigma, T, is_call)
    return (p - m) / dr


import numpy as np

# multipool_matrix is a function defining multipool matrices based on the Standard Transversal Design if
# the pool size is a prime number n
# the population size is N = n**2
# the multiplicity k is a number between 1 and n+1

def multipool_matrix(n,k):
	# check constraint k <= n+1
	if k > n+1:
		print "Error! k must not be larger than n+1."
		return
	# check constraint n prime if k > 3
	if k > 3:
		for i in range(2,n):
			if (n % i) == 0:
 				print("Error! If k > 3 then n must be a prime number.")
 				return

	A = np.zeros((n*k,n**2))

	for i in range(n):
		for j in range(n):
			A[i][i*n+j] = 1
	for l in range(1,k):
		for i in range(n):
			for j in range(n):
				A[l*n+i][n*j+((i+(l-1)*j)%n)] = 1
	return A


# -*- coding: utf-8 -*-
import functools

import numpy as np
import matplotlib.pyplot as plt

def meshgrid3d(X,Y):
    """
        input:
            X : (m,d) tensor
            Y : (n,d) tensor
        returns:
            Xd : (m x n x d) tensor
            Yd : (m x n x d) tensor
    """
    Xd = X.reshape(X.shape[0],1,X.shape[1])
    Yd = Y.reshape(1,Y.shape[0],Y.shape[1])
    Xd = np.tile(Xd,[1,Y.shape[0],1])
    Yd = np.tile(Yd,[X.shape[0],1,1])
    return Xd,Yd

def kernel_matrix(X,Y,k,symmetrize=False):
    Xd,Yd = meshgrid3d(X,Y)
    K = k(Xd,Yd,keepdims=False)
    if symmetrize:
        K = 0.5*(K + K.transpose(1,0))
    return K

def rbf_kernel_f(x,y,l,theta,keepdims=True):
    return theta*np.exp(-0.5*np.sum((x-y)**2/(l**2),axis=-1,
                                          keepdims=keepdims))

#%% Priors
theta = 1.0
l = 1.0
X = np.linspace(-2,2)
kfunc = functools.partial(rbf_kernel_f,l=l,theta=theta)
K = kernel_matrix(X.reshape(-1,1),X.reshape(-1,1),kfunc)
ncurves = 200
y = np.random.multivariate_normal(np.zeros(len(X)),K,ncurves)
for yy in y:
    plt.plot(X.flatten(),yy,'b--',alpha=0.2)
plt.fill_between(X,- 2*np.sqrt(np.diag(K)),
                    2*np.sqrt(np.diag(K)),
                   color='red',alpha=0.2)

plt.savefig("/home/danilo/Danilo/Dissertação/Tex/figs/gprex1a")
plt.show()
#%% Posterior
def f(x):
    return np.sin(np.pi*x)
xpoints = np.linspace(-0.1,1.5,5)
ypoints = f(xpoints)
Kxx = kernel_matrix(xpoints.reshape(-1,1),xpoints.reshape(-1,1),kfunc)
Kxx_inv = np.linalg.inv(Kxx)
Kplot = kernel_matrix(X.reshape(-1,1),xpoints.reshape(-1,1),kfunc)
Kplot2 = kernel_matrix(X.reshape(-1,1),X.reshape(-1,1),kfunc)
pos_mean = Kplot@Kxx_inv@(ypoints.reshape(-1,1))
pos_cov = Kplot2 - Kplot@Kxx_inv@Kplot.T
y2 = np.random.multivariate_normal(pos_mean.flatten(),pos_cov,ncurves)
plt.fill_between(X,pos_mean.flatten() - 2*np.sqrt(np.diag(pos_cov)),
                   pos_mean.flatten() + 2*np.sqrt(np.diag(pos_cov)),
                   color='red',alpha=0.2)
#plt.plot(X,pos_mean,'red',alpha=0.5)
for yy2 in y2:
    plt.plot(X.flatten(),yy2,'b--',alpha=0.2)
plt.plot(X,f(X),'black',linestyle='--',alpha=0.8)
plt.plot(xpoints,ypoints,'go')
plt.savefig("../tex/figs/gprex1b")
plt.show()


from __future__ import annotations

import math

import numpy
import numpy as np

class UnivariateGaussian:
    """
    Class for univariate Gaussian Distribution Estimator
    """

    def __init__(self, biased_var: bool = False) -> UnivariateGaussian:
        """
        Estimator for univariate Gaussian mean and variance parameters

        Parameters
        ----------
        biased_var : bool, default=False
            Should fitted estimator of variance be a biased or unbiased estimator

        Attributes
        ----------
        fitted_ : bool
            Initialized as false indicating current estimator instance has not been fitted.
            To be set as True in `UnivariateGaussian.fit` function.

        mu_: float
            Estimated expectation initialized as None. To be set in `UnivariateGaussian.fit`
            function.

        var_: float
            Estimated variance initialized as None. To be set in `UnivariateGaussian.fit`
            function.
        """
        self.biased_ = biased_var
        self.fitted_, self.mu_, self.var_ = False, None, None

    def fit(self, X: np.ndarray) -> UnivariateGaussian:
        """
        Estimate Gaussian expectation and variance from given samples

        Parameters
        ----------
        X: ndarray of shape (n_samples, )
            Training data

        Returns
        -------
        self : returns an instance of self.

        Notes
        -----
        Sets `self.mu_`, `self.var_` attributes according to calculated estimation (where
        estimator is either biased or unbiased). Then sets `self.fitted_` attribute to `True`
        """
        self.mu_ = X.mean()
        if self.biased_:
            self.var_ = X.var(ddof=0)
        else:
            self.var_ = X.var(ddof=1)

        self.fitted_ = True
        return self

    def get_mean(self):
        return self.mu_

    def get_var(self):
        return self.var_

    def pdf(self, X: np.ndarray) -> np.ndarray:
        """
        Calculate PDF of observations under Gaussian model with fitted estimators

        Parameters
        ----------
        X: ndarray of shape (n_samples, )
            Samples to calculate PDF for

        Returns
        -------
        pdfs: ndarray of shape (n_samples, )
            Calculated values of given samples for PDF function of N(mu_, var_)

        Raises
        ------
        ValueError: In case function was called prior fitting the model
        """
        if not self.fitted_:
            raise ValueError("Estimator must first be fitted before calling `pdf` function")
        pdfs = []
        for x in X:
            pdfs.append(UnivariateGaussian._calculate_pdf(x, self.mu_, self.var_))
        return pdfs

    @staticmethod
    def log_likelihood(mu: float, sigma: float, X: np.ndarray) -> float:
        """
        Calculate the log-likelihood of the data under a specified Gaussian model

        Parameters
        ----------
        mu : float
            Expectation of Gaussian
        sigma : float
            Variance of Gaussian
        X : ndarray of shape (n_samples, )
            Samples to calculate log-likelihood with

        Returns
        -------
        log_likelihood: float
            log-likelihood calculated
        """
        var_sum = 0
        const = (-1.0) / (2 * sigma)
        for x in X:
            var_sum += pow(x - mu, 2)
        return (- len(X) / 2) * (numpy.log(2 * math.pi) + numpy.log(sigma)) + (const * var_sum)

    @staticmethod
    def _calculate_pdf(x, mu, var):
        return (1.0 / (var * math.sqrt(2 * math.pi))) * math.exp(-0.5 * pow((x - mu) / var, 2))

class MultivariateGaussian:
    """
    Class for multivariate Gaussian Distribution Estimator
    """

    def __init__(self):
        """
        Initialize an instance of multivariate Gaussian estimator

        Attributes
        ----------
        fitted_ : bool
            Initialized as false indicating current estimator instance has not been fitted.
            To be set as True in `MultivariateGaussian.fit` function.

        mu_: ndarray of shape (n_features,)
            Estimated expectation initialized as None. To be set in `MultivariateGaussian.fit`
            function.

        cov_: ndarray of shape (n_features, n_features)
            Estimated covariance initialized as None. To be set in `MultivariateGaussian.fit`
            function.
        """
        self.mu_, self.cov_ = None, None
        self.fitted_ = False

    def fit(self, X: np.ndarray) -> MultivariateGaussian:
        """
        Estimate Gaussian expectation and covariance from given samples

        Parameters
        ----------
        X: ndarray of shape (n_samples, n_features)
            Training data

        Returns
        -------
        self : returns an instance of self

        Notes
        -----
        Sets `self.mu_`, `self.cov_` attributes according to calculated estimation.
        Then sets `self.fitted_` attribute to `True`
        """
        self.mu_ = np.mean(X, 0)
        self.cov_ = np.cov(X.T)
        self.fitted_ = True
        return self

    def get_mean(self):
        return self.mu_

    def get_cov(self):
        return self.cov_

    def pdf(self, X: np.ndarray):
        """
        Calculate PDF of observations under Gaussian model with fitted estimators

        Parameters
        ----------
        X: ndarray of shape (n_samples, n_features)
            Samples to calculate PDF for

        Returns
        -------
        pdfs: ndarray of shape (n_samples, )
            Calculated values of given samples for PDF function of N(mu_, cov_)

        Raises
        ------
        ValueError: In case function was called prior fitting the model
        """
        if not self.fitted_:
            raise ValueError("Estimator must first be fitted before calling `pdf` function")
        pdfs = []
        for x in X:
            pdfs.append(MultivariateGaussian._calculate_pdf(x, self.mu_, self.cov_))
        return pdfs

    @staticmethod
    def _calculate_pdf(x, mu, cov):
        d = float(len(x))
        det = numpy.linalg.det(cov)
        if det == 0:
            raise ValueError("The covariance matrix has to be invertible")
        inverted = np.linalg.inv(cov)
        distance_x_mu = ((x - mu).T.dot(inverted)).dot(x - mu)
        return (1.0 / math.sqrt(pow(2 * math.pi, d) * det)) * math.exp(-0.5 * distance_x_mu)

    @staticmethod
    def _calculate_likelihood(x, cov_det, k, inverted):
        return -0.5 * (np.log(cov_det) + x.T @ inverted @ x + k * np.log(2 * math.pi))

    @staticmethod
    def log_likelihood(mu: np.ndarray, cov: np.ndarray, X: np.ndarray) -> float:
        """
        Calculate the log-likelihood of the data under a specified Gaussian model

        Parameters
        ----------
        mu : ndarray of shape (n_features,)
            Expectation of Gaussian
        cov : ndarray of shape (n_features, n_features)
            covariance matrix of Gaussian
        X : ndarray of shape (n_samples, n_features)
            Samples to calculate log-likelihood with

        Returns
        -------
        log_likelihood: float
            log-likelihood calculated over all input data and under given parameters of Gaussian
        """
        v = X - mu
        inverted = np.linalg.inv(cov)
        k = mu.shape[0]
        cov_det = np.linalg.det(cov)
        log_likelihood = np.apply_along_axis(
            lambda x: MultivariateGaussian._calculate_likelihood(x, cov_det, k, inverted), 1, v)
        return log_likelihood.sum()


#
#
# Regression and Classification programming exercises
#
#

#
#	In this exercise we will be taking a small data set and computing a linear function
#	that fits it, by hand.
#

#	the data set

import numpy as np

sleep = [5, 6, 7, 8, 10]
scores = [65, 51, 75, 75, 86]

def compute_regression(sleep, scores):

    #	First, compute the average amount of each list
    avg_sleep = np.mean(sleep)
    avg_scores = np.mean(scores)
    print avg_sleep, avg_scores

    #	Then normalize the lists by subtracting the mean value from each entry
    normalized_sleep = np.array(sleep) - avg_sleep
    normalized_scores = np.array(scores) - avg_scores
    print normalized_sleep
    print normalized_scores

    #	Compute the slope of the line by taking the sum over each student
    #	of the product of their normalized sleep times their normalized test score.
    #	Then divide this by the sum of squares of the normalized sleep times.

    slope = np.sum(normalized_sleep * normalized_scores) / np.sum(np.square(normalized_sleep))
    print slope

    #	Finally, We have a linear function of the form
    #	y - avg_y = slope * ( x - avg_x )
    #	Rewrite this function in the form
    #	y = m * x + b
    #	Then return the values m, b

    return slope, avg_scores - (avg_sleep * slope)

#
#	Polynomial Regression
#
#	In this exercise we will examine more complex models of test grades as a function of
#	sleep using numpy.polyfit to determine a good relationship and incorporating more data.
#
#
#   at the end, store the coefficients of the polynomial you found in coeffs
#

def polyRegression():
    sleep = [5,6,7,8,10,12,16]
    scores = [65,51,75,75,86,80,0]
    coeffs = np.polyfit(sleep, scores, 2)
    return coeffs

if __name__ == "__main__":
    m, b = compute_regression(sleep, scores)
    print "Your linear model is y={}*x+{}".format(m, b)
    print "Polynomial regression: ", polyRegression()


# -*- coding: utf-8 -*-
"""
Gaussian Error Linear Unit (GELU)
This module is implementation of "GAUSSIAN ERROR LINEAR UNITS (GELUS)"
(https://arxiv.org/abs/1606.08415)
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import numpy as np
import keras.backend as K

def gelu(x, approximate='tanh'):
    """Gaussian Error Linear Unit.
    (implementation of https://arxiv.org/abs/1606.08415)

    # Arguments
        x: Input tensor.
        approximate: Approximation method of GELU.
            "tanh" or "sigmoid" (defaults to "tanh".)

    # Returns
        The gaussian error unit activation: x * Phi(x)
            where Phi(x) = P(X <= x), X ~ Normal(0, 1)
            (approximated by "tanh" or "sigmoid")
    """
    if not approximate in ('tanh', 'sigmoid'):
        raise ValueError('approximate must be in ("tanh", "sigmoid")')
    if approximate == 'tanh':
        return 0.5*x * (1 + K.tanh(np.sqrt(2/np.pi) * \
                                   (x + 0.044715*K.pow(x, 3))))
    else:
        return x * K.sigmoid(1.702*x)


#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Tue Jan 29 19:41:58 2019

@author: alankar
"""
import numpy as np

def func(z):
    z1, z2 = z[0], z[1]
    equations = []
    equations.append( z2**2*(1-z1) - z1**3 )
    equations.append( z1**2 + z2**2 -1 )
    return np.array(equations).T

def jac(z):
    z1, z2 = z[0], z[1]
    J = np.array( [
                  [-z2**2-3*z1**2,    2*z2*(1-z1)],
                  [ 2*z1,                2*z2] ])
    return J

tol = 1e-6

z_new = np.array([-1.6+0.j,0.8+0.j])
z_old = np.array([0.+0.j,0.+0.j])

while( np.abs(z_old[0]-z_new[0])>tol or np.abs(z_old[1]-z_new[1])>tol ):
    z_old = z_new
    equations = func(z_old)
    delz = np.linalg.solve(jac(z_old),-equations)
    z_new = z_old + delz.T

print('Roots are: ')
for i in range(len(z_new)):
    print('z%d = '%(i+1),end='')
    if (np.abs(z_new[i].real) >=tol): print(" %.2f"%z_new[i].real,end='')
    if((np.abs(z_new[i].real) >=tol) and (np.abs(z_new[i].imag) >=tol)): print(' + ',end='')
    if (np.abs(z_new[i].imag) >=tol): print("%.2f i"%z_new[i].imag)
    print('')

"""
Output

Roots are:
z1 =  -1.62
z2 = 1.27 i
"""


import math
import cmath
import numpy as np
import matplotlib.pyplot as plt
import datetime
import time

def main():

    sair = False
    while(not sair):

        menu = int(input("\n\nSelecione vizualizar testes de transformada(1) Testes de dados(2) ou Resolução do problema proposto (3): "))

        if menu == 1:

            menu_teste = int(input("\n\nSelecione teste a) (1), b) (2) ou c) (3): "))

            if menu_teste == 1:

                print("\n\nTeste a) f = [5,-1,3,1]",end="\n\n")
                f = [5,-1,3,1]

                print("Implementação ineficiente da Transformada de Fourier direta",end="\n")
                c1 = fft_direta(f)
                print_vector(c1)

                print("Implementação recursiva da FFT direta",end="\n")
                c = [0,0,0,0]
                fftrec(c,f,2,True)
                print_vector(np.divide(c,4))

                print("\n\nFazendo c = (2, 1/2 + i/2, 2, 1/2 − i/2)",end="\n\n")
                c = [complex(2,0),complex(0.5,0.5),complex(2,0),complex(0.5,-0.5)]

                print("Implementação ineficiente da Transformada de Fourier inversa",end="\n")
                f1 = fft_inversa(c)
                print_vector(f1)

                print("Implementação recursiva da FFT inversa",end="\n")
                f = [0,0,0,0]
                fftrec(f,c,2,False)
                print_vector(f)

            if menu_teste == 2:

                print("\n\nTeste b) f = [6, 2, 5, 2, 11, 2, 8, 8]",end="\n\n")
                f = [6, 2, 5, 2, 11, 2, 8, 8]

                print("Implementação ineficiente da Transformada de Fourier direta",end="\n")
                c1 = fft_direta(f)
                print_vector(c1)

                print("Implementação recursiva da FFT direta",end="\n")
                c = [0,0,0,0,0,0,0,0]
                fftrec(c,f,4,True)
                print_vector(np.divide(c,8))

                print("Implementação recursiva da FFT direta pelo numpy (rfft)",end="\n")
                print("Note que são mostrados apenas os coeficientes de frequencias positivas pois o input é real")
                c2 = np.fft.rfft(f)
                print_vector(np.divide(c2,8))

                print("Implementação ineficiente da Transformada de Fourier inversa",end="\n")
                f1 = fft_inversa(c1)
                print_vector(f1)

                print("Implementação recursiva da FFT inversa",end="\n")
                f3 = [0,0,0,0,0,0,0,0]
                fftrec(f3,np.divide(c,8),4,False)
                print_vector(f3)

                print("Implementação recursiva da FFT inversa pelo numpy (rfft)",end="\n")
                f2 = np.fft.irfft(c2)
                print_vector(f2)

            if menu_teste == 3:

                print("\n\nTeste b) F(x) = 10sin(x) + 7cos(30x) + 11sin(352x) − 8cos(711x) com xj = jπ/512, j = 0, 1023",end="\n\n")
                dois_n = 1024
                f = [ 0 for j in range(dois_n)]
                x = [j*math.pi/(dois_n/2) for j in range(dois_n)]
                f = [10*math.sin(x[j])+7*math.cos(30*x[j])+11*math.sin(352*x[j])-8*math.cos(711*x[j]) for j in range(dois_n)]

                print("Implementação ineficiente da Transformada de Fourier direta",end="\n")
                c1 = fft_direta(f)

                print("Implementação recursiva da FFT direta",end="\n")
                c = [0 for i in range(dois_n)]
                fftrec(c,f,int(dois_n/2),True)

                print("Implementação recursiva da FFT direta pelo numpy (rfft)",end="\n")
                print("Note que são mostrados apenas os coeficientes de frequencias positivas pois o input é real")
                c2 = np.fft.rfft(f)

                print("Implementação ineficiente da Transformada de Fourier inversa",end="\n")
                f1 = fft_inversa(c1)

                print("Implementação recursiva da FFT inversa",end="\n")
                f3 = [0 for i in range(dois_n)]
                fftrec(f3,np.divide(c,dois_n),int(dois_n/2),False)

                print("Implementação recursiva da FFT inversa pelo numpy (rfft)",end="\n")
                f2 = np.fft.irfft(c2)

                plt.figure(1)
                ax1 = plt.subplot(231)
                t = [j for j in range(int(dois_n))]
                plt.plot(t, np.absolute(c1), 'bo')
                ax1.set_title('TF Ineficiente')
                k = [j for j in range(dois_n)]
                plt.subplot(234)
                plt.plot(k, f1, 'r-')

                ax2 = plt.subplot(232)
                t = [j for j in range(int(dois_n))]
                plt.plot(t, np.absolute(np.divide(c,dois_n)), 'bo')
                ax2.set_title('FFT recursiva')
                k = [j for j in range(dois_n)]
                plt.subplot(235)
                plt.plot(k, f3, 'r-')

                ax3 = plt.subplot(233)
                t = [j for j in range(int(dois_n/2+1))]
                plt.plot(t, np.absolute(np.divide(c2,dois_n)), 'bo')
                ax3.set_title('FFT numpy')
                k = [j for j in range(dois_n)]
                plt.subplot(236)
                plt.plot(k, f2, 'r-')

                plt.show()

        if menu == 2:

            Alum = read_file("Alum.csv")
            nome = 'Aluminio'
            print("===============================")
            tipo_tf = int(input("Selecione transformada Lenta N2(0), recursiva implementada(1) ou numpy(2): "))
            tipo_filtro = int(input("Selecione filtro passa baixas(0) ou passa altas(1): "))
            K = int(input("Digite o indice K de corte do filtro: "))
            print("===============================")

            Alum_preco = [x[2] for x in Alum]

            if tipo_tf == 0:
                timedir,timeinv = plotComTF_Lenta(Alum_preco, nome, K, tipo_filtro)
            if tipo_tf == 1:
                timedir,timeinv = plotComTF_rec(Alum_preco, nome, K, tipo_filtro)
            if tipo_tf == 2:
                timedir,timeinv = plotComTF_numpy(Alum_preco, nome, K, tipo_filtro)

            print("===============================")
            print("\n\nTempo decorrido para transformada direta: %f ms" %(timedir))
            print("\nTempo decorrido para transformada inversa: %f ms" %(timeinv))
            print("\nDesvio Padrao da serie temporal: %f" %(desvPad(Alum)))
            print("===============================")

        if menu == 3:

            Cattle = read_file("Cattle.csv")

            for i in range(30):
                print(Cattle[i])

        sair = bool(int(input("Sair: 1 Continuar: 0\n")))

def fft_inversa(c):
    n = int(len(c)/2)
    f = [0 for i in range(2*n)]

    for j in range(0,2*n):

        for k in range(0,2*n):

            f[j] += c[k]*cmath.exp(complex(0,k*j*math.pi/n))

    return f

def fft_direta(f):
    n = int(len(f)/2)
    c = [0 for i in range(2*n)]

    for k in range(0,2*n):

        for j in range(0,2*n):

            c[k] += (f[j]*cmath.exp(complex(0,-k*j*math.pi/n)))/(2*n)

    return c

def fftrec(c,f,n,direto):
    fe = [0 for i in range(2*n)]
    fo = [0 for i in range(2*n)]
    even = [c[2*j] for j in range(n)]
    odd = [c[2*j+1] for j in range(n)]

    if n == 1:

        c[0] = f[0]+f[1]
        c[1] = f[0]-f[1]

    else:

        for j in range(n):

            fe[j] = f[2*j]
            fo[j] = f[2*j+1]

        fftrec(even,fe,int(n/2),direto)
        fftrec(odd,fo,int(n/2),direto)

        for j in range(n):

            if direto:
                eij = cmath.exp(complex(0,-j*math.pi/n))

            else:
                eij = cmath.exp(complex(0,j*math.pi/n))

            c[j] = even[j] + eij*odd[j]
            c[j+n] = even[j] - eij*odd[j]

#=======================================================================

def print_vector(vetor):
    ''' matriz -> null '''
    for k in range(45):
        print("=", end="")
    print("",end="\n")
    for n in range (0,len(vetor),1):
        print ("| (%.5f, %.5fi)" %(vetor[n].real, vetor[n].imag), end="")
    print("|")
    for k in range(45):
        print("=", end="")
    print("", end="\n\n")
    return

#=======================================================================

def read_file(file):
    '''
        Lê arquivo
    '''
    f = open(file)
    arquivo = [x.strip('\n') for x in f.readlines()]
    arquivo = [x.split(', ') for x in arquivo]

    for l in range(len(arquivo)):
        for i in range(3):
            if i == 0:
                arquivo[l][i] = datetime.date(int(arquivo[l][i][0:4]),
                                              int(arquivo[l][i][4:6]),
                                              int(arquivo[l][i][6:9]))
            if i == 1:
                arquivo[l][i] = int(arquivo[l][i])

            if i == 2:
                arquivo[l][i] = float(arquivo[l][i])

    return arquivo

#=======================================================================

def mean(numbers):
    '''
        Calcula media de vetor
    '''
    return float(sum(numbers)) / max(len(numbers), 1)

#=======================================================================

def desvPad(file):
    '''
        Calcula desvio padrao
    '''

    valores = [x[2] for x in file]
    media = mean(valores)
    argsum = np.divide(np.power(np.add(valores,-media),2),len(valores))
    desvPad = math.sqrt( sum( argsum ) )

    return float(desvPad)

#=======================================================================

def passa_baixa(c,K):
    c2 = [c[j] if j < K-1 else 0 for j in range(len(c))]
    return c2

def passa_alta(c,K):
    c2 = [c[j] if j >= K-1 else 0 for j in range(len(c))]
    return c2

def passa_baixa2(c,K):
    tam = len(c)
    c2 = [0 for j in range(tam)]
    for j in range(tam):
        if (j < K-1) or (j > tam-K-1 ):
            c2[j] = c[j]
    return c2

def passa_alta2(c,K):
    tam = len(c)
    c2 = [0 for j in range(tam)]
    for j in range(tam):
        if (j >= K-1) and (j <= tam-K-1 ):
            c2[j] = c[j]
    return c2

#=======================================================================

def plotComTF_Lenta(valor, nome, K, tipo_filtro):
    '''
        valor: vetor de floats
        nome: string
        K: inteiro
        tipo_filtro: inteiro
    '''
    print("\n\nNote que a frequencia de indice 0 está omitida do plot")
    time_dir_st = time.time()
    c = fft_direta(valor)
    time_dir_end = time.time()
    time_dir = (time_dir_end - time_dir_st)*1000

    plt.figure(1)
    ax1 = plt.subplot(211)
    t = [j for j in range(1,len(valor))]
    plt.plot(t, np.absolute(c)[1:len(c)], 'bo')
    ax1.set_title(nome)
    plt.subplot(212)
    k = [j for j in range(len(valor))]
    plt.plot(k, valor, 'b-')

    if tipo_filtro == 0:
        c_filtrado = passa_baixa2(c,K)

    if tipo_filtro == 1:
        c_filtrado = passa_alta2(c,K)

    time_inv_st = time.time()
    f_filtrado = fft_inversa(c_filtrado)
    time_inv_end = time.time()
    time_inv = (time_inv_end - time_inv_st)*1000

    plt.subplot(212)
    plt.plot(k, np.real(f_filtrado), 'r-')
    plt.show()

    return (time_dir,time_inv)

#=======================================================================

def plotComTF_rec(valor, nome, K, tipo_filtro):
    '''
        valor: vetor de floats
        nome: string
        K: inteiro
        tipo_filtro: inteiro
    '''
    print("\n\nNote que a frequencia de indice 0 está omitida do plot")
    c = [0 for i in range(len(valor))]
    time_dir_st = time.time()
    fftrec(c,valor,int(len(valor)/2),True)
    time_dir_end = time.time()
    time_dir = (time_dir_end - time_dir_st)*1000

    plt.figure(1)
    ax1 = plt.subplot(211)
    t = [j for j in range(1,len(valor))]
    plt.plot(t, np.absolute(c)[1:len(c)], 'bo')
    ax1.set_title(nome)
    plt.subplot(212)
    k = [j for j in range(len(valor))]
    plt.plot(k, valor, 'b-')

    if tipo_filtro == 0:
        c_filtrado = passa_baixa2(c,K)

    if tipo_filtro == 1:
        c_filtrado = passa_alta2(c,K)

    valor_filtrado = [0 for i in range(len(valor))]
    time_inv_st = time.time()
    fftrec(valor_filtrado,np.divide(c_filtrado,len(c_filtrado)),int(len(valor)/2),False)
    time_inv_end = time.time()
    time_inv = (time_inv_end - time_inv_st)*1000

    plt.subplot(212)
    plt.plot(k, np.real(valor_filtrado), 'r-')
    plt.show()

    return (time_dir,time_inv)

#=======================================================================

def plotComTF_numpy(valor, nome, K, tipo_filtro):
    '''
        valor: vetor de floats
        nome: string
        K: inteiro
        tipo_filtro: inteiro
    '''
    print("\n\nNote que a frequencia de indice 0 está omitida do plot")
    time_dir_st = time.time()
    c = np.fft.rfft(valor)
    time_dir_end = time.time()
    time_dir = (time_dir_end - time_dir_st)*1000

    plt.figure(1)
    ax1 = plt.subplot(211)
    t = [j for j in range(1,int(len(valor)/2)+1)]
    plt.plot(t, np.absolute(c)[1:], 'bo')
    ax1.set_title(nome)
    k = [j for j in range(len(valor))]
    plt.subplot(212)
    plt.plot(k, valor, 'b-')

    if tipo_filtro == 0:
        c_filtrado = passa_baixa(c,K)

    if tipo_filtro == 1:
        c_filtrado = passa_alta(c,K)

    time_inv_st = time.time()
    valor_filtrado = np.fft.irfft(c_filtrado)
    time_inv_end = time.time()
    time_inv = (time_inv_end - time_inv_st)*1000

    plt.subplot(212)
    plt.plot(k, valor_filtrado, 'r-')
    plt.show()

    return (time_dir,time_inv)


"""
Linear Regression Algorithm
David Felipe Alvear Goyes
Artificial Intelligence Course 2020 - I
Columbia University
04/2020
"""

import sys
import pandas as pd
import numpy as np
import sklearn
import matplotlib.pyplot as plt

class linear_regression(object):
    def __init__(self, data, labels, lr, iterations):
        self.data = data
        self.labels = labels
        self.lr = lr
        self.iterations = iterations

    def run(self):
        betas = np.zeros(self.data.shape[1])
        log_betas = []

        # iterate over all the learning rates
        for lrate in self.lr:
            counter = 0
            betas = np.zeros(self.data.shape[1]) # reset betas

            # Gradient descent for the given learning rate
            for counter in range(self.iterations+1):
                betas = self.gradient_descent(self.data, self.labels, betas, lrate)

            log_betas.append([lrate, counter, betas[0], betas[1], betas[2]])

        return log_betas

    def gradient_descent(self, data, labels, betas, lr):
        # Calculate the derivative of the risk function
        functionx = np.dot(data, betas)
        temp1 = (functionx[:,None]-labels).flatten()
        loss = (data*temp1[:,None]).sum(axis=0)
        dRisk = lr * loss / (data.shape[0])

        # Update betas
        betas = betas - dRisk
        return betas

class data_utils(object):
    def __init__(self, input, output):
        self.input_string = input
        self.output = output

    def load_data(self):
        """
            load_data - Read csv file with the linear dataset
            - input_string : name of the csv file to read
            - return array with the data loaded
        """
        input_data = pd.read_csv(self.input_string, sep=',',header=None)

        return input_data.values

    def normalization(self, data):
        """
        Data - age, weight, height
        return - normalized data
        """
        # Nomrmalize the data
        #mean = np.mean(data, axis=0) #[age_m, we_m, hei_m]
        mean = data.mean(axis=0)
        # Standard deviation
        #std = np.std(data, axis=0) #[age_std, we_std, hei_std]
        std = data.std(axis=0)
        data = (data - mean[None,:]) / std[None,:]
        return data

    def output_file(self, list_betas):
        pd.DataFrame(list_betas).to_csv(self.output, header=None, index=None)

def main():
    ## First - load the input1.csv
    sm = sys.argv[1].lower()
    input_name = sys.argv[1].split(" ")[0]
    output_name = sys.argv[2].split(" ")[0]

    lrList = [0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10, 0.67]
    #iterationMaxList = [100, 100, 100, 100, 100, 100, 100, 100, 100, 58]

    data_manager = data_utils(input_name, output_name)
    raw_data = data_manager.load_data()
    data = data_manager.normalization(raw_data[:,:-1]) # Normalize data

    #data = data[:,:-1] # extract X inputs from the data
    labels = raw_data[:,-1:] # extract labels from the data
    datax = np.c_[np.ones(raw_data.shape[0]), data] # add one column at end for w0

    # linearRegression
    linearRegression = linear_regression(datax, labels, lrList, 100)
    log_betas = linearRegression.run()

    ## save results
    data_manager.output_file(log_betas)
    #data_maneger.graph(raw_data, final_weights)

if __name__ == '__main__':
    main()


import math
import numpy as np

def cal_quartile(arr):
    # 计算四分位的各个数值
    sortedArr = np.sort(arr)  # small --> big
    num = arr.size

    Q1_index = (num + 1) / 4
    if (num + 1) % 4 == 0:
        Q1 = sortedArr[int(Q1_index)]
    else:
        decimal_part = Q1_index - math.floor(Q1_index)
        if decimal_part < 0.5:
            Q1 = 0.75 * sortedArr[int(Q1_index)] + 0.25 * sortedArr[int(Q1_index) + 1]
        else:
            Q1 = 0.25 * sortedArr[int(Q1_index)] + 0.75 * sortedArr[int(Q1_index) + 1]

    Q2_index = (num + 1) / 2
    if (num + 1) % 2 == 0:
        Q2 = sortedArr[int(Q2_index)]
    else:
        decimal_part = Q2_index - math.floor(Q2_index)
        if decimal_part < 0.5:
            Q2 = 0.75 * sortedArr[int(Q2_index)] + 0.25 * sortedArr[int(Q2_index) + 1]
        else:
            Q2 = 0.25 * sortedArr[int(Q2_index)] + 0.75 * sortedArr[int(Q2_index) + 1]

    Q3_index = (num + 1) * 3 / 4
    if (num + 1) * 3 % 4 == 0:
        Q3 = sortedArr[int(Q3_index)]
    else:
        decimal_part = Q3_index - math.floor(Q3_index)
        if decimal_part < 0.5:
            Q3 = 0.75 * sortedArr[int(Q3_index)] + 0.25 * sortedArr[int(Q3_index) + 1]
        else:
            Q3 = 0.25 * sortedArr[int(Q3_index)] + 0.75 * sortedArr[int(Q3_index) + 1]

    # IQR: (Inter-Quartile Range) 在统计中叫内距.内距又称为四分位差
    IQR = Q3 - Q1

    up_boundary = Q3 + 1.5 * IQR
    low_boundary = Q1 - 1.5 * IQR
    return [low_boundary, Q1, Q2, Q3, up_boundary]


# -*- coding: utf-8 -*-

import numpy as np
import matplotlib.pyplot as plt

def rectangle(f, a, b, N):
    h = (b-a) / N
    d = np.arange(a, b, h) + h/2
    return h * np.sum(f(d))

def trapez(f, a, b, N):
    h = (b-a) / N
    y = f(np.arange(a, b+h, h))
    y[0] /= 2
    y[-1] /= 2
    return h * np.sum(y)

def simpson(f, a, b, N):
    h = (b-a) / N
    d = np.arange(a, b+h, h)
    ar = f(d[0]) + f(d[-1])
    au = np.sum(f(d[1:-1:2]))
    ag = np.sum(f(d[2:-2:2]))
    return h * (ar + 4*au + 2*ag) / 3

test_functions = {
    'x^3 - x^2' : (lambda x: x**3-x**2,
                   lambda x: x**4/4-x**3/3),
    'sin^2 x'   : (lambda x: np.sin(x)**2,
                   lambda x: x/2-np.sin(x)*np.cos(x)/2),
    # divide by zero in [0, 1] (log(0), duh...)
    #  'x * log(x)': (lambda x: x*np.log(x),
    #                 lambda x: x**2*np.log(x)/2-x**2/4),
    'x * e^{-x}': (lambda x: x*np.exp(-x),
                   lambda x: -(x+1)*np.exp(-x)),
    'fn' : (lambda x: np.exp(-x)*np.sin(x),
            lambda x: -np.exp(-x)*np.sin(x)/2-np.exp(-x)*np.cos(x)/2)
}

A, B = 0., 1.
#  M = 25
#  Ns = np.arange(2, 2*M+2, 2)
Ns = 2**np.arange(0, 13)
M = len(Ns)
L = len(test_functions)

res = {
    'rectangle' : (rectangle, np.zeros((M, L)), np.zeros(M)),
    'trapez'    : (trapez, np.zeros((M, L)), np.zeros(M)),
    'simpson'   : (simpson, np.zeros((M, L)), np.zeros(M))
}

fig, ax = plt.subplots()
ax.set_xscale('log')
ax.set_yscale('log')

for ni, (integ, resarr, errarr) in res.items():
    for i, (nf, (f, F)) in enumerate(test_functions.items()):
        resarr[:,i] = np.array([integ(f, A, B, n) for n in Ns])
        print(ni, nf, np.min(resarr[:,i]))
        resarr[:,i] = np.abs(resarr[:,i] - (F(B)-F(A)))
    errarr = np.mean(resarr, axis=1)
    ax.plot(Ns, errarr, label=ni)

plt.legend()


import numpy as np

from .utils import nans_in_same_positions

def se(arr1: np.ndarray, arr2: np.ndarray) -> np.ndarray:
    """
    Computes SE – squared error
    """
    return (arr1 - arr2) ** 2

def ae(arr1: np.ndarray, arr2: np.ndarray) -> np.ndarray:
    """
    Computes AE – absolute error
    """
    return np.abs(arr1 - arr2)

def is_(y: np.ndarray, u: np.ndarray, l: np.ndarray, a=0.05) -> np.ndarray:
    """
    Computes IS (Interval Score), used in M4 competition.

    :param y: predicted mean values
    :param u: prediction interval upper bound
    :param l: prediction interval lower bound
    :return: IS
    """
    assert nans_in_same_positions(y, u, l), "Arrays have NaNs in different positions"

    ml = u - l

    # NaNs can be in the dataset and this will cause lt/gt comparisons to raise a warning
    with np.errstate(invalid="ignore"):
        p1 = 2 / a * np.where(y < l, l - y, 0)
        p2 = 2 / a * np.where(y > u, y - u, 0)

    return ml + p1 + p2

def mse(arr1: np.ndarray, arr2: np.ndarray) -> np.ndarray:
    return se(arr1, arr2).mean()

def mae(arr1: np.ndarray, arr2: np.ndarray) -> np.ndarray:
    return ae(arr1, arr2).mean()

def mis(y: np.ndarray, u: np.ndarray, l: np.ndarray, a=0.05) -> np.ndarray:
    """
    Computes MIS (Mean Interval Score), used in M4 competition. Lower = better.

    Resources:
        - https://www.m4.unic.ac.cy/wp-content/uploads/2018/03/M4-Competitors-Guide.pdf
        - https://stats.stackexchange.com/questions/194660/forecast-accuracy-metric-that-involves-prediction-intervals

    :param y: predicted mean values
    :param u: prediction interval upper bound
    :param l: prediction interval lower bound
    :return: IS
    """
    return is_(y, u, l, a=a).mean()

def smape(a, b):
    """
    From the M4 repo

    Calculates sMAPE
    :param a: actual values
    :param b: predicted values
    :return: sMAPE
    """
    a = np.reshape(a, (-1,))
    b = np.reshape(b, (-1,))
    return np.mean(2.0 * np.abs(a - b) / (np.abs(a) + np.abs(b))).item()

def mase(insample, y_test, y_hat_test, freq):
    """
    From the M4 repo

    Calculates MAsE
    :param insample: insample data
    :param y_test: out of sample target values
    :param y_hat_test: predicted values
    :param freq: data frequency
    :return:
    """
    y_hat_naive = []
    for i in range(freq, len(insample)):
        y_hat_naive.append(insample[(i - freq)])

    masep = np.mean(abs(insample[freq:] - y_hat_naive))

    return np.mean(abs(y_test - y_hat_test)) / masep


#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Wed Dec 16 13:50:20 2020

@author: joaovitor
"""

import time
import multiprocessing as mp
import numpy as np
from scipy import signal as ss
from matplotlib import pyplot as plt
#import sounddevice as sd

# 1. Variáveis e álgebra
fs = 44100  # taxa de amostragem [Hz]
dt = 1/fs  # intervalo de tempo entre amostras [s]
nsamp = 2**9  # número total de amostras [-]
tlen = (nsamp - 1) / fs  # duração do sinal [s]

print (f"{fs=}    {dt=}\n{nsamp=}    {tlen=}")

b = 0  # Número de semitons  [-]
c = 343  # Velocidade de propagação do som no ar [m/s]
f = 440 * 2**(b/12)  # frequência [Hz]
T = 1/f  #  Período [s]
l = c / f  # comprimento de onda [m]

print(f"{c=}    {f=}\n{T=}    {l=}")

omega = 2 * np.pi * f  # frequencia angular [rad/s]
k = 2 * np.pi / l  # número de onda [rad/m]
kl = 1/l

print (f"{omega=}    {k=}    {kl=}")

nmark = 2 * nsamp  # número de marcações no espaço [-]
dx = 1 / nmark  # intervalo entre as marcações [m]

print (f"{nmark=}    {dx=}")

xm = np.linspace(0, 1 - dx, nmark)  # [m] marcações no espaço
ts = np.linspace(0, tlen, nsamp)  # [s] instantes de tempo

cosx = np.exp(k * xm * 1j)
cost = np.exp(-omega * ts * 1j)
"""
Explique o que está sendo calculado nas linhas 59 e 60:

    Escreva aqui

"""

print (f"{xm.shape=}    {cosx.shape=}")
print (f"{ts.shape=}    {cost.shape=}")

fig1, axs = plt.subplots(2, 1)

xlines = []
xlines.append(*axs[0].plot(xm, np.real(cosx)))
xlines.append(*axs[0].plot(xm, np.imag(cosx)))
xlines.append(axs[0].hlines(np.abs(cosx)/(2**0.5), xm[0], xm[-1]))

tlines = []
tlines.append(*axs[1].plot(ts, np.real(cost)))
tlines.append(*axs[1].plot(ts, np.imag(cost)))
tlines.append(axs[1].hlines(np.abs(cost)/(2**0.5), ts[0], ts[-1]))

axs[0].set_title("Gráfico de Re{exp(jkx)} variando no tempo")
axs[0].set_xlabel("Posição [m]")
axs[0].set_ylabel("Amplitude [-]")

axs[1].set_title("Gráfico de Re{exp(-jωt)} variando no espaço")
axs[1].set_xlabel("Tempo [s]")
axs[1].set_ylabel("Amplitude [-]")

fig1.tight_layout()
fig1.canvas.draw()
fig1.show()

def loop(fig1, axs, xlines, tlines, cosx, cost, nsamp):
    from time import time
    tstart = time()
    for n in range(nsamp):
        extn = cosx * cost[n]
        exnt = cosx[2*n] * cost

        xlines[0].set_ydata(np.real(extn))
        xlines[1].set_ydata(np.imag(extn))
        tlines[0].set_ydata(np.real(exnt))
        tlines[1].set_ydata(np.imag(exnt))

        axs[0].draw_artist(axs[0].patch)
        [axs[0].draw_artist(xline) for xline in xlines]

        axs[1].draw_artist(axs[1].patch)
        [axs[1].draw_artist(tline) for tline in tlines]

        fig1.canvas.update()
        fig1.canvas.flush_events()
    tend = time()
    return tend - tstart

tloop = loop(fig1, axs, xlines, tlines, cosx, cost, nsamp)
print(f"{tloop=:.6f}")

def rms(x: np.ndarray):
    return (x**2).mean(axis=0)**0.5

def dB(x: np.ndarray):
    return 20*np.log10(rms(x))

print(f"{rms(np.real(cosx))=:.3f}  {dB(np.real(cosx))=:.1f}")
print(f"{rms(np.real(cost))=:.3f}  {dB(np.real(cost))=:.1f}")

cos_xt = np.zeros((xm.size, ts.size), dtype='complex64')

print (f"{cos_xt.shape=}")

print(f"começo do cálculo de exp(jkx - jωt)")

tstart = time.time()

for n, x in enumerate(xm):

    cos_xt[n, :] = np.exp(1j*k*x - 1j*omega*ts)
    # print(f"{n=}  {x=:.2f}  {dB(np.real(cos_xt[n, :]))=:.1f}",
    #       end=('\t' if (n+1) % 3 else '\n'))  # operador ternário
    #            # expressão diz que:
    #            # Se a sobra inteira da divisão de n por três  for diferente de zero,
    #            # a mensagem deve terminar com uma tabulação.
    #            # Senão, termina com nova linha.

tend = time.time()

print(f"Complete in {tend - tstart:.6f} seconds.")

fig2, ax = plt.subplots(1, 1)
im = ax.pcolormesh(ts, xm, np.real(cos_xt), cmap='jet')

ax.set_title("Gráfico de cos(kx - ωt)")
ax.set_ylabel("Posição [m]")
ax.set_xlabel("Tempo [s]")
fig2.colorbar(im)

fig2.tight_layout()
fig2.show()

fig3, ax3 = plt.subplots(1, 1)
im3 = ax3.pcolormesh(ts, xm, np.imag(cos_xt), cmap='jet')

ax3.set_title("Gráfico de cos(kx - ωt)")
ax3.set_ylabel("Posição [m]")
ax3.set_xlabel("Tempo [s]")
fig3.colorbar(im3)

fig3.tight_layout()
fig3.show()

input("'Enter' para sair.")


'''This script implements a Bayesian network for the Monty Hall problem.
   See http://staff.utia.cas.cz/vomlel/mh-puzzle.html for a description'''

import numpy as np
from pomegranate import *

# The guests initial door selection is completely random
guest = DiscreteDistribution( { 'A': 1./3, 'B': 1./3, 'C': 1./3 } )

# The door the prize is behind is also completely random
prize = DiscreteDistribution( { 'A': 1./3, 'B': 1./3, 'C': 1./3 } )

    # Monty is dependent on both the guest and the prize.
monty = ConditionalProbabilityTable(
            [[ 'A', 'A', 'A', 0.0 ],
             [ 'A', 'A', 'B', 0.5 ],
             [ 'A', 'A', 'C', 0.5 ],
             [ 'A', 'B', 'A', 0.0 ],
             [ 'A', 'B', 'B', 0.0 ],
             [ 'A', 'B', 'C', 1.0 ],
             [ 'A', 'C', 'A', 0.0 ],
             [ 'A', 'C', 'B', 1.0 ],
             [ 'A', 'C', 'C', 0.0 ],
             [ 'B', 'A', 'A', 0.0 ],
             [ 'B', 'A', 'B', 0.0 ],
             [ 'B', 'A', 'C', 1.0 ],
             [ 'B', 'B', 'A', 0.5 ],
             [ 'B', 'B', 'B', 0.0 ],
             [ 'B', 'B', 'C', 0.5 ],
             [ 'B', 'C', 'A', 1.0 ],
             [ 'B', 'C', 'B', 0.0 ],
             [ 'B', 'C', 'C', 0.0 ],
             [ 'C', 'A', 'A', 0.0 ],
             [ 'C', 'A', 'B', 1.0 ],
             [ 'C', 'A', 'C', 0.0 ],
             [ 'C', 'B', 'A', 1.0 ],
             [ 'C', 'B', 'B', 0.0 ],
             [ 'C', 'B', 'C', 0.0 ],
             [ 'C', 'C', 'A', 0.5 ],
             [ 'C', 'C', 'B', 0.5 ],
             [ 'C', 'C', 'C', 0.0 ]], [guest, prize] )

# State objects hold both the distribution, and a high level name.
s1 = State( guest, name="guest" )
s2 = State( prize, name="prize" )
s3 = State( monty, name="monty" )

# Create the Bayesian network object with a useful name
network = BayesianNetwork( "Monty Hall Problem" )

# Add the three nodes to the network
network.add_nodes( [ s1, s2, s3 ] )

# Add transitions which represent conditional dependencies, where the second node is conditionally dependent on the first node (Monty is dependent on both guest and prize)
network.add_edge( s1, s3 )
network.add_edge( s2, s3 )
network.bake()

observations = { 'guest' : 'A' }

# beliefs will be an array of posterior distributions or clamped values for each state, indexed corresponding to the order
# in self.states.
beliefs = network.forward_backward( observations )

# Convert the beliefs into a more readable format
beliefs = map( str, beliefs )

# Print out the state name and belief for each state on individual lines
print "\n".join( "{}\t{}".format( state.name, belief ) for state, belief in zip( network.states, beliefs ) )

observations = { 'guest' : 'A', 'monty' : 'B' }
beliefs = map( str, network.forward_backward( observations ) )
print "\n".join( "{}\t{}".format( state.name, belief ) for state, belief in zip( network.states, beliefs ) )


"""
Module for doing the training
"""

from __future__ import division
import numpy as np
import plot as plot
from numpy import matrix
from numpy import linalg

def train (train_X, train_Y, learning_rate=1, delay=0.2, type="curve"):
    """
    Trains the linear regression model on training data
    Sets the hyperparameter according to the validation data
    and final accuracy on test data
    """

    Theta = np.matrix([0.0, 0.0]).T
    X_matrix = np.matrix(train_X).T
    Y_matrix = np.matrix(train_Y).T

    X_ones = np.ones((X_matrix.shape[0], 1))
    X_matrix = np.hstack((X_ones, X_matrix))

    # normalisation (approximate)
    X_matrix[:,1] = X_matrix[:,1] / 10

    epoch = 0
    eta = learning_rate
    costData = [] # for plotting real time data
    old_cost = 0
    converged = None
    while (True):
        Theta = SGD(X_matrix, Y_matrix, Theta, eta, 20)
        cost = compute_cost(X_matrix, Y_matrix, Theta)
        # print ('Epoch: %d | Cost: %.8f | Theta: %f, %f' % (epoch, cost, Theta[0,0], Theta[1,0]) )
        epoch = epoch + 1
        # Stopping condition
        if (float(Theta[0]) >= 10000 or float(Theta[1]) >= 10000 or (cost is float('inf')) or (epoch > 6 and cost > 1.5 * old_cost)): # 50 iters for the purpose of plotting
            # Diverged
            print ("The model is diverging :(, please change the learning rate :)")
            converged = False
            break
        if (epoch > 5000):
            # too slow
            print ("The learning rate is too small :(. Stopping since taking >5000 epochs. To train faster use eta close to 1...")
            converged = False
            break

        if (epoch > 20 and cost < 1e-5 and abs(old_cost - cost) / old_cost < 0.0001):
            # Change in cost is less than 0.01% => Stop
            converged = True
            break
        costData.append( (float(cost), float(Theta[0, 0]), float(Theta[1,0]) / 10.0) )
        old_cost = cost

    Theta[1,:] = Theta[1,:] / 10
    X_matrix[:,1] = X_matrix[:,1] * 10

    # Print result
    print("Theta0: %.6f | Theta1: %.6f | #Epochs: %d" % (Theta[0], Theta[1], epoch))

    plot.regressionPlot(train_X, train_Y, Theta[1,0], Theta[0,0], Xlabel="Acidity", Ylabel="Density of Wine", marker="bx", fileName="Q1/plots/regression.png")
    animatedDesent(X_matrix, Y_matrix, np.array(costData), delay, converged=converged)
    return Theta

def animatedDesent (X_matrix, Y_matrix, costData, delay=0.2, converged=True):
    """
    Plots the J(Theta) curve in 3D space and contours
    Shows the real time gradient descent
    """
    theta0s = np.linspace(0.3, 1.7, 100)
    theta1s = np.linspace(-0.080, 0.080, 105)

    if (not converged):
        theta0s = np.linspace(-2000, 2000, 100)
        theta1s = np.linspace(-2000, 2000, 100)

    costMatrix = np.zeros((len(theta1s), len(theta0s)))
    for i in range(len(theta1s)):
        for j in range(len(theta0s)):
            Theta = np.matrix([theta0s[j], theta1s[i]]).T
            # compute cost
            costMatrix[i][j] = compute_cost(X_matrix, Y_matrix, Theta)

    plot.costPlot(theta0s, theta1s, costMatrix, costData, delay=delay, Xlabel="Theta 0", Ylabel="Theta 1", Zlabel="Cost")
    plot.contourPlot(theta0s, theta1s, costMatrix, costData, delay=delay, converged=converged, Xlabel="Theta 0", Ylabel="Theta 1", Zlabel="Cost")

def SGD (X, Y, Theta, eta, batch_size=20):
    """
    Computes one epoch of the batch gradient decent algorithm
    """

    Theta = Theta - eta * compute_gradient(X, Y, Theta)

    return Theta

def compute_gradient (X, Y, Theta):
    """
    Computes the cost gradient
    X = m*n
    Y = m*1
    Theta = n*1
    gradient = (1/m) * X_transpose * (X*Theta - Y)
    """
    (m, n) = X.shape
    return (1.0/m) * (X.T) * (X*Theta - Y)

def compute_cost (X, Y, Theta):
    """
    Computes the J(Theta)
    X = m*n
    Y = m*1
    Theta = n*1
    Cost = (1/2m) * (Y-X*Theta)_tran * (Y-X*Theta)
    """

    (m, n) = X.shape
    error = Y - X*Theta
    return (0.5 / m) * error.T * error


# -*- coding: utf-8 -*-
"""
Created on Tue Nov 17 14:28:30 2015

@author: Hanna
"""
import numpy as np
import scipy as sp
import matplotlib.pyplot as plt
import scipy.optimize as op

def plotData(data):
    pos = data[data[:,2] == 1]
    neg = data[data[:,2] == 0]

    plt.figure()
    plt.scatter(pos[:,0], pos[:,1], c='k', marker='+', s=40, linewidths=2)
    plt.scatter(neg[:,0], neg[:,1], c='y', marker='o', s=40, linewidths=1)
    plt.legend(['y=1', 'y=0'], loc='upper right', scatterpoints=1)
    plt.xlabel("Microchip test 1")
    plt.ylabel("Microchip test 2")
    plt.xlim([-1, 1.5])
    plt.ylim([-0.8, 1.2])

def plotDecisionBdryNonlin(data,X,theta,lambbda):
    plotData(data)
    u = np.linspace(-1,1.5,50)
    v = np.linspace(-1,1.5,50)
    #print(np.shape(theta))
    z = np.zeros((len(u),len(v)))
    for i in range(0, len(u)):
        for j in range(0, len(v)):
            mapped = mapFeature(u[i],v[j],6)
            #print(np.shape(mapped))
            z[i,j] = np.dot(mapped,theta)
    u, v = np.meshgrid(u,v)
    plt.contour(u, v, z.T, [0.0, 0.0], colors = 'Green')
    plt.annotate('—', xy=(1.02, 0.79), color='Green')
    plt.annotate('Decision bdry', xy=(1.11, 0.79),fontsize=10)
    plt.title("lambda=" + str(lambbda))
    plt.savefig("NonLinDecisionBdry_lambda"+str(lambbda)+".pdf")

def mapFeature(X1,X2,degree):
    #maps features X1,X2 into polynomial of degree 6
    #Returns a new feature array with more features, comprising of
    #X1, X2, X1.^2, X2.^2, X1*X2, X1*X2.^2, etc..
    #adds intercept to a new feature matrix
    out = np.ones((np.shape(X1)))
    for i in range(1,degree+1):
        for j in range(0,i+1):
            out=np.hstack((out,(X1**(i-j))*(X2**j)))

    return out

def sigmoid(z):
    #works elementwise for any array z

    return sp.special.expit(z)  # expit(x) = 1/(1+exp(-x)) elementwise for array x

def hypothesis(theta,X):
    #works for any # of features

    return sigmoid(np.dot(X,theta))

def computeCostReg(theta,X,y,lambbda):  # J(theta) with regularization
    #works for any # of features
    m = len(y)
    h = hypothesis(theta,X)
    thetaReg = theta[1:]  #remove the first row(corresponding to theta0) from matrix theta
    cost = (-np.dot(y.T,sp.log(h))-np.dot((1.0-y).T,sp.log(1.0-h)))/m + lambbda*np.dot(thetaReg.T,thetaReg)/(2.0*m)
    #check for nans: can occur since have log
    #if skip this, precision loss occurs
    if np.isnan(cost):
        return np.inf

    return cost

def computeCostDerivReg(theta,X,y,lambbda):  # partial derivatives of J(theta) with regularization
    #works for any # of features
    n = np.shape(X)[1]  # # of features including itercept; in main n is the # of original features
    m = np.shape(X)[0]
    #minimize routine changes dimensions of theta from matrix to vector,
    #i.e from (3,1) to (3,) in this case
    #so need to reshape to original dimensions
    #since in computeCostDeriv routine it is important that theta is matrix
    #does not matter for computeCost routine
    theta = theta.reshape(n,1)
    h = hypothesis(theta,X)
    #initialize matrix of gradients
    gradient = np.zeros(np.shape(theta))
    #use formula for gradient of regularized cost function
    gradient = np.dot(X.T,h-y)/m + lambbda*theta/m
    #do not need to regularize theta0
    #it's computationally more efficient to compute all elements with regularization
    #and then compensate for theta0
    gradient[0,0] = gradient[0,0] - lambbda*theta[0,0]/m

    return gradient.flatten()  # minimize needs vector of gradients not matrix

def predict(theta,X):
    if hypothesis(theta,X) >= 0.5:
        prediction = 1
    else:
        prediction = 0

    return prediction

if __name__ == '__main__':

    data = np.genfromtxt("ex2data2.txt", delimiter=',')
    m = np.shape(data)[0]  # number of examples in training set
    n = np.shape(data)[1]-1  # number of features
    X, y = data[:,:n], data[:,n:]

    #plot data
    plotData(data)
    plt.savefig("scatterData2.pdf")

    #bdry looks nonlinear so enlarge feature space to degree 6 polynomial
    X = mapFeature(X[:,0:1],X[:,1:2],6)

    #initialize theta
    theta = np.zeros((np.shape(X)[1],1))
    #set regularization parameter
    lambbda = 1.0

    #test cost function on initial theta, should be 0.693
    print("cost function for initial theta = ",computeCostReg(theta,X,y,lambbda),"should be 0.693")  # get [[ 0.69314718]]

    #find minimum of cost function using BFGS
    result = op.minimize(computeCostReg, theta, args=(X,y,lambbda), method='BFGS', jac=computeCostDerivReg, options={'disp': True})
    theta = result.x  # note, result.x is a vector so below theta is a vector ie when passed to plotDecisionBdryNonlin
    cost = result.fun
    print("optimal theta =",theta)
    print("optimal cost funstion =",cost)

    #plot decision bdry
    plotDecisionBdryNonlin(data,X,theta,lambbda)

    #try other values of lambda
    lambbdas = [0.0,0.5,100]
    for lambbda in lambbdas:
        theta = np.zeros((np.shape(X)[1],1))
        #using TNC (truncated Newton) gives decision bdry similar to matlab's fmiunc for lambbda=0;
        #using BFGS gives different boundary then fminunc for lambbda=0 although fminunc uses BFGS;
        #for other tested values of lambda there is no difference between TNC and BFGS
        #result = op.minimize(computeCostReg, theta, args=(X,y,lambbda), method='BFGS', jac=computeCostDerivReg, options={'disp': True})
        result = op.minimize(computeCostReg, theta, args=(X,y,lambbda), method='TNC', jac=computeCostDerivReg, options={'disp': True})
        theta = result.x
        plotDecisionBdryNonlin(data,X,theta,lambbda)

    plt.show()


import numpy as np
import matplotlib.pyplot as plt

def tpr( y_pred, y_actual, threshold):
        """
        y_pred    : numpy array of predicted values between 0 and 1.
        y_actual  : numpy array of actual values, either 0 or 1.
        threshold : value at which roundup begins.

        Returns true positive rate when `y_pred` is rounded up when greater than `threshold` and down otherwise.
        """
        y_rounded = np.array([ pred > threshold for pred in y_pred ])
        num_true_positive = np.logical_and(y_rounded, y_actual).sum()
        num_total_positive = y_actual.sum()
        return num_true_positive / num_total_positive

def fpr( y_pred, y_actual, threshold):
        """
        y_pred    : numpy array of predicted values between 0 and 1.
        y_actual  : numpy array of actual values, either 0 or 1.
        threshold : value at which roundup begins.

        Returns false positive rate when `y_pred` is rounded up when greater than `threshold` and down otherwise.
        """
        y_rounded = np.array([ pred > threshold for pred in y_pred ])
        num_false_positive = np.logical_and(y_rounded, np.logical_not(y_actual)).sum()
        num_total_positive = np.logical_not(y_actual).sum()
        return num_false_positive / num_total_positive

def compute_auc(y_pred, y_actual):
        """
        Approximates the Area Under the Curve of the Receiver Operating Characteristic.

        roc_auc_score from sklearn.metrics is definitely a closer approximation. Will improve here.
        """
        thresholds = np.arange(0,1,0.01)
        fprs = [ fpr(y_pred, y_actual, threshold) for threshold in thresholds]
        tprs = [tpr(y_pred, y_actual, threshold) for threshold in thresholds]
        rectangle_bases = np.array([ np.abs(fprs[i+1] - fprs[i]) for i in range(len(fprs) - 1) ])
        rectangle_heights = tprs[:-1]
        rectangle_areas = rectangle_bases * rectangle_heights
        return rectangle_areas.sum()

def plot_auc(y_pred, y_actual, include_chance=True):
        """
        include_chance : if True, plots the straight line y=x which can be compared to ROC score.

        Uses matplotlib.pyplot to plot the ROC curve.
        """
        thresholds = np.arange(0,1,0.01)
        fprs = [ fpr(y_pred, y_actual, threshold) for threshold in thresholds]
        tprs = [tpr(y_pred, y_actual, threshold) for threshold in thresholds]
        plt.plot(fprs, tprs)
        if include_chance:
                plt.plot([0,1], [0,1], color='orange')
        plt.xlabel('False Positive Rate')
        plt.ylabel('True Positive Rate')
        plt.title('Receiver Operating Characteristic')

# y_actual = np.array([0] * 75 + [1] * 100)
# y_pred = np.concatenate([
#         np.random.uniform(0,   0.7, 75 ),
#         np.random.uniform(0.3, 1.0, 100 )
# ])

# tpr( y_pred, y_actual, 0.5)
# fpr( y_pred, y_actual, 0.5)

# compute_auc(y_pred, y_actual)
# plot_auc(y_pred, y_actual, include_chance=True)


from numpy import abs as abs
from numpy import array as array
from numpy import floor as floor
from numpy import max as max
from numpy import sqrt as sqrt
from numpy import sum as sum
from numpy import zeros as zeros
from scipy.stats import norm as norm

class CumulativeSums:

    @staticmethod
    def cumulative_sums_test(binary_data:str, mode=0, verbose=False):
        """
        from the NIST documentation http://csrc.nist.gov/publications/nistpubs/800-22-rev1a/SP800-22rev1a.pdf

        The focus of this test is the maximal excursion (from zero) of the random walk defined by the cumulative sum of
        adjusted (-1, +1) digits in the sequence. The purpose of the test is to determine whether the cumulative sum of
        the partial sequences occurring in the tested sequence is too large or too small relative to the expected
        behavior of that cumulative sum for random sequences. This cumulative sum may be considered as a random walk.
        For a random sequence, the excursions of the random walk should be near zero. For certain types of non-random
        sequences, the excursions of this random walk from zero will be large.

        :param      binary_data:    a binary string
        :param      mode            A switch for applying the test either forward through the input sequence (mode = 0)
                                    or backward through the sequence (mode = 1).
        :param      verbose         True to display the debug messgae, False to turn off debug message
        :return:    (p_value, bool) A tuple which contain the p_value and result of frequency_test(True or False)

        """

        length_of_binary_data = len(binary_data)
        counts = zeros(length_of_binary_data)

        # Determine whether forward or backward data
        if not mode == 0:
            binary_data = binary_data[::-1]

        counter = 0
        for char in binary_data:
            sub = 1
            if char == '0':
                sub = -1
            if counter > 0:
                counts[counter] = counts[counter -1] + sub
            else:
                counts[counter] = sub

            counter += 1
        # Compute the test statistic z =max1≤k≤n|Sk|, where max1≤k≤n|Sk| is the largest of the
        # absolute values of the partial sums Sk.
        abs_max = max(abs(counts))

        start = int(floor(0.25 * floor(-length_of_binary_data / abs_max) + 1))
        end = int(floor(0.25 * floor(length_of_binary_data / abs_max) - 1))

        terms_one = []
        for k in range(start, end + 1):
            sub = norm.cdf((4 * k - 1) * abs_max / sqrt(length_of_binary_data))
            terms_one.append(norm.cdf((4 * k + 1) * abs_max / sqrt(length_of_binary_data)) - sub)

        start = int(floor(0.25 * floor(-length_of_binary_data / abs_max - 3)))
        end = int(floor(0.25 * floor(length_of_binary_data / abs_max) - 1))

        terms_two = []
        for k in range(start, end + 1):
            sub = norm.cdf((4 * k + 1) * abs_max / sqrt(length_of_binary_data))
            terms_two.append(norm.cdf((4 * k + 3) * abs_max / sqrt(length_of_binary_data)) - sub)

        p_value = 1.0 - sum(array(terms_one))
        p_value += sum(array(terms_two))

        if verbose:
            print('Cumulative Sums Test DEBUG BEGIN:')
            print("\tLength of input:\t", length_of_binary_data)
            print('\tMode:\t\t\t\t', mode)
            print('\tValue of z:\t\t\t', abs_max)
            print('\tP-Value:\t\t\t', p_value)
            print('DEBUG END.')

        return (p_value, (p_value >= 0.01))