# Chapter 11

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

## Review

[The previous notebook](https://colab.research.google.com/github/AllenDowney/BiteSizeBayes/blob/master/10_joint.ipynb) I introduced cross tabulation, joint distributions, conditional disrtribution, and marginal distributions.

In this notebook, we'll apply these ideas to Bayesian inference.

But first I want to introduce a computational tool we will need, outer operations.

## Outer operations

Suppose you have two sequences, like `t1` and `t2`:

t1 = [1,3,5]

t2 = [2,4]

Many useful operations can be expressed in the form of an "outer operation" of these sequences.

The most common outer operation is the outer product, which computes the product of every pair of values, one from each sequence.

For example, here is the outer product of `t1` and `t2`:

a = np.multiply.outer(t1, t2)
a

We can understand this result more easily if we put it in a DataFrame:

pd.DataFrame(a, index=t1, columns=t2)

The values from `t1` appear along the rows of the result; the values from `t2` appear along the columns.

Each element of the result is the product of an element from `t1` and an element from `t2`.

The outer sum is similar, except that each element is the *sum* of an element from `t1` and an element from `t2`.

a = np.add.outer(t1, t2)
a

pd.DataFrame(a, index=t1, columns=t2)

We can do the same thing with almost any operation.  For example, the "outer greater than" operation compares each element from `t1` to each element of `t2`; the result is an array of Boolean values.

a = np.greater.outer(t1, t2)
a

pd.DataFrame(a, index=t1, columns=t2)

These outer operations work with Python lists and tuples, and NumPy arrays, but not Pandas Series.

However, the following function works with Pandas Series, and puts the result into a DataFrame.

def outer_product(s1, s2):
    """Compute the outer product of two Series.

    First Series goes down the rows;
    second goes across the columns.

    s1: Series
    s2: Series

    return: DataFrame
    """
    a = np.multiply.outer(s1.to_numpy(), s2.to_numpy())
    return pd.DataFrame(a, index=s1.index, columns=s2.index)

It might not be obvious yet why these operations are useful, but we'll see some examples soon.

With that, we are ready to take on a new Bayesian problem.

## How tall is A?

Suppose I choose two people from the population of adult males in the U.S.; I'll call them A and B.  If we see that A taller than B, how tall is A?

To answer this question:

1. I'll use background information about the height of men in the U.S. to form a prior distribution of height,

2. I'll construct a joint distribution of height for A and B, and update it with the information that A is taller, and

3. I'll extract from the posterior joint distribution the posterior distribution of height for A.

In the U.S. the average height of male adults in 178 cm and the standard deviation is 7.7 cm.  The distribution is not exactly normal, because nothing in the real world is, but the normal distribution is a pretty good model of the actual distribution, so we can use it as a prior distribution for A and B.

Here's an array of equally-spaced values from roughly 3 standard deviations below the mean to 3 standard deviations above.

mean = 178
std = 7.7

xs = np.arange(mean-24, mean+24, 0.5)

SciPy provides a function called `norm` that represents a normal distribution with a given mean and standard deviation, and provides `pdf`, which evaluates the probability distribution function (PDF), which we will use as the prior probabilities.

from scipy.stats import norm

ps = norm(mean, std).pdf(xs)

I'll store the `xs` and `ps` in a Series that represents the prior PMF.

prior = pd.Series(ps, index=xs)

And normalize it:

prior /= prior.sum()

And here's what it looks like.

prior.plot()

plt.xlabel('Height in cm')
plt.ylabel('Probability')
plt.title('Distribution of height for men in U.S.');

We can think of this prior distribution as the marginal distribution for A and B, but what we want is the joint probability of their heights.

## Joint distribution

As we saw in the previous notebook, it is not *generally* possible to construct a joint distribution if we only have the marginals, because the marginals don't contain information about correlations between the variables.

However, in the special case where there are no correlations, or they are small enough to ignore, it *is* possible to construct the joint distribution.

To see how, let's consider one element of the joint distribution,

$P(A_y~\mathrm{and}~B_x)$

which is the probability that `A` is $y$ cm tall and `B` is $x$ cm tall.  We can rewrite this conjuction in terms of conditional probability:

$P(A_y)~P(B_x~|~A_y)$

We can compute $P(A_y)$ from the marginal distribution, but how should we compute the conditional probability, $P(B_x~|~A_y)$?

In this case, the heights of `A` and `B` are "independent", which means that knowing the height of `A` provides no additional information about the height of `B`.

And that means that the conditional probability, $P(B_x~|~A_y)$, is just the marginal probability $P(B_y)$.

Which means that in this case, the joint probability is just the product of the marginal probabilities.

$P(A_y~\mathrm{and}~B_x) = P(A_y)~P(B_x)$

Now, to compute the joint distribution, we have to compute this product for all values of $x$ and $y$.  And we can do that by computing the outer product of the marginal distributions, like this:

joint = outer_product(prior, prior)
joint.shape

If the prior is normalized, the joint prior should also be normalized.

joint.to_numpy().sum()

The following function uses `pcolormesh` to plot the joint distribution.

Recall that `outer_product` puts the values of `A` along the rows and the values of `B` across the columns.

def plot_joint(joint):
    """Plot a joint distribution.

    joint: DataFrame representing a joint PMF
    """
    plt.pcolormesh(joint.index, joint.index, joint)
    plt.ylabel('A height in cm')
    plt.xlabel('B height in cm')

And here's what the result looks like.

plot_joint(joint)
plt.colorbar()
plt.title('Joint prior distribution of height for A and B');

As you might expect, the probability is highest near the mean and drops off away from the mean.

Another way to visualize the joint distribution is a contour plot.

def plot_contour(joint):
    """Plot a joint distribution.

    joint: DataFrame representing a joint PMF
    """
    plt.contour(joint.index, joint.index, joint)
    plt.ylabel('A height in cm')
    plt.xlabel('B height in cm')

plot_contour(joint)
plt.title('Joint prior distribution of height for A and B');

Each circle represents a level of equal probability.

## Likelihood

Now that we have a joint PMF that represents the prior distribution, we can update it with the data, which is that `A` is taller than `B`.

Each element in the joint distribution represents a hypothesis about the heights of `A` and `B`; for example:

* The element `(180, 170)` represents the hypothesis that `A` is 180 cm tall and `B` is 170 cm tall.  Under this hypothesis, the probability that `A` is taller than `B` is 1.

* The element `(170, 180)` represents the hypothesis that `A` is 170 cm tall and `B` is 180 cm tall.  Under this hypothesis, the probability that `A` is taller than `B` is 0.

To compute the likelihood of every pair of values, we can extract the values from the prior, like this:

Y = prior.index.to_numpy()
X = prior.index.to_numpy()

And then apply the `outer` version of `np.greater`, which compares every element of `Y` (height of `A`) to every element of `X` (height of `B`).

a = np.greater.outer(Y, X)

The result is an array, which we can put in a DataFrame with the corresponding `index` and `columns`.

likelihood = pd.DataFrame(a, index=Y, columns=X)

Here's what it looks like:

plot_joint(likelihood)
plt.title('Likelihood of A>B');

The likelihood of the data is 1 where `Y>X` and 0 otherwise.

## The update

We have a prior, we have a likelihood, and we are ready for the update.  As usual, the unnormalized posterior is the product of the prior and the likelihood.

unnorm_posterior = joint * likelihood

And we can get the normalized posterior by dividing through by the total.

total = unnorm_posterior.to_numpy().sum()
joint_posterior = unnorm_posterior / total
total

The total probability of the data is a little less than $1/2$.

Here's what the normalized posterior looks like.

plot_joint(joint_posterior)
plt.colorbar()

plt.title('Joint posterior distribution of height for A and B');

It looks like a sunrise as seen from the deck of a [heeling sailboat](https://en.wikipedia.org/wiki/Sailing#Heeling).

## The marginals

From the posterior joint distribution we can extract the posterior marginal distribution of `A` and `B`.

def marginal(joint, axis):
    """Compute a marginal distribution.

    axis=0 returns the marginal distribution of the second variable
    axis=1 returns the marginal distribution of the first variable

    joint: DataFrame representing a joint PMF
    axis: int axis to sum along

    returns: Series representing a marginal PMF
    """
    return joint.sum(axis=axis)

marginal_A = marginal(joint_posterior, axis=1)
marginal_B = marginal(joint_posterior, axis=0)

Here's what they look like.

prior.plot(label='Prior')
marginal_A.plot(label='Posterior for A')
marginal_B.plot(label='Posterior for B')

plt.xlabel('Height in cm')
plt.ylabel('Probability')
plt.title('Prior and posterior distributions for A and B')
plt.legend();

As you might expect, the posterior distribution for `A` is shifted to the right and the posterior distribution for `B` is shifted to the left.

We can summarize the results by computing the posterior means:

def pmf_mean(pmf):
    """Compute the mean of a PMF.

    pmf: Series representing a PMF

    return: float
    """
    return np.sum(pmf.index * pmf)

pmf_mean(prior)

pmf_mean(marginal_A), pmf_mean(marginal_B)

Based on the observation that `A` is taller than `B`, we are inclined to believe that `A` is a little taller than average, and `B` is a little shorter.

Notice that the posterior distribution are a little taller and narrower than the prior.  We can quantify that my computing their standard deviations.

def pmf_std(pmf):
    """Compute the standard deviation of a PMF.

    pmf: Series representing a PMF

    return: float
    """
    deviation = pmf.index - pmf_mean(pmf)
    var = np.sum(deviation**2 * pmf)
    return np.sqrt(var)

pmf_std(prior), pmf_std(marginal_A)

The standard deviation of the posterior distributions are a little smaller, which means we are a little more certain about the heights of `A` and `B` after we compare them.

## Conditional posteriors

Now suppose we measure `B` and find that he is 185 cm tall.  What does that tell us about `A`?

We can answer that question by extracting the conditional posterior distribution for `A`, conditioned on `B=185`.

Possible heights for `A` run down the rows of the joint PMF, so each row is an unnormalized posterior distribution conditioned on `A`.

And possible heights for `B` run across the columns, so each column is an unnormalized posterior distribution conditioned on `B`.

So we can condition on `B` by selecting a column and normalizing it.

cond_A = joint_posterior[185].copy()
cond_A /= cond_A.sum()

prior.plot(label='Prior')
marginal_A.plot(label='Posterior for A')
cond_A.plot(label='Posterior for A given B=185', color='C4')

plt.xlabel('Height in cm')
plt.ylabel('Probability')
plt.title('Prior, posterior and conditional distribution for A')
plt.legend();

The posterior conditional distribution is cut off at 185 cm, because we have established that `A` is taller than `B` and `B` is 185 cm.

And the posterior conditional is substantially different from the unconditional posterior; that is, for each value of $y$

$P(A_y | B_x) \ne P(A_y)$

which means that in the posterior distribution, `A` and `B` are not independent.

## Elo rating

[The Elo rating system](https://en.wikipedia.org/wiki/Elo_rating_system) is a way to quantify the skill level of players for games like chess.

It is based on a model of the relationship between the ratings of players and the outcome of a game.  Specifically, if $R_A$ is the rating of player $A$ and $R_B$ is the rating of player $B$, the probability that $A$ beats $B$ is given by the [logistic function](https://en.wikipedia.org/wiki/Logistic_function):

$P(A~\mathrm{wins}) = 1 / (1 + 10^{(R_B-R_A)/400})$

The parameters $10$ and $400$ are arbitrary choices that determine the range of the ratings.  In chess, values range from 100 to 2800.

Notice that the probability of winning depends only on the difference in rankings.  As an example, if $R_A$ exceeds $R_B$ by 100 points, the probability that $A$ wins is

1 / (1 + 10**(-100/400))

**Exercise:** Suppose `A` has a current rating of 1600, but we are not sure it is accurate.  We could describe their true rating with a normal distribution with mean 1600 and standard deviation 100, to indicate our uncertainty.

And suppose `B` has a current rating of 1800, with the same level of uncertaintly.

Finally, `A` and `B` play and `A` wins.  How should we update their ratings?

To answer this question:

1. Construct prior distributions for `A` and `B`.

2. Use them to construct a joint distribution, assuming that the prior distributions are independent.

3. Use the logistic function above to compute the likelihood of the outcome under each joint hypothesis.  Hint: use `np.subtract.outer`.

4. Use the joint prior and likelihood to compute the joint posterior.

5. Extract and plot the marginal posteriors for `A` and `B`.

6. Compute the posterior means for `A` and `B`.  How much should their ratings change based on this outcome?

# Solution goes here

# Solution goes here

# Solution goes here

# Solution goes here

# Solution goes here

# Solution goes here

# Solution goes here

# Solution goes here

# Solution goes here

# Solution goes here

# Solution goes here

# Solution goes here

## Summary

In this notebook I started with the "outer" operations, like outer product and outer sum; then we used them to construct a joint distribution.

In general, you cannot construct a joint distrubution from two marginal distributions, but in the special case where the distributions are independent, you can.

We extended the Bayesian update process we've seen in previous notebook and applied it to a joint distribution.  Then from the posterior joint distribution we extracted posterior marginal distributions and posterior conditional distributions.

As an exercise, you had a chance to apply the same process to a slightly more difficult problem, updating Elo ratings based on the outcome of a chess game.

[In the next notebook](https://colab.research.google.com/github/AllenDowney/BiteSizeBayes/blob/master/12_sum.ipynb) we'll get back to a problem we left half-finished: the Euro problem.


import scipy as sp
from timer import timer

def entropy(values):
    """A slow way to calculate the entropy of the input values"""

    values = sp.asarray(values).flatten()
    #calculate the probablility of a value in a vector
    vUni = sp.unique(values)
    lenval = float(values.size)

    FreqData = sp.zeros(vUni.shape, dtype=float)
    for i in xrange(FreqData.size):
        FreqData[i] = sum(values==vUni[i])/lenval

    return -sum([FreqData[i]*sp.math.log(FreqData[i],2) for i in xrange(FreqData.size)])

def entropy2(values):
    """Calculate the entropy of vector values.

    values will be flattened to a 1d ndarray."""

    values = sp.asarray(values).flatten()
    p = sp.diff(sp.c_[0,sp.diff(sp.sort(values)).nonzero(), values.size])/float(values.size)
    H = (p*sp.log2(p)).sum()
    return -H

def chebyshev2(values, degree=1):
    """Calculate the Chebyshev Polynomials using previous results"""

    values = sp.asarray(values)
    A = sp.zeros((degree, len(values)))

    A[0,:]=1
    try:
        A[1,:]=values
    except IndexError:
        return A

    for i in range(2,degree):
        for x in range(len(values)):
            A[i,x] = 2*values[x]*A[i-1,x]-A[i-2,x]

    return A

def chebyshev2_lc(values, degree=1):
    """Calculate the Chebyshev Polynomials using previous results"""

    values = sp.asarray(values)
    A = sp.zeros((degree, len(values)))

    A[0,:]=1
    try:
        A[1,:]=values
    except IndexError:
        return A

    for i in range(2,degree):
        A[i,:] = [2*x for x in values]*A[i-1,:]-A[i-2,:]

    return A

def chebyshev_sp(values, degree=1):
    """Calculate the Chebyshev Polynomials using the scipy functions"""

    values = sp.asarray(values)
    A = sp.zeros((degree, len(values)))

    A[0,:]=1
    try:
        A[1,:]=values
    except IndexError:
        return A

    for i in range(2,degree):
        A[i,:] = sp.cos(i*sp.arccos(values))

    return A

def chebyshev_vec(values, degree=1):
    """Calculate the Chebyshev Polynobials

    This implementation uses sp.vectorize to vectorize python's math functions)"""

    values = sp.asarray(values)
    A = sp.zeros((degree, len(values)))

    A[0,:]=1
    try:
        A[1,:]=values
    except IndexError:
        return A

    cos = sp.vectorize(sp.math.cos)
    acos = sp.vectorize(sp.math.acos)

    for i in range(2,degree):
        A[i,:] = cos(i*acos(values))

    return A

def chebyshev_lc(values, degree=1):
    """Calculate the Chebyshev Polynomials using list comprehensions"""

    values = sp.asarray(values)
    A = sp.zeros((degree, len(values)))

    A[0,:]=1
    try:
        A[1,:]=values
    except IndexError:
        return A

    for i in range(2,degree):
        A[i,:] = [sp.math.cos(y) for y in [i*sp.math.acos(x) for x in values]]

    return A

def chebyshev(values, degree=1):
    """Calculate the Chebyshev Polynomial using

    Tn(x) = cos(n*cosh(x))"""
    values = sp.asarray(values)
    A = sp.zeros((degree, len(values)))

    A[0,:]=1
    try:
        A[1,:]=values
    except IndexError:
        return A

    for i in range(2,degree):
        for x in values:
            A[i,:] = sp.math.cos(i*sp.math.acos(x))
    return A

if __name__ == '__main__':
    from timer import timer
    testvals = sp.linspace(-1,1,500)
    funcs = [chebyshev, chebyshev_lc, chebyshev_vec, chebyshev_sp, chebyshev2, chebyshev2_lc]
    with timer(loops=5) as t:
        for f in funcs:
            t.time(f, testvals, 100)
        t.printTimes()


#
#  file:  gd_2d.py
#
#  2D example of gradient descent
#
#  RTK, 14-Feb-2021
#  Last update:  14-Feb-2021
#
################################################################

import numpy as np
import matplotlib.pylab as plt

#  Function and partial derivatives
def f(x,y):
    return 6*x**2 + 9*y**2 - 12*x - 14*y + 3

def dx(x):
    return 12*x - 12

def dy(y):
    return 18*y - 14

#  Gradient descent steps
N = 100
x,y = np.meshgrid(np.linspace(-1,3,N), np.linspace(-1,3,N))
z = f(x,y)
plt.contourf(x,y,z,10, cmap="Greys")
plt.contour(x,y,z,10, colors='k', linewidths=1)
plt.plot([0,0],[-1,3],color='k',linewidth=1)
plt.plot([-1,3],[0,0],color='k',linewidth=1)
plt.plot(1,0.7777778,color='k',marker='+')

x = xold = -0.5
y = yold = 2.9
for i in range(12):
    plt.plot([xold,x],[yold,y], marker='o', linestyle='dotted', color='k')
    xold = x
    yold = y
    x = x - 0.02 * dx(x)
    y = y - 0.02 * dy(y)

x = xold = 1.5
y = yold = -0.8
for i in range(12):
    plt.plot([xold,x],[yold,y], marker='s', linestyle='dotted', color='k')
    xold = x
    yold = y
    x = x - 0.02 * dx(x)
    y = y - 0.02 * dy(y)

x = xold = 2.7
y = yold = 2.3
for i in range(12):
    plt.plot([xold,x],[yold,y], marker='<', linestyle='dotted', color='k')
    xold = x
    yold = y
    x = x - 0.02 * dx(x)
    y = y - 0.02 * dy(y)

plt.xlabel("$x$")
plt.ylabel("$y$")
plt.tight_layout(pad=0, w_pad=0, h_pad=0)
plt.savefig("gd_2d_steps.png", dpi=300)
plt.show()
plt.close()

#  New function and partial derivatives
def f(x,y):
    return 6*x**2 + 40*y**2 - 12*x - 30*y + 3

def dx(x):
    return 12*x - 12

def dy(y):
    return 80*y - 30

#  Large stepsize
N = 100
x,y = np.meshgrid(np.linspace(-1,3,N), np.linspace(-1,3,N))
z = f(x,y)
plt.contourf(x,y,z,10, cmap="Greys")
plt.contour(x,y,z,10, colors='k', linewidths=1)
plt.plot([0,0],[-1,3],color='k',linewidth=1)
plt.plot([-1,3],[0,0],color='k',linewidth=1)
plt.plot(1,0.375,color='k',marker='+')

x = xold = -0.5
y = yold = 2.3
for i in range(14):
    plt.plot([xold,x],[yold,y], marker='o', linestyle='dotted', color='k')
    xold = x
    yold = y
    x = x - 0.02 * dx(x)
    y = y - 0.02 * dy(y)

x = xold = 2.3
y = yold = 2.3
for i in range(14):
    plt.plot([xold,x],[yold,y], marker='s', linestyle='dotted', color='k')
    xold = x
    yold = y
    x = x - 0.01 * dx(x)
    y = y - 0.01 * dy(y)

plt.xlabel("$x$")
plt.ylabel("$y$")
plt.tight_layout(pad=0, w_pad=0, h_pad=0)
plt.savefig("gd_2d_oscillating.png", dpi=300)
plt.show()
plt.close()


from __future__ import division
import unittest
from n88tools import transformations
import numpy as np
import math

class TestTransformations(unittest.TestCase):

    def test_rotationZ_phi_zero(self):
        '''Zero phi returns identity for rotationZ'''
        M = transformations.rotationZ(0.0)
        self.assertTrue(
            np.allclose(M, np.eye(3)),
            'rotationZ(0) does not return identity'
        )

    def test_rotationY_phi_zero(self):
        '''Zero phi returns identity for rotationY'''
        M = transformations.rotationY(0.0)
        self.assertTrue(
            np.allclose(M, np.eye(3)),
            'rotationY(0) does not return identity'
        )

    def test_rotationX_phi_zero(self):
        '''Zero phi returns identity for rotationX'''
        M = transformations.rotationX(0.0)
        self.assertTrue(
            np.allclose(M, np.eye(3)),
            'rotationX(0) does not return identity'
        )

    def test_rotationZ_phi_180(self):
        '''180 phi returns negative identity for rotationZ'''
        M = transformations.rotationZ(math.radians(180.0))
        T = np.eye(3)
        T[0,0]*=-1
        T[1,1]*=-1

        self.assertTrue(np.allclose(M,T))

    def test_rotationY_phi_180(self):
        '''180 phi returns negative identity for rotationY'''
        M = transformations.rotationY(math.radians(180.0))
        T = np.eye(3)
        T[0,0]*=-1
        T[2,2]*=-1

        self.assertTrue(np.allclose(M,T))

    def test_rotationX_phi_180(self):
        '''180 phi returns negative identity for rotationX'''
        M = transformations.rotationX(math.radians(180.0))
        T = np.eye(3)
        T[1,1]*=-1
        T[2,2]*=-1

        self.assertTrue(np.allclose(M,T))

    def test_rotationZ(self):
        '''rotationZ returns correct for phi'''
        phi=18.967
        M = transformations.rotationZ(math.radians(phi))
        T = np.array([
            [0.9457060, -0.3250235,  0.0000000],
            [0.3250235,  0.9457060,  0.0000000],
            [0.0000000,  0.0000000,  1.0000000]
        ])

        self.assertTrue(np.allclose(M,T))

    def test_rotationY(self):
        '''rotationY returns correct for phi'''
        phi=18.967
        M = transformations.rotationY(math.radians(phi))
        T = np.array([
            [0.9457060,  0.0000000,  0.3250235],
            [0.0000000,  1.0000000,  0.0000000],
            [-0.3250235,  0.0000000,  0.9457060]
        ])

        self.assertTrue(np.allclose(M,T))

    def test_rotationX(self):
        '''rotationX returns correct for phi'''
        phi=18.967
        M = transformations.rotationX(math.radians(phi))
        T = np.array([
            [1.0000000,  0.0000000,  0.0000000],
            [0.0000000,  0.9457060, -0.3250235],
            [0.0000000,  0.3250235,  0.9457060]
        ])

        self.assertTrue(np.allclose(M,T))

if __name__ == '__main__':
    unittest.main()


import numpy as np
import math
from sklearn.neighbors import kneighbors_graph
from sklearn.linear_model import LassoLars, Lars
from scipy.linalg import eigh
from fsfc.base import KBestFeatureSelector

class MCFS(KBestFeatureSelector):
    """
    Multi-Class Feature selection algorithm.

    Uses k-NN graph of samples in dataset and Spectral Graph Theory to find the most explaining features.

    Based on the article `"Unsupervised feature selection for multi-cluster data." <https://dl.acm.org/citation.cfm?id=1835848>`_.

    Algorithm selects features in the following way:

        1. Computes k-NN graph for the dataset.
        2. Computes heat matrix for this graph, degree and laplacian matrix.
        3. Solves eigen-problem: L y = `lambda` D y, selects K smallest eigen-values and corresponding vectors.
        4. Solves K regression problems, trying to predict every eigenvector by regression using dataset.
        5. Computes score of each feature using found regression coefficients.
        6. Select k features with the top scores.

    Parameters
    ----------
    k: int
        Number of features to select.
    clusters: int
        Expected number of datasets.
    p: int (default 8)
        Number of nearest neighbours for construction of k-NN graph.
    sigma: int (default 1)
        Coefficient for computation of heat matrix.
    mode: 'default' or 'lasso' (default 'default')
        Type of penalty for the method: with 'default' algorithm uses no penalty, with 'lasso' it uses L1-penalty.
    alpha: float (default 0.01)
        Importance of penalty for algorithm with **mode='lasso'**.
    """

    def __init__(self, k, clusters, p=8, sigma=1, mode='default', alpha=0.01):
        super().__init__(k)
        self.clusters = clusters
        self.p = p
        self.mode = mode
        self.sigma = sigma
        self.alpha = alpha

    def _create_regressor(self):
        if self.mode == 'default':
            return Lars()
        if self.mode == 'lasso':
            return LassoLars(alpha=self.alpha)
        raise ValueError('Unexpected mode ' + self.mode + '. Expected "default" or "lasso"')

    def _calc_scores(self, x):
        graph = kneighbors_graph(
            x,
            n_neighbors=self.p,
        )
        # Construct the heat matrix
        w = np.zeros([x.shape[0], x.shape[0]])
        rows, cols = graph.nonzero()
        for i, j in zip(rows, cols):
            w[i, j] = math.exp(-np.linalg.norm(x[i] - x[j])**2/self.sigma)

        # Compute degree and Laplacian matrices
        degree_vector = np.sum(w, 1)
        degree = np.diag(degree_vector)
        laplacian = degree - w

        # Solve the eigen-problem
        values, vectors = eigh(laplacian, degree)
        smallest = vectors[:, 0:self.clusters].T

        # Find coefficients for each cluster
        coefs = []
        for i in range(self.clusters):
            this_coefs = self._create_regressor().fit(x, smallest[i]).coef_
            coefs.append(this_coefs)
        coefs = np.array(coefs)

        # Compute MCFS-scores
        scores = np.max(coefs, 0)
        return scores


from numpy import *
import matplotlib.pyplot as plt
#http://systems-sciences.uni-graz.at/etextbook/sw2/lyapunov.html

result = []
lambdas = []
lambda_max=[]
maps = []
xmin = 2.9
xmax = 3.1
mult = int((xmax - xmin)*2000)

rvalues = arange(xmin, xmax, 0.001)

for r in rvalues:
    x = 0.1
    result = []
    for t in range(100):
        x = r * x * (1 - x)
        result.append(log(abs(r - 2*r*x)))
    lambdas.append(mean(result))
    lambda_max.append(max(result))
    # ignore first 100 iterations as transient time
    # then iterate anew
    for t in range(20):
        x = r * x * (1 - x)
        maps.append(x)

fig = plt.figure(figsize=(10,7))
ax1 = fig.add_subplot(1,1,1)

xticks = linspace(xmin, xmax, mult)

# zero line
zero = [0]*mult
ax1.plot(xticks, zero, 'g-')
ax1.plot(xticks, maps, 'r.',alpha = 0.3, label = 'Logistic map')
ax1.set_xlabel('r')
ax1.plot(rvalues, lambdas, 'b-', linewidth = 3, label = 'Lyapunov exponent mean')
ax1.plot(rvalues, lambda_max, 'p-', linewidth = 3, label = 'Lyapunov exponent max')
ax1.grid('on')
ax1.set_ylim(-1, 1)
ax1.set_xlabel('r')
ax1.legend(loc='best')
ax1.set_title('Logistic map versus Lyapunov exponent')

plt.show()


#!/usr/bin/env python
# coding: utf-8

# In[1]:

import pandas as pd
import numpy as np
import math
import random
import matplotlib.pyplot as plt
plt.style.use("seaborn-darkgrid")
import os

# In[2]:

def coordinates_on_circle(n):
    """Returns x,y coordinates of points on an unit circle with spacing 2π/n"""
    if type(n)!=int:
        raise Exception("Wrong input: \n the argument must be an integer number of points on the unit circle!")
    x,y = [],[]
    step_angle = 2*math.pi/n
    for i in range(0,n):
        x.insert(len(x),math.cos(i*step_angle))
        y.insert(len(y),math.sin(i*step_angle))
    return x,y

# In[3]:

def create_starting_graph(n,r):
    if type(r)!=int:
        raise Exception("Wrong input: \n r must be an integer number of edges between vertices")
    if r>n-1:
        raise Exception("Wrong input: \n r must not exceed n-1!")
    coords = coordinates_on_circle(n)

    #create adjacency_matrix as pandas df
    #Initialize adjacency matrix
    adj_mat_df = pd.DataFrame(np.zeros([n,n]),dtype='bool')
    #Make starting connections with pbc
    for i in range(0,n):
        #left
        if(i-r>=0):
            adj_mat_df.iloc[i][i-r:i] = True
        else:
            diff = r-i
            adj_mat_df.iloc[i][0:i] = True
            adj_mat_df.iloc[i][n-diff:n+1] = True
        #right
        if(i+r<n):
            adj_mat_df.iloc[i][i+1:i+r+1] = True #+1 to avoid self loop and up to sym value
        else:
            diff = i+r-n
            adj_mat_df.iloc[i][i+1:n+1] = True
            adj_mat_df.iloc[i][0:diff+1] = True

    return adj_mat_df

# In[4]:

def create_strogatz(n, r ,p, place_labels=False ):
    """Plots the graph of the Strogatz model on a unit circle."""

    #Procedure to create results folder automatically
    path = os.getcwd()
    results_dir = "/results_WS"
    try:
        os.mkdir(path+results_dir)
    except OSError:
        print ("Creation of the directory %s failed" % results_dir)
    else:
        print ("Successfully created the directory %s " % results_dir)
    #names for file paths
    name_plot = "/plot_n%d_r%d_p%.3f.png" %(n,r,p)
    name_csv = "/data_n%d_r%d_p%.3f.csv" %(n,r,p)
    name_plot_rewired = "/plot_rewired_n%d_r%d_p%.3f.png" %(n,r,p)
    name_csv_rewired = "/data_rewired_n%d_r%d_p%.3f.csv" %(n,r,p)

    #check for errors
    if p>1 or p<0:
        raise Exception("Wrong input: \n p must be in [0,1]")
    coords = coordinates_on_circle(n)
    adj_mat = create_starting_graph(n,r)
    labels_nodes = []
    nodes_coords = coordinates_on_circle(n)
    #figure settings
    fig, (ax1, ax2) = plt.subplots(1, 2,figsize=(16,9))
    plt.subplots_adjust(wspace=0.3)
    plt.suptitle("WS(N=%d; 2r = %d), Starting configuration"%(n,2*r),fontsize=25)
    #plot graph
    for i in range(0,n):
        connections_list = adj_mat[adj_mat.iloc[i] == True].index.tolist()
    #print(connections_list)
        for k in range(0,len(connections_list)):
            ax1.plot([nodes_coords[0][i],nodes_coords[0][connections_list[k]]],[nodes_coords[1][i],nodes_coords[1][connections_list[k]]],linewidth=.5,color='indianred')
    ax1.plot(nodes_coords[0],nodes_coords[1],color='steelblue',ls='none',marker='o',markersize=10,label=labels_nodes)
    ax1.set_title("Graph representation",fontsize=20)
    ax1.set_xticks([])
    ax1.set_yticks([])
    #labels on vertices
    if place_labels==True:
        for i in range(0,n):
            labels_nodes.append("%d"%i)
            ax1.text(nodes_coords[0][i],nodes_coords[1][i],labels_nodes[i],fontsize=15)

    #plot adjacency matrix
    ax2.set_xlabel("Edges",fontsize=20)
    ax2.set_ylabel("Vertices",fontsize=20)
    ax2.matshow(adj_mat,cmap='cividis')
    ax2.set_title("Adjacency matrix",fontsize=25)

    #save things!
    adj_mat.to_csv(path+results_dir+name_csv,header=False, index=False)
    plt.savefig(path+results_dir+name_plot,dpi=200)
    plt.show()

    #print("PRE REWIRING:",sum(adj_mat))

    #rewiring! (anticlockwise, for sake of indices)

    for i in range(0,n):
        #print("working on row # %d"%(i))
        #edge_list = list(adj_mat[adj_mat.iloc[i] == True].index.tolist())
        #edge_list = [k for k in edge_list if k > i]
        for j in range(0,r):  #for each link to vertex i

            if (random.random()<p): #attempt a rewire
                #performing the rewire
                #    - Choose which of the connected edge to rewire->deleated_edge
                #    - Choose were to rewire it among the available positions->candidates
                #    - Perform the connection/deleate old connection/update mirror adjmat

                #choose which edge to remove: [+periodic boundary conditions]
                deleated_edge = i+1+j
                if deleated_edge>n-1:
                    deleated_edge = deleated_edge-n

                #chose available position:
                candidates = list(adj_mat[adj_mat.iloc[i] == False].index.tolist())
                candidates.remove(i) #take out self loop
                new_edge = random.choice(candidates)

                #print("candidates list = ",candidates)
                #print("new edge chosen = ",new_edge)

                #create new wire
                adj_mat.iloc[i][new_edge]=True
                adj_mat.iloc[new_edge][i]=True

                #deleate old wire
                adj_mat.iloc[i][deleated_edge]=False
                adj_mat.iloc[deleated_edge][i]=False

    #print("AFTER REWIRING:",sum(adj_mat))

    #Plot rewired

    fig, (ax1, ax2) = plt.subplots(1, 2,figsize=(16,9))
    plt.subplots_adjust(wspace=0.3)
    plt.suptitle("WS(N=%d; 2r = %d; p = %.3f)"%(n,2*r, p),fontsize=25)

    #plot graph
    for i in range(0,n):
        connections_list = adj_mat[adj_mat.iloc[i] == True].index.tolist()
    #print(connections_list)
        for k in range(0,len(connections_list)):
            ax1.plot([nodes_coords[0][i],nodes_coords[0][connections_list[k]]],[nodes_coords[1][i],nodes_coords[1][connections_list[k]]],linewidth=.5,color='indianred')
    ax1.plot(nodes_coords[0],nodes_coords[1],color='steelblue',ls='none',marker='o',markersize=10,label=labels_nodes)
    ax1.set_title("Graph representation",fontsize=20)
    ax1.set_xticks([])
    ax1.set_yticks([])
    #labels on vertices
    if place_labels==True:
        for i in range(0,n):
            labels_nodes.append("%d"%i)
            ax1.text(nodes_coords[0][i],nodes_coords[1][i],labels_nodes[i],fontsize=15)

    #plot adjacency matrix
    ax2.set_xlabel("Edges",fontsize=20)
    ax2.set_ylabel("Vertices",fontsize=20)
    ax2.matshow(adj_mat,cmap='cividis')
    ax2.set_title("Adjacency matrix",fontsize=25)

    #save things!
    adj_mat.to_csv(path+results_dir+name_csv_rewired,header=False, index=False)
    plt.savefig(path+results_dir+name_plot_rewired,dpi=200)
    plt.show()
    return adj_mat

# In[82]:

n,r,p=50,5,.3
adj_mat = create_strogatz(n,r,p)


import numpy as np
import scipy.io
import matplotlib.pyplot as plt

# Import pagerank_adj.mat matrix (2571 x 2571)
mat = scipy.io.loadmat('pagerank_adj.mat')["J"]

x = np.ones((2571, 1))

# Sum of all columns in mat
column_sums = np.zeros(2571)
for col in range(2571):
    for row in range(2571):
        column_sums[col] += mat[row][col]

# Create matrix A
A = np.zeros((2571, 2571))
for row in range(2571):
    for col in range(2571):
        A[row][col] = mat[row][col] / column_sums[col]

# Verify columns of A sum to 1 using matrix multiplication
result = A.transpose().dot(x)  # Results in vector of ones

# Normalize x
x_norm = x / np.linalg.norm(x)

# Vector of log of error terms
log_error_PI = np.zeros(10)
log_error_SIPI = np.zeros(10)
log_error_rayleigh = np.zeros(10)

# Power iteration algorithm
for k in range(10):
    y = A.dot(x_norm)
    x_norm = y / np.linalg.norm(y)
    evalue = x_norm.transpose().dot(A.dot(x_norm))
    log_error_PI[k] = np.log10(np.linalg.norm(A.dot(x_norm) - x_norm))

# Shift-invert power iteration algorithm
x_norm = x / np.linalg.norm(x)
sigma = 0.99
for k in range(10):
    y = np.linalg.inv(A - sigma * np.identity(2571)).dot(x_norm)
    x_norm = y / np.linalg.norm(y)
    evalue = x_norm.transpose().dot(A.dot(x_norm))
    log_error_SIPI[k] = np.log10(np.linalg.norm(A.dot(x_norm) - x_norm))

# Rayleigh quotient iteration algorithm
x_norm = x / np.linalg.norm(x)
for k in range(10):
    # Index starts at 0
    if k > 1:
        sigma = (x_norm.transpose().dot(A).dot(x_norm)) / (x_norm.transpose().dot(x_norm))
    y = np.linalg.inv(A - sigma * np.identity(2571)).dot(x_norm)
    x_norm = y / np.linalg.norm(y)
    log_error_rayleigh[k] = np.log10(np.linalg.norm(A.dot(x_norm) - x_norm))

# Plot log of the error
plt.plot(range(10), log_error_PI, label="Power iteration error")
plt.plot(range(10), log_error_SIPI, label="Shift-invert power iteration error")
plt.plot(range(10), log_error_rayleigh, label="Rayleigh quotient iteration error")
plt.xlabel('k value')
plt.ylabel('log(e(k+1))')
plt.title('Log error value plot')
plt.legend()

# PageRank score is the approximated eigenvector x(k+1)
top_five_indices = sorted(range(len(x_norm)), key=lambda i: x_norm[i])[-5:]
bottom_five_indices = sorted(range(len(x_norm)), key=lambda i: x_norm[i])[:5]

for p in top_five_indices: print(p)
for p in bottom_five_indices: print(p)

plt.show()


import numpy as np

def getAngleOfCircle(percentage):
    return np.around(2 * np.pi * percentage / 100.0, 5)

def pointWithinRadius(x, y):
    if (x**2 + y**2) <= 50 * 50:
        return True
    return False

def getPointAngle(x, y):
    ang1 = np.arctan2(1, 0)
    ang2 = np.arctan2(y, x)
    angle = np.around((ang1 - ang2) % (2 * np.pi), 5)
    return angle

def withinAngleOfCircle(circleAngle, pointAngle):
    if pointAngle <= circleAngle:
        return True
    return False

def check(p, x, y):
    if p == 0:
        return "white"
    if withinAngleOfCircle(getAngleOfCircle(p), getPointAngle(x, y)) and pointWithinRadius(x, y):
        return "black"
    return "white"

def __main__():
    f = open("progress_pie.txt", 'r')
    o = open('out.txt', 'w')
    noOfTests = int(f.readline())
    for i in range(0, noOfTests):
        input = f.readline().split()
        output = check(int(input[0]), int(input[1]) - 50, int(input[2]) - 50)
        o.write("Case #" + str(i + 1) + ": " + output + "\n")
    o.close()
    f.close()

if __name__ == '__main__':
    __main__()


from __future__ import annotations
from typing import Union
import numpy as np
import random

class KMeans:
    """KMeans
    Parameters:
    -----------
    n_clusters: int = 2
        Number of clusters
    tol: float = 0.0001
        Relative tolerance
    max_iter: int = 300
        Maximum number of iterations of the k-means algorithm for a single run
    """
    def __init__(self, n_clusters: int = 2, tol: float = 0.0001, max_iter: int = 300) -> None:
        self.n_clusters = n_clusters
        self.tol = tol
        self.max_iter = max_iter
        self.centroids = {}
        self.classifications = {}

    def fit(self, data: Union[list, np.ndarray]) -> KMeans:
        self.centroids = {}

        # Randomly initialize centroids
        for i in range(self.n_clusters):
            self.centroids[i] = data[random.randint(0, len(data)-1)]

        for _ in range(self.max_iter):
            self.classifications = {}

            for i in range(self.n_clusters):
                self.classifications[i] = []

            # put row into the right cluster
            for row in data:
                distances = [np.linalg.norm(
                    row-self.centroids[centroid]) for centroid in self.centroids]
                classification = distances.index(min(distances))
                self.classifications[classification].append(row)

            prev_centroids = dict(self.centroids)

            for classification in self.classifications:
                self.centroids[classification] = np.average(
                    self.classifications[classification], axis=0)

            # Check if we are finished
            optimized = True
            for c in self.centroids:
                original_centroid = prev_centroids[c]
                current_centroid = self.centroids[c]
                if np.sum((current_centroid - original_centroid) / original_centroid * 100.0) > self.tol:
                    optimized = False

            if optimized:
                break
        return self

    def predict(self, data: Union[list, np.ndarray]) -> list:
        classifications = []
        for row in data:
            distances = [np.linalg.norm(row-self.centroids[centroid])
                         for centroid in self.centroids]
            classification = distances.index(min(distances))
            classifications.append(classification)
        return classifications

if __name__ == '__main__':
    import matplotlib.pyplot as plt
    from matplotlib import style
    from sklearn.datasets import make_blobs
    style.use('ggplot')

    X, y = make_blobs(n_samples=30, centers=3, n_features=2)

    model = KMeans(n_clusters=3)
    model.fit(X)

    centroids = model.centroids

    colors = 10 * ['r', 'g', 'b', 'c', 'k', 'y']

    for classification, featureset in zip(model.predict(X), X):
        color = colors[classification]
        plt.scatter(featureset[0], featureset[1], marker="x",
                    color=color, s=150, linewidths=5, zorder=10)

    for c in centroids:
        plt.scatter(centroids[c][0], centroids[c][1],
                    color='k', marker="*", s=150, linewidths=5)

    plt.show()


import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.animation as animation

data = pd.read_csv('data/train.csv')

# Grab the relevant data, scale the predictor variable,
# and add a column of 1s for the gradient descent...
x = data['GrLivArea']
y = data['SalePrice']

x = (x - x.mean()) / x.std()
x = np.c_[np.ones(x.shape[0]), x]

# GRADIENT DESCENT

alpha = 0.01  # Step size
iterations = 2000  # No. of iterations
m = y.size  # No. of data points
np.random.seed(123)  # Set the seed
theta = np.random.rand(2)  # Pick some random values to start with

# GRADIENT DESCENT
def gradient_descent(x, y, theta, iterations, alpha):
    past_costs = []
    past_thetas = [theta]
    for i in range(iterations):
        prediction = np.dot(x, theta)
        error = prediction - y
        cost = 1/(2*m) * np.dot(error.T, error)
        past_costs.append(cost)
        theta = theta - (alpha * (1/m) * np.dot(x.T, error))
        past_thetas.append(theta)

    return past_thetas, past_costs

# Pass the relevant variables to the function and get the new values back...
past_thetas, past_costs = gradient_descent(x, y, theta, iterations, alpha)
theta = past_thetas[-1]

# Print the results...
print("Gradient Descent: {:.2f}, {:.2f}".format(theta[0], theta[1]))

# Plot the cost function...
plt.title('Cost Function J')
plt.xlabel('No. of iterations')
plt.ylabel('Cost')
plt.plot(past_costs)
plt.show()

# Animation

# Set the plot up,
fig = plt.figure()
ax = plt.axes()
plt.title('Sale Price vs Living Area')
plt.xlabel('Living Area in square feet (normalised)')
plt.ylabel('Sale Price ($)')
plt.scatter(x[:, 1], y, color='red')
line, = ax.plot([], [], lw=2)
annotation = ax.text(-1, 700000, '')
annotation.set_animated(True)
plt.close()

# Generate the animation data,
def init():
    line.set_data([], [])
    annotation.set_text('')
    return line, annotation

# animation function.  This is called sequentially
def animate(i):
    x = np.linspace(-5, 20, 1000)
    y = past_thetas[i][1]*x + past_thetas[i][0]
    line.set_data(x, y)
    annotation.set_text('Cost = %.2f e10' % (past_costs[i]/10000000000))
    return line, annotation

anim = animation.FuncAnimation(fig, animate, init_func=init,
                               frames=300, interval=0, blit=True)

anim.save('animation.gif', writer='imagemagick', fps=30)


import numpy as np
import sys

print(f'Python Version is: {sys.version}')

# Define a Scalar
x = 6

# Define a Vector
z = np.array((1, 2, 3, 4))

print(f'Vector Dimension {z.shape}')
print(f'Vector size {z.size}')

# Define a matrix

m = np.array(([1, 2, 3],
              [4, 5, 6],
              [7, 8, 9]))

print(f'Matrix Dimension {m.shape}')
print(f'Matrix size {m.size}')

one = np.zeros((3, 4), dtype=np.float)
"""
Output:

[[0. 0. 0. 0.]
 [0. 0. 0. 0.]
 [0. 0. 0. 0.]]
"""

# Define a Tensor

tensor = np.ones((2, 2, 2), dtype=np.float)
"""
Output:
[[[1. 1.]
  [1. 1.]]

 [[1. 1.]
  [1. 1.]]]
"""

# Indexing

tensor[0, 1, 1] = 3.4
print(tensor)
"""
Output:
[[[1.  1. ]
  [1.  3.4]]

 [[1.  1. ]
  [1.  1. ]]]
"""

# Matrix Transpose

array = np.arange(6).reshape((2, 3))
"""
Output:
[[0 1 2]
 [3 4 5]]
"""
array = array.transpose()
"""
Output:
[[0 3]
 [1 4]
 [2 5]]
"""


#!/usr/bin/env python3
"""
Operations for laplacian decomposition.

Attributes
----------
LGR
    Logger
"""

import logging

import numpy as np

LGR = logging.getLogger(__name__)

def symmetric_normalisation(mtx):
    """
    Compute symmetrically normalised Laplacian matrix.

    Parameters
    ----------
    mtx : numpy.ndarray
        A [structural] matrix

    Returns
    -------
    numpy.ndarray
        The symmetrically normalised version of mtx

    See Also
    --------
    https://en.wikipedia.org/wiki/Laplacian_matrix#Symmetrically_normalized_Laplacian_2
    """
    d = np.diag(mtx.sum(axis=-1) ** (-1/2))

    symm_norm = (d @ mtx @ d)

    return (np.eye(mtx.shape[0]) - symm_norm)

def decomposition(mtx):
    """
    Run a eigenvector decomposition on input.

    Parameters
    ----------
    mtx : numpy.ndarray
        The matrix to decompose

    Returns
    -------
    numpy.ndarray
        The eigenvalues resulting from the decomposition
    numpy.ndarray
        The eigenvectors resulting from the decomposition
    """
    eigenval, eigenvec = np.linalg.eig(mtx)

    idx = np.argsort(eigenval)
    eigenval = eigenval[idx]
    # #!# Check that eigenvec has the right index and not inverted
    eigenvec = eigenvec[:, idx]

    return eigenval, eigenvec

"""
Copyright 2022, Stefano Moia.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""


from numpy import zeros

# Define euler_set function
def euler_set(f,t0,tf,y0,v0,n):
    h = (tf - t0)/n
    t = zeros(n+1)
    y = zeros(n+1)
    v = zeros(n+1)
    t[0] = t0
    y[0] = y0
    v[0] = v0
    for i in range(n):
        y[i+1] = y[i] + h*v[i]
        v[i+1] = v[i] + h*f(t[i],y[i],v[i])
        t[i+1] = t[i] + h
    return t,y,v

# Define functions
def f(t,y,v):
    return (t**2 - 1.0)*y

# Set initial conditions
t0 = 0
tf = 1
y0 = 1.0
v0 = 0.0
n = 10

# Execute EULER_SET
t, y, v = euler_set(f,t0,tf,y0,v0,n)

# Print results
print('%5s %8s %8s' % ('t','y','v'))
for i in range(n+1):
    print("%9.4f %9.4f %9.4f" % (t[i],y[i],v[i]))


# -*- coding: utf-8 -*-
import sympy as sp

p = sp.symbols('p')

print("log'(p) = {0}".format(sp.diff(sp.log(p)).simplify()))
print("(p*log(p))' = {0}".format(sp.diff(p * sp.log(p)).simplify()))
print("((1-p)*log(1-p))' = {0}".format(sp.diff((1-p) * sp.log(1 - p)).simplify()))

H = -(p * sp.log(p) + (1 - p) * sp.log(1 - p))

print("H'(p) = {0}".format(sp.diff(H, p).simplify()))


import numpy as np
from scipy import stats
import math

x = np.array([5,15,25,35,45])
x_square = np.square(x)

f = np.array([6,16,24,25,19])
cum_freq = np.array([6,22,46,71,90])
f_x_square = f*x_square

# mean
mean = np.average(x,weights=f)

# mode for now find manually
l = 30
f0 = 10
f1 = 12
f2 = 8
c = 10
mode  = l + ((f1 - f0)/(2*f1 - c*f0 + f2))*c
mode  = 31.4

# standard deviation
sum_f_x_square = np.sum(f_x_square)
sum_f = np.sum(f)
mean_square = math.pow(mean,2)
standard_deviation = math.sqrt((sum_f_x_square/sum_f) - mean_square)

print("mean : ",mean)
print("mode : ",mode)
print("standard_deviation : ",standard_deviation)

coeff = (mean - mode)/standard_deviation

print("Coefficent : ",coeff)


# Kalman Filtering Example
#
# Noisy Sensor
#
# Author: Adam Noel
#

import random
import numpy as np
import matplotlib.mlab as mlab
import matplotlib.pyplot as plt

class NoisySensor:
	def __init__(self, sensorValue, noise):
		self.sensorValue = sensorValue
		self.noise = noise
	def getValue(self):
		return (self.sensorValue + np.random.normal(0, 0.5))

class Averager:
	def __init__(self, initialValue):
		self.currentValue = initialValue
		self.numValues = 1
	def updateEstimate(self, newObservation):
		self.numValues = self.numValues + 1
		self.currentValue = (1/float(self.numValues))*newObservation + ((self.numValues - 1)/float(self.numValues))*self.currentValue
	def getValue(self):
		return self.currentValue

class KalmanFilter:
	def __init__(self, shi, Q, H, R, x, P):
		self.shi = shi
		self.Q = Q
		self.H = H
		self.R = R
		self.x = x
		self.P = P
	def updateState(self, newObservation):
		#Now, project to current state
		self.x = self.shi.dot(self.x)
		self.P = self.shi.dot(self.P).dot(np.transpose(self.shi)) + self.Q

		#First, compute the Kalman Gain
		innerKalmanProduct = np.linalg.inv(H.dot(P).dot(np.transpose(H)) + R)
		kalmanGain = self.P.dot(np.transpose(H)).dot(innerKalmanProduct)

		#Next, update the estimate
		self.x = self.x + kalmanGain.dot(np.array(newObservation) - H.dot(self.x))

		#Next, update the error covariance matrix
		sizeP = self.P.shape[0] #Get the shape of the error matrix
		self.P = (np.eye(sizeP) - kalmanGain.dot(H)).dot(self.P)

	def getState(self):
		return np.asarray(self.x)[0][0] ##Convert from matrix to array

numSteps = 100
sensorNoise = 0.5
sensorValue = 3
firstStateEstimate = 2.6
firstErrorEstimate = 0.5

actualValues = []
observedValues = []
averagedValues = [] #Averager to serve as a kalman filter reference
kalmanValues = []

shi = np.matrix([1])
Q = np.matrix([0.00001])
H = np.matrix([1])
R = np.matrix([sensorNoise])
x = np.matrix([firstStateEstimate])
P = np.matrix([firstErrorEstimate])

kalmanFilter = KalmanFilter(shi, Q, H, R, x, P)
noisySensor = NoisySensor(sensorValue, sensorNoise)
averager = Averager(firstStateEstimate)

for i in range(0, numSteps):
	newObservation = noisySensor.getValue()
	actualValues.append(sensorValue)
	observedValues.append(newObservation)
	averager.updateEstimate(newObservation)
	averagedValues.append(averager.getValue())
	kalmanFilter.updateState(newObservation)
	kalmanValues.append(kalmanFilter.getState())

steps = range(0, numSteps)
plt.plot(steps, actualValues, 'r--', label='Actual')
plt.plot(steps, observedValues, 'bs', label='Observed')
plt.plot(steps, averagedValues, 'g^', label='Averaged')
plt.plot(steps, kalmanValues, 'ko', label='Kalman')
plt.legend(loc='lower right')
#plt.plot(steps, actualValues, 'r--', label='Actual', steps, observedValues, 'bs', label='Observed', steps, averagedValues, 'g^', label='Averaged', steps, kalmanValues, 'ko', label='Kalman')
plt.show()


import numpy as np
import matplotlib.pyplot as plt

def plot_complex_exponential(A, f, phi, fs):

    Ts    = 1/fs
    Tr    = 1/(f*100)
    ts    = np.arange(0, 1+Ts, Ts)
    tr    = np.arange(0, 1+Tr, Tr)
    cexps = np.exp(1j*(2*np.pi*f*ts+phi))
    cexpr = np.exp(1j*(2*np.pi*f*tr+phi))

    real_cexpr = np.real(cexpr)
    imag_cexpr = np.imag(cexpr)

    real_cexps = np.real(cexps)
    imag_cexps = np.imag(cexps)

    ys     = [(real_cexps, real_cexpr), (imag_cexps, imag_cexpr)]
    titles = ["Parte real", "Parte imaginária"]
    fig, axs = plt.subplots(2,1,figsize = (20,6))

    for y, title, ax in zip(ys, titles, axs.flatten()):
        y_sample, y_real = y
        ax.scatter(ts, y_sample, label = "Amostras", c = 'k')
        ax.plot(tr, y_real, label = '"Dados reais"', c='r')
        ax.set_xlabel("Tempo (s)", fontsize = 14)
        ax.set_ylabel("Amplitude",fontsize = 14)
        ax.set_title(title,fontsize = 22)
        ax.legend(fontsize=16)

    plt.tight_layout()
    plt.show()

def plot_DFT(input_signal, plot=True):

    X_k = []

    N  = len(input_signal)
    ns = np.arange(-N/2, N/2)
    ks = np.arange(-N/2, N/2)

    for k in ks:
        s_k = np.exp(-1j*(2*np.pi*k)/N * ns)
        X_k.append(sum(input_signal*s_k))

    if plot:
        fig, axs = plt.subplots(1,2,figsize = (20,6))

        axs[0].stem(ks, np.abs(X_k))
        axs[0].set_ylabel("Decibels",fontsize = 16)

        axs[1].stem(ks, np.unwrap(np.angle(X_k,deg=True),discont=180))
        axs[1].set_ylabel("Graus",fontsize = 16)

        for ax in axs.flatten():
            ax.set_xlabel("Frequência",fontsize = 16)
            ax.set_xlim(-N/2, N/2-1)

        plt.suptitle("Discrete Fourier Transform",fontsize = 22)
        plt.tight_layout()
        plt.show()

    return X_k


import numpy as np
rows = [[1, 2, 1],
        [3, 8, 1],
        [0, 4,  1]]

A = np.array(rows)
rank_A = np.linalg.matrix_rank(A)
assert rank_A == 3

## So the vectors are linear independent. And they span R3. The vectors form a basis in R3.

#Check the dimension
dim_A = A.shape

assert dim_A[0] == 3
assert dim_A[1] == 3

print(dim_A)


import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression

df = pd.read_csv('housing.data.txt', header=None, sep='\s+')
df.columns = ['CRIM', 'ZN', 'INDUS', 'CHAS',
              'NOX', 'RM', 'AGE', 'DIS', 'RAD',
              'TAX', 'PTRATIO', 'B', 'LSTAT', 'MEDV']

X = df[['RM']].values
y = df['MEDV'].values

slr = LinearRegression()
slr.fit(X, y)
y_pred = slr.predict(X)

print('Slope: %.3f' % slr.coef_[0])
print('Intercept: %.3f' % slr.intercept_)

def lin_regplot(X, y, model):
    plt.scatter(X, y, c='steelblue', edgecolor='white', s=70)
    plt.plot(X, model.predict(X), color='black', lw=2)
    return

lin_regplot(X, y, slr)
plt.xlabel('Average number of rooms [RM]')
plt.ylabel('Price in $1000s [MEDV]')
plt.show()

# Normal Equations: Alternative:
# adding a column vector of "ones"
Xb = np.hstack((np.ones((X.shape[0], 1)), X))
w = np.zeros(X.shape[1])
z = np.linalg.inv(np.dot(Xb.T, Xb))
w = np.dot(z, np.dot(Xb.T, y))

print('Slope: %.3f' % w[1])
print('Intercept: %.3f' % w[0])

Chapter 10/Implementing an ordinary least squares linear regression model/program2.py


# Cálculo da razão áurea (phi)
import sympy

d = 20
phi = sympy.symbols('phi', nonnegative=True)

eqn = sympy.Eq(1/phi, phi - 1)

sol = sympy.solve(eqn)
sympy.pprint(sol)
phiAprox = sympy.N(sol[0], d)

print('Para ', d, ' dígitos significativos, ϕ = ', phiAprox)


#import multiple libraries
import matplotlib.pyplot as plt
import numpy as np
from matplotlib import animation
import time

#CONSTANTS
#---------
iterations=1000 #quantity of "parts" that the flower is going to be divided by (the larger it becomes, the smoother it looks, the more time it takes)
fps = 100 #frames per second on the animation
make_animated = True #animation boolean
sleep_seconds = 10 #time the flower pauses after being plotted
color = 'red' #specify the color of the rose

#CODE
#----
first = True #boolean to check when to pause
fig = plt.figure() #initialize the figure
ax = plt.axes(xlim=(-1,1), ylim=(-1,1)) #set limits of the axes
rose, = ax.plot([],[],color) #initialize the rose, variable
user_input = input("Type in a number (in case of a fraction in its lowest terms [a/b]): ") #ask for k
coeffs = user_input.split("/") #split the user input by the fraction slash
num = float(coeffs[0]) #assign numerator value
if len(coeffs)>1: #check for a denominator
	den = float(coeffs[1]) #assign denominator value
else:
	den = 1 #if there's no denominator give a value of 1
k=num/den #assign the value to k
num_odd = False
den_odd = False
if num%2 != 0: #check if the numerator is odd
	num_odd = True #assign value True to boolean checking if the numerator is odd
if den%2 != 0: #check if the denominator is odd
	den_odd = True #assign value True to boolean checking if the denominator is odd

if num_odd and den_odd: #check if both numerator and denominator are odd
	angle = np.pi*den #assign the value of the angle to close
else:
	angle = 2*np.pi*den #assign the value of the angle to close
thetas = np.linspace(0, angle, iterations) #creation of all the values that are going to be computed
x = np.cos(k*thetas)*np.cos(thetas) #multiply by the cosinus to get all x coordinates
y = np.cos(k*thetas)*np.sin(thetas) #multiply by the sinus to get all y coordinates

def initialize(): #definition of initialize
	rose.set_data([],[]) #establish the array on the rose variable
	return rose,

def animate(i): #definition for the animation
	global first #modification of global variable final
	rose.set_data(x[0:i], y[0:i]) #add one more value of x and y to the rose array
	if i == 0 and not first: #check if the flower has been completed and it's not the first plotting
		time.sleep(sleep_seconds) #sleep for the specified time
	if first: #check if it is the very start of the plotting
		first=False; #know it's not the first time
	return rose,

if make_animated: #if animated
	animation.FuncAnimation(fig, animate, init_func=initialize, frames=iterations, interval=1000/fps, blit=True) #call the animation with all functions and user specifications
else:
	plt.plot(x,y, color) #plot the flower directly
plt.show() #show plots


# -*- coding: utf-8 -*-

import numpy as np
import scipy.stats as sts
from tqdm import tqdm
import os
# from pywt import WaveletPacket
epsn = 1e-8

# =================== STATISTICAL FEATURES IN TIME DOMAIN =====================
def mean_fea(a):
	return np.mean(a)

def rms_fea(a):
	return np.sqrt(np.mean(np.square(a)))

def sr_fea(a):
	return np.square(np.mean(np.sqrt(np.abs(a))))

def am_fea(a):
	return np.mean(np.abs(a))

def skew_fea(a):
	return np.mean((a-mean_fea(a))**3)

def kurt_fea(a):
	return np.mean((a-mean_fea(a))**4)

def max_fea(a):
	return np.max(a)

def min_fea(a):
	return np.min(a)

def pp_fea(a):
	return max_fea(a)-min_fea(a)

def var_fea(a):
	n = len(a)
	return np.sum((a-mean_fea(a))**2)/(n-1)

def waveform_index(a):
	return rms_fea(a)/(am_fea(a)+epsn)

def peak_index(a):
	return max_fea(a)/(rms_fea(a)+epsn)

def impluse_factor(a):
	return max_fea(a)/(am_fea(a)+epsn)

def tolerance_index(a):
	return max_fea(a)/(sr_fea(a)+epsn)

def skew_index(a):
	n = len(a)
	temp1 = np.sum((a-mean_fea(a))**3)
	temp2 = (np.sqrt(var_fea(a)))**3
	return temp1/((n-1)*temp2)

def kurt_index(a):
	n = len(a)
	temp1 = np.sum((a-mean_fea(a))**4)
	temp2 = (np.sqrt(var_fea(a)))**4
	return temp1/((n-1)*temp2)
# ============================= END ======================================
# def wave_fea(a):
# 	wp = WaveletPacket(a,'db1', maxlevel=8)
# 	nodes = wp.get_level(8, "freq")
# 	return np.linalg.norm(np.array([n.data for n in nodes]), 2)

# =============== STATISTICAL FEATURES IN TIME DOMAIN =======================
# def fft_fft(sequence_data):
# 	fft_trans = np.abs(np.fft.fft(sequence_data))
# 	dc = fft_trans[0]
# 	freq_spectrum = fft_trans[1:int(np.floor(len(sequence_data) * 1.0 / 2)) + 1]
# 	freq_sum_ = np.sum(freq_spectrum)
# 	return dc, freq_spectrum, freq_sum_

def fft_mean(sequence_data):
	def fft_fft(sequence_data):
		fft_trans = np.abs(np.fft.fft(sequence_data))
		# dc = fft_trans[0]
		freq_spectrum = fft_trans[1:int(np.floor(len(sequence_data) * 1.0 / 2)) + 1]
		_freq_sum_ = np.sum(freq_spectrum)
		return freq_spectrum, _freq_sum_
	freq_spectrum, _freq_sum_ = fft_fft(sequence_data)
	return np.mean(freq_spectrum)

def fft_var(sequence_data):
	def fft_fft(sequence_data):
		fft_trans = np.abs(np.fft.fft(sequence_data))
		# dc = fft_trans[0]
		freq_spectrum = fft_trans[1:int(np.floor(len(sequence_data) * 1.0 / 2)) + 1]
		_freq_sum_ = np.sum(freq_spectrum)
		return freq_spectrum, _freq_sum_
	freq_spectrum, _freq_sum_ = fft_fft(sequence_data)
	return np.var(freq_spectrum)

# def fft_std(sequence_data):
# 	def fft_fft(sequence_data):
# 		fft_trans = np.abs(np.fft.fft(sequence_data))
# 		# dc = fft_trans[0]
# 		freq_spectrum = fft_trans[1:int(np.floor(len(sequence_data) * 1.0 / 2)) + 1]
# 		_freq_sum_ = np.sum(freq_spectrum)
# 		return freq_spectrum, _freq_sum_
# 	freq_spectrum, _freq_sum_ = fft_fft(sequence_data)
# 	return np.std(freq_spectrum)

def fft_entropy(sequence_data):
	def fft_fft(sequence_data):
		fft_trans = np.abs(np.fft.fft(sequence_data))
		# dc = fft_trans[0]
		freq_spectrum = fft_trans[1:int(np.floor(len(sequence_data) * 1.0 / 2)) + 1]
		_freq_sum_ = np.sum(freq_spectrum)
		return freq_spectrum, _freq_sum_
	freq_spectrum, _freq_sum_ = fft_fft(sequence_data)
	pr_freq = freq_spectrum * 1.0 / _freq_sum_
	entropy = -1 * np.sum([np.log2(p+1e-5) * p for p in pr_freq])
	return entropy

def fft_energy(sequence_data):
	def fft_fft(sequence_data):
		fft_trans = np.abs(np.fft.fft(sequence_data))
		# dc = fft_trans[0]
		freq_spectrum = fft_trans[1:int(np.floor(len(sequence_data) * 1.0 / 2)) + 1]
		_freq_sum_ = np.sum(freq_spectrum)
		return freq_spectrum, _freq_sum_
	freq_spectrum, _freq_sum_ = fft_fft(sequence_data)
	return np.sum(freq_spectrum ** 2) / len(freq_spectrum)

def fft_skew(sequence_data):
	def fft_fft(sequence_data):
		fft_trans = np.abs(np.fft.fft(sequence_data))
		# dc = fft_trans[0]
		freq_spectrum = fft_trans[1:int(np.floor(len(sequence_data) * 1.0 / 2)) + 1]
		_freq_sum_ = np.sum(freq_spectrum)
		return freq_spectrum, _freq_sum_
	freq_spectrum, _freq_sum_ = fft_fft(sequence_data)
	_fft_mean, _fft_std = fft_mean(sequence_data), fft_std(sequence_data)
	return np.mean([0 if _fft_std < epsn else np.power((x - _fft_mean) / _fft_std, 3)
					for x in freq_spectrum])

def fft_kurt(sequence_data):
	def fft_fft(sequence_data):
		fft_trans = np.abs(np.fft.fft(sequence_data))
		# dc = fft_trans[0]
		freq_spectrum = fft_trans[1:int(np.floor(len(sequence_data) * 1.0 / 2)) + 1]
		_freq_sum_ = np.sum(freq_spectrum)
		return freq_spectrum, _freq_sum_
	freq_spectrum, _freq_sum_ = fft_fft(sequence_data)
	_fft_mean, _fft_std = fft_mean(sequence_data), fft_std(sequence_data)
	return np.mean([0 if _fft_std < epsn else np.power((x - _fft_mean) / _fft_std, 4)
					for x in freq_spectrum])

def fft_shape_mean(sequence_data):
	def fft_fft(sequence_data):
		fft_trans = np.abs(np.fft.fft(sequence_data))
		# dc = fft_trans[0]
		freq_spectrum = fft_trans[1:int(np.floor(len(sequence_data) * 1.0 / 2)) + 1]
		_freq_sum_ = np.sum(freq_spectrum)
		return freq_spectrum, _freq_sum_
	freq_spectrum, _freq_sum_ = fft_fft(sequence_data)
	shape_sum = np.sum([x * freq_spectrum[x]
						for x in range(len(freq_spectrum))])
	return 0 if _freq_sum_ < epsn else shape_sum * 1.0 / _freq_sum_

def fft_shape_std(sequence_data):
	def fft_fft(sequence_data):
		fft_trans = np.abs(np.fft.fft(sequence_data))
		# dc = fft_trans[0]
		freq_spectrum = fft_trans[1:int(np.floor(len(sequence_data) * 1.0 / 2)) + 1]
		_freq_sum_ = np.sum(freq_spectrum)
		return freq_spectrum, _freq_sum_
	freq_spectrum, _freq_sum_ = fft_fft(sequence_data)
	shape_mean = fft_shape_mean(sequence_data)
	var = np.sum([0 if _freq_sum_ < epsn else np.power((x - shape_mean), 2) * freq_spectrum[x]
				  for x in range(len(freq_spectrum))]) / _freq_sum_
	return np.sqrt(var)

def fft_shape_skew(sequence_data):
	def fft_fft(sequence_data):
		fft_trans = np.abs(np.fft.fft(sequence_data))
		# dc = fft_trans[0]
		freq_spectrum = fft_trans[1:int(np.floor(len(sequence_data) * 1.0 / 2)) + 1]
		_freq_sum_ = np.sum(freq_spectrum)
		return freq_spectrum, _freq_sum_
	freq_spectrum, _freq_sum_ = fft_fft(sequence_data)
	shape_mean = fft_shape_mean(sequence_data)
	return np.sum([np.power((x - shape_mean), 3) * freq_spectrum[x]
				   for x in range(len(freq_spectrum))]) / _freq_sum_

def fft_shape_kurt(sequence_data):
	def fft_fft(sequence_data):
		fft_trans = np.abs(np.fft.fft(sequence_data))
		# dc = fft_trans[0]
		freq_spectrum = fft_trans[1:int(np.floor(len(sequence_data) * 1.0 / 2)) + 1]
		_freq_sum_ = np.sum(freq_spectrum)
		return freq_spectrum, _freq_sum_
	freq_spectrum, _freq_sum_ = fft_fft(sequence_data)
	shape_mean = fft_shape_mean(sequence_data)
	return np.sum([np.power((x - shape_mean), 4) * freq_spectrum[x] - 3
				   for x in range(len(freq_spectrum))]) / _freq_sum_
# =================================== END =====================================


"""
one hot encoding allows us to turn nominal categorical data into features
with numerical values, while not mathematically imply any ordinal relationship
between the classes.
"""

# load libraries
import numpy as np
import pandas as pd
from sklearn.preprocessing import OneHotEncoder

# Create NumPy array
x = np.array([['Texas'],
              ['California'],
              ['Texas'],
              ['Delaware'],
              ['Texas']])

# method one
# Create LabelBinzarizer object
one_hot = OneHotEncoder()

# One-hot encode data
one_hot.fit_transform(x)

# view classes
one_hot.categories_

# dumy feature
pd.get_dummies(x[:,0])


#!/usr/bin/env python

"""
This script tries to discover the secret behind the Winograd algorithm.
"""

import numpy as np

def FIR(x, b):
    """
    Standard FIR algorithm implementation.

    :param x: an input array
    :param b: a coefficient array of length r
    :return: FIR result
    """
    r = len(b)
    y = np.zeros(len(x) - r + 1)
    for i in range(len(y)):
        y[i] = np.sum(x[i:i+r] * b)
    return y

def winograd_FIR(x, b, p = 2):
    """
    Winograd FIR algorithm implementation.

    :param x: an input array
    :param b: a coefficient array of length r
    :param p: Winograd matrix, number of rows
    :return: FIR result
    """

    r = len(b)
    n = len(x)
    y = np.zeros(n - r + 1)
    m = np.zeros(r + p - 1)

    assert p == 2, "only support p = 2"
    assert len(y) % p == 0, "len(y) should be divisible by p"

    num_tiles = int(len(y) / p)

    for t in range(num_tiles):
        i = t * p
        d = x[i:i+len(m)]

        m[0] = (d[0] - d[2]) * b[0]
        m[1] = (d[1] + d[2]) * (b[0] + b[1] + b[2]) / 2
        m[2] = (d[2] - d[1]) * (b[0] - b[1] + b[2]) / 2
        m[3] = (d[1] - d[3]) * b[2]

        y[i] = m[0] + m[1] + m[2]
        y[i + 1] = m[1] - m[2] - m[3]

    return y

def winograd_FIR_mm(x, b, p = 2):
    """
    Winograd FIR implemented by matrix multiplication.

    :param x: an input array
    :param b: a coefficient array of length r
    :param p: Winograd matrix, number of rows
    :return: FIR result
    """

    B = np.array([[1, 0, -1, 0], [0, 1, 1, 0], [0, -1, 1, 0], [0, 1, 0, -1]]).T
    G = np.array([[1, 0, 0], [0.5, 0.5, 0.5], [0.5, -0.5, 0.5], [0, 0, 1]])
    A = np.array([[1, 1, 1, 0], [0, 1, -1, -1]]).T

    r = len(b)
    n = len(x)
    y = np.zeros(n - r + 1)
    m = np.zeros(r + p - 1)

    assert p == 2, "only support p = 2"
    assert len(y) % p == 0, "len(y) should be divisible by p"

    num_tiles = int(len(y) / p)

    for t in range(num_tiles):
        i = t * p
        d = x[i:i+len(m)]
        g = b

        y[i:i+2] = np.dot(A.T, np.dot(G, g) * np.dot(B.T, d))

    return y

if __name__ == '__main__':
    n = 16
    r = 3
    x = np.random.random(n)
    b = np.random.random(r)

    assert np.isclose(FIR(x, b), winograd_FIR(x, b)).all(), \
           "FIR and winograd should match"
    assert np.isclose(FIR(x, b), winograd_FIR_mm(x, b)).all(), \
           "FIR and winograd (MM) should match"

    print("FIR PASSED")


import numpy as np

dados = np.array([[44410., 5712., 37123., 0., 25757.],
                 [2003, 1991, 1990, 2019, 1006]])

cont = np.arange(10)

print(dados.shape)      # retorna quantidade de linhas e colunas

print(dados.ndim)       # retorna a quantidade de dimnensões do array

print(dados.size)       # retorna o numeros de elementos do array

print(dados.dtype)      # retorna o tipo de dados do array

print(dados.T)          # retorna o array transposto, coverte linhas em colunas e vice versa

print(dados.transpose)  # mesma função do array.T

print(dados.tolist)     # converte o array para lista do python

print(cont.reshape((5, 2), order='C'))    # retorna array contendo uma nova forma, order='C', order='F

km = [44410, 5712, 37123, 0, 25757]
anos = [2003, 1991, 1990, 2019, 2006]

info_carros = km + anos # concatenação das listas
print(info_carros)
print(np.array(info_carros).reshape((5, 2), order='F')) # concatenação com reshape

dados_new = dados.copy()    # cria uma cópia do array
dados_new.resize((3, 5), refcheck=False)  # adiciona mais uma linha(ou coluna) no array | refcheck ignora referencia
print(dados_new)
dados_new[2] = dados_new[0] / (2019 - dados_new[1])


import numpy as np

def amean(*args):
    return sum(args)/len(args)
def gmean(*args):
    return np.array(args).prod()**(1.0/len(args))
def hmean(*args):
    return len(args)*(1.0/sum(np.array(args)**(-1)))
def median(*args):
    l = np.array(args)
    l=np.sort(l)
    leng = len(l)
    if leng%2 ==1:
        return l[int((leng+1)/2)-1]
    else:
        lf = np.floor(leng/2)
        lf = int(lf)
        return (l[lf-1]+l[lf])/2

def riesum(func,strt,stp):
    dx = 1e-6
    val = np.arange(strt,stp,dx)
    return sum(func(val))*dx
def derivative(func,at):
    d = 1e-7
    dm = -1*d
    t1 = (func(at+d)-func(at))/d
    t2 = (func(at+dm)-func(at))/dm
    return amean(t1,t2)


import numpy as np

def multiscale(x, scales):
    return np.hstack([x.reshape(-1, 1)/pow(2., i) for i in scales])

def encode_scalar_column(x, scales=[-1, 0, 1, 2, 3, 4, 5, 6]):
    return np.hstack([np.sin(multiscale(x, scales)), np.cos(multiscale(x, scales))])


import pandas as pd
import numpy as np
import scipy.optimize as opt

def loglike(beta, Y, X):
    """ Log likelihood function.

    Parameters
    ----------
    beta: matrix.
        N x 1 matrix of the coefficients.
        N is the number of observations.

    Y : matrix
        N x 1 matrix of the dependent variable.
        N is the number of observations.

    X : matrix
        N x K matrix of the independent variables.
        N is the number of observations.
        K is the number of independent variables.

    Returns
    -------
    logvalue : matrix
        Log likelihood function
    """

    logvalue = - (Y[None, :] @ X @ beta - np.sum(np.log(1 + np.exp(X @ beta))))

    return(logvalue[0])

def inverselogit(beta, Y, X):
    """ Inverse of logit function.

    Parameters
    ----------
    beta: matrix.
        N x 1 matrix of the coefficients.
        N is the number of observations.

    Y : matrix
        N x 1 matrix of the dependent variable.
        N is the number of observations.

    X : matrix
        N x K matrix of the independent variables.
        N is the number of observations.
        K is the number of independent variables.

    Returns
    -------
    inverse : matrix
        Inverse of logit function
    """

    inverse = 1 / (1 + np.exp(-X @ beta))

    return(inverse)

def dlogit(beta, Y, X):
    """ Derivative of logit function.

    Parameters
    ----------
    beta: matrix.
        N x 1 matrix of the coefficients.
        N is the number of observations.

    Y : matrix
        N x 1 matrix of the dependent variable.
        N is the number of observations.

    X : matrix
        N x K matrix of the independent variables.
        N is the number of observations.
        K is the number of independent variables.

    Returns
    -------
    dlogit : matrix
        Derivative of the logit function
    """

    dlogit = sum(inverselogit(beta, Y, X)*(1 - inverselogit(beta, Y, X)))

    return(dlogit)

def logithessian(beta, Y, X):
    """ Hessian matrix of logit function.

    Parameters
    ----------
    beta: matrix.
        N x 1 matrix of the coefficients.
        N is the number of observations.

    Y : matrix
        N x 1 matrix of the dependent variable.
        N is the number of observations.

    X : matrix
        N x K matrix of the independent variables.
        N is the number of observations.
        K is the number of independent variables.

    Returns
    -------
    hessian : matrix
        Hessian matrix
    """

    dp = np.diag(inverselogit(beta, Y, X)*(1 - inverselogit(beta, Y, X)))
    hessian = X.T @ dp @ X

    return(hessian)

def logit(Y, X):

    """ Logistic regression of Y on X.

    Parameters
    ----------
    Y : matrix
        N x 1 matrix of the dependent variable.
        N is the number of observations.

    X : matrix
        N x K matrix of the independent variables.
        N is the number of observations.
        K is the number of independent variables.

    Returns
    -------
    (coef, test_status)

    coef : matrix
        K x 1 matrix of coefficients

    test_status : string
        Status of the optimization: Success or False

    """

    # Initial beta values
    k = np.size(X, 1)
    beta0 = np.zeros(k)

    # Optimization
    beta = opt.minimize(
        loglike, beta0, args = (Y, X), method = "BFGS", hess = logithessian
    )

    # Coefficients
    coef = beta.x
    test_status = beta.success

    # Standard errors
    cov = -dlogit(coef, Y, X)
    se = np.sqrt(cov.diagonal().T)

    # Return
    return (coef, se, test_status)


import numpy as np

import pandas as pd

class Perceptron(object):

    def __init__(self, no_of_inputs, threshold=100, learning_rate=0.01):
        self.threshold = threshold
        self.learning_rate = learning_rate
        self.weights = np.zeros(no_of_inputs)
        self.bias = 0

    def predict(self, inputs):
        sum = np.dot(inputs, self.weights) + self.bias  # dot product matrx multiplication bias add
        if sum > 0:
            activation = 1
        else:
            activation = 0
        return activation

    def train(self, train_data, labels):
        for _ in range(self.threshold):
            for inputs, label in zip(train_data, labels):
                prediction = self.predict(inputs)
                self.bias += self.learning_rate * (label - prediction)
                self.weights += self.learning_rate * (label - prediction) * inputs

                #leru type of sigmoid function.
                #to predict either 1/0.

# read and  modify data
data = pd.read_csv(r'C:\Users\ALPARSLAN\Desktop\perceptron.csv')
data = data[:]
data[4] = np.where(data.iloc[:, -1] == 'Iris-setosa', 0, 1)
train_input = data.iloc[:, 0:4].values
labels = data.iloc[:, -1].values
# activate the perceptron
perceptron = Perceptron(4)
# train the perceptron
perceptron.train(train_input, labels)

# test the perceptron
test_data = pd.read_csv(r'C:\Users\ALPARSLAN\Desktop\perceptron.test.csv')

test_data = test_data[:]
test_data[4] = np.where(test_data.iloc[:, -1] == 'Iris-setosa', 0, 1)

test_data_input = test_data.iloc[:, 0:4].values

# count number of total missclassified
miss_classified = 0
miss_classified_list = []
test_data.iat[0, 5]
for i in range(0, test_data_input.shape[0]):
    test_prediction1 = perceptron.predict(test_data_input[i])
    miss_classified_list.append(test_prediction1)
    if test_prediction1 != test_data.iat[i, 5]:
        miss_classified = miss_classified + 1
accuracy = (1 - (miss_classified / test_data.shape[0])) * 100
print("Accuracy: ", accuracy)  # Accuracy is here giving 100%, with the test data as provided in Assignment.

# take a input from user (UI input of Data)
listFromUser = [0, 0, 0, 0]
for i in range(0, 4):
    userData = float(input('enter i th value :  '))
    listFromUser[i] = userData
print('the data you have Provided is\n=> ', listFromUser)

output_prediction = perceptron.predict(listFromUser)
if output_prediction == 0:
    print('According to the input you have given, : Iris-setosa')
elif output_prediction == 1:
    print('According to the input you have given : Iris-versicolor')


'''
01 - Hierarchical clustering of the grain data

In the video, you learned that the SciPy linkage() function performs hierarchical
clustering on an array of samples. Use the linkage() function to obtain a hierarchical
clustering of the grain samples, and use dendrogram() to visualize the result. A sample
of the grain measurements is provided in the array samples, while the variety of each
grain sample is given by the list varieties.

Instructions

- Import:
- linkage and dendrogram from scipy.cluster.hierarchy.
- matplotlib.pyplot as plt.
- Perform hierarchical clustering on samples using the linkage() function with the method='complete'
  keyword argument. Assign the result to mergings.
- Plot a dendrogram using the dendrogram() function on mergings. Specify the keyword arguments
  labels=varieties, leaf_rotation=90, and leaf_font_size=6.
'''
# Perform the necessary imports
from scipy.cluster.hierarchy import linkage, dendrogram
import matplotlib.pyplot as plt

# Calculate the linkage: mergings
mergings = linkage(samples, method='complete')

# Plot the dendrogram, using varieties as labels
dendrogram(mergings,
           labels=varieties,
           leaf_rotation=90,
           leaf_font_size=6,
)
plt.show()


# -----------------------------------------------------------------------------
# Copyright (C) 2018  Nicolas P. Rougier
# Distributed under the terms of the BSD License.
# -----------------------------------------------------------------------------
import numpy as np

# Create a vector with values ranging from 10 to 49
Z = np.arange(10,50)

# Create a null vector of size 100 but the fifth value which is 1
Z = np.zeros(100)
Z[4] = 1

# Reverse a vector (first element becomes last)
Z = np.arange(50)[::-1]

# Create a 3x3 matrix with values ranging from 0 to 8
Z = np.arange(9).reshape(3,3)

# Create a 3x3 identity matrix
Z = np.eye(3)

# Create a 2d array with 1 on the border and 0 inside
Z = np.ones((10,10))
Z[1:-1,1:-1] = 0

# Given a 1D array, negate all elements which are between 3 and 8, in place
Z = np.arange(11)
Z[(3 < Z) & (Z <= 8)] *= -1