# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.4'
#       jupytext_version: 1.2.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---

#

#

#

#
Computational Seismology

#
Finite Differences - Taylor Operators and Numerical Differentiation

#

#

#

#

#
#
#
#

#
#
# ---
#
# This notebook is part of the supplementary material
# to [Computational Seismology: A Practical Introduction](https://global.oup.com/academic/product/computational-seismology-9780198717416?cc=de&lang=en&#),
# Oxford University Press, 2016.
#
#
# ##### Authors:
# * Lion Krischer ([@krischer](https://github.com/krischer))
# * Heiner Igel ([@heinerigel](https://github.com/heinerigel))
# ---

# This exercise covers the following aspects:
#
# * Use the Taylor series to calculate finite-difference operators
# * Learn how to define high-order finite-difference operators
# * Demonstrate the accuracy improvement by testing the operators on (arbitrary) functions
#
# ---

# ## Basic Equations
#
# The Taylor expansion of $f(x + dx)$ around $x$ is defined as
#
# $$
# f(x+dx)=\sum_{n=0}^\infty \frac{f^{(n)}(x)}{n!}dx^{n}
# $$
#
#
# Finite-difference operators can be calculated by seeking weights (here: $a$, $b$, $c$) with which function values have to be multiplied to obtain an interpolation or a derivative. Example:
#
# $$
# \begin{align}
# a ~ f(x + dx) & \ = \ a ~ \left[ ~  f(x) +  f^{'} (x) dx +  \frac{1}{2!}  f^{''} (x) dx^2   + \dotsc  ~ \right] \\
# b ~ f(x) & \  = \ b ~ \left[ ~  f(x)  ~ \right] \\
# c ~ f(x - dx) & \ = \ c ~ \left[ ~  f(x) -  f^{'} (x) dx +  \frac{1}{2!}  f^{''} (x) dx^2   - \dotsc  ~ \right]
# \end{align}
# $$
#
# This can  be expressed in matrix form by comparing coefficients, here seeking a 2nd derivative
#
# $$
# \begin{align}
# &a  ~~+~~            ~~~~b           &+~~  c & = & 0 \\
# &a  ~~\phantom{+}~~ \phantom{b}  &-~~  c & = & 0 \\
# &a  ~~\phantom{+}~~ \phantom{b}  &+~~  c & = & \frac{2!}{\mathrm{d}x^2}
# \end{align}
# $$
#
# which leads to
#
# $$
# \begin{pmatrix}
#   1 & 1 & 1 \\
#   1 & 0 & -1 \\
#   1 & 0 & 1
#  \end{pmatrix}
# \begin{pmatrix}
#   a\\
#  b \\
#  c
#  \end{pmatrix}
#  =
#  \begin{pmatrix}
#   0\\
#  0 \\
#  \frac{2!}{dx^2}
#  \end{pmatrix}
# $$
#
# and using matrix inversion we obtain
#
# $$
# \begin{pmatrix}
# a \\
# b\\
# c
# \end{pmatrix}
# =
# \begin{pmatrix}
# \frac{1}{2 \mathrm{d}x^2} \\
#  - \frac{2}{2 \mathrm{d}x^2} \\
#   \frac{1}{2 \mathrm{d}x^2}
# \end{pmatrix}
# $$
#
# This is the the well known 3-point operator for the 2nd derivative. This can easily be generalized to higher point operators and higher order derivatives. Below you will find a routine that initializes the system matrix and solves for the Taylor operator.

# ---
#
# ## Calculating the Taylor operator
#
# The subroutine `central_difference_coefficients()` initalizes the system matrix and solves for the difference weights assuming $dx=1$. It calculates the centered differences using an arbitrary number of coefficients, also for higher derivatives. The weights are defined at $x\pm i dx$ and $i=0,..,(nop-1)/2$, where $nop$ is the length of the operator. Careful! Because it is centered $nop$ has to be an odd number (3,5,...)!
#
# It returns a central finite difference stencil (a vector of length $nop$)  for the `n`th derivative.

# +
import math
import numpy as np

def central_difference_coefficients(nop, n):
    """
    Calculate the central finite difference stencil for an arbitrary number
    of points and an arbitrary order derivative.

    :param nop: The number of points for the stencil. Must be
        an odd number.
    :param n: The derivative order. Must be a positive number.
    """
    m = np.zeros((nop, nop))
    for i in range(nop):
        for j in range(nop):
            dx = j - nop // 2
            m[i, j] = dx ** i

    s = np.zeros(nop)
    s[n] = math.factorial(n)

    # The following statement return oper = inv(m) s
    oper = np.linalg.solve(m, s)
    # Calculate operator
    return oper

# -

# ---
#
# ## Plot Taylor operators
#
# Investigate graphically the Taylor operators. Increase $nop$ for the first $n=1$ or higher order derivatives. Discuss the results and try to understand the interpolation operator (derivative order $n=0$).

# +
# %matplotlib inline

# Some necessary initialization
import matplotlib.pyplot as plt
plt.style.use('ggplot')

# Give length of operator (odd)
nop = 13
# Give order of derivative (0 - interpolation, 1 - first derivative, 2 - second derivative)
n = 1

# Get operator from routine 'central_difference_coefficients'
oper = central_difference_coefficients(nop, n)

# Initialize location of operator
x = np.linspace(-(nop - 1) / 2, (nop - 1) / 2, nop)

# Simple plot with operator
plt.figure(figsize=(10, 4))
plt.plot(x, oper)
plt.show()
# -

# ---
#
# ## Accuracy of centered finite differences
#
# Define a simple function, e.g. sine, cosine, gaussian, some polynomial, ..., differentiate it using the stencils returned from the above function and compare the result to the analytically calculated derivative. Keep the physical domain (e.g., 3$\pi$) constant but change 1) the number of points nx (i.e., change dx) or 2) change the wavenumber k. In both cases you change the number of grid points per wavelength. Calculate the error of the solution (in the center of the domain) and look at the error as a function of grid points per wavelength.

# +
# Let us initialize space between 0 and 10 pi with 10000 points
# Note: here you can be creative, change the function as you like,
# as long as you know the analytical derivative
nx = 100        # Number of points in x
x = np.linspace(0, 3 * np.pi, nx)

# We initialite a function f as a sin curve with wavenumber k
k = 3
f = np.sin(k * x)

# The analytical derivative is given by
dfa = k * np.cos(k * x)

# We need to know the grid spacing dx
dx = x[2] - x[1]

# Now let's get the finite difference operator with paramters
nop = 5 # Length of operator
n = 1   #  1 st derivative
oper = central_difference_coefficients(nop, n)

# Calculate the derivative of f(x) by applying the finite-difference operator
df = np.zeros(nx)
# Loop over the central points where we calculate the derivatives
for i in range(nop // 2, nx - nop // 2):  # Avoid the boundaries
    # Loop over operator
    for j in range(0, nop):
       df[i] = df[i] + oper[j] * f[i - (nop - 1) // 2 + j]

# Scale by dx.
df /= dx ** n

plt.plot(df, label="Numerical derivative", lw=2)
plt.plot(dfa, ls="--",label="Analytic derivative")
plt.legend()
plt.show()

notebooks/Computational Seismology/The Finite-Difference Method/fd_taylor_operators.py


import numpy
import time
import math

import multiprocessing as mp
import joblib

# own function definitions
import tasks

def clear_all():
    gl = globals().copy()
    for var in gl:
        if var[0] == '_': continue
        if 'func' in str(globals()[var]): continue
        if 'module' in str(globals()[var]): continue

        del globals()[var]

# Main guard - put main script code in here
# (avoids multiprocessing problems when executing functions from same module in parallel)
if __name__ == "__main__":
    clear_all()
    ###############################################################################
    # Problem:
    # Evaluate logistic map for selected values of r from [1, 2] interval after
    # assumed number of iterations
    #
    # y(n) = r * y(n-1) * (1 - y(n-1))
    #
    # Evaluations for different r are independent - embarrassingly parallel problem
    print("Problem: Logistic map")

    # Case 1
    # - small number of large tasks (all of same size)
    # - few memory accesses
    print("Case 1: small number of large tasks")
    N = 100                 # number of r values to be evaluated
    numIters = 1000000      # number of iterations

    A = numpy.empty(N)
    t = time.perf_counter()
    for i in range(N):
        r = 1 + (i + 1) / N
        A[i] = tasks.logisticMap(r, numIters)
    dt_serial = (time.perf_counter() - t)
    print("Serial:", dt_serial)

    # do the same in parallel with multiprocessing Pool
    print("Number of Cores: ",  mp.cpu_count())
    numWorkers = 4 # use 4 threads anyway

    t = time.perf_counter()
    with mp.Pool(processes = numWorkers) as pool: # pool only visible in the context below
       # starmap method
       # arguments: task to be invoked, itereble object with task parameters
       # return value: list
       B = pool.starmap(tasks.logisticMap, ((1 + (i + 1) / N, numIters) for i in range(N)) )
    dt = (time.perf_counter() - t)
    print("Pool:", dt, ", speedup:", dt_serial/dt, ", correct: ", (A == B).all())

    # alternative solution: joblib library
    t = time.perf_counter()
    C = joblib.Parallel(n_jobs = numWorkers)(joblib.delayed(tasks.logisticMap)((1 + (i + 1) / N), numIters) for i in range(N))
    dt = (time.perf_counter() - t)
    print("Joblib:", dt, ", speedup:", dt_serial/dt, ", correct: ", (A == C).all())

    ##############################################################################
    # Case 2
    # - different amount of job for each task

    print("Case 2: uneven tasks")
    N = 100                 # number of r values to be evaluated
    numIters = 1000000      # number of iterations
    numWorkers = 4

    A = numpy.empty(N)
    t = time.perf_counter()
    for i in range(N):
        r = 1 + (i + 1) / N
        n = (numIters // N) * i * 2
        A[i] = tasks.logisticMap(r, n)
    dt_serial = (time.perf_counter() - t)
    print("Serial:", dt_serial)

    # parallel - pool
    t = time.perf_counter()
    with mp.Pool(processes = numWorkers) as pool: # pool only visible in the context below
       B = pool.starmap(tasks.logisticMap, ((1 + (i + 1) / N, (numIters // N) * i * 2) for i in range(N)) )
    dt = (time.perf_counter() - t)
    print ("Pool:", dt, ", speedup:", dt_serial/dt, ", correct: ", (A == B).all())

    # parallel - joblib
    t = time.perf_counter()
    C = joblib.Parallel(n_jobs = numWorkers)(joblib.delayed(tasks.logisticMap)((1 + (i + 1) / N), (numIters // N) * i * 2) for i in range(N))
    dt =  (time.perf_counter() - t)
    print ("Joblib:", dt, ", speedup:", dt_serial/dt, "correct: ", (C == A).all())

    ##############################################################################
    # Case 3
    # - large number of small tasks (all of same size)
    # - more memory accesses
    print("Case 3: large number of small tasks")
    N = 10000000
    numIters = 10
    numWorkers = 4

    # serial
    A = numpy.empty(N)
    t = time.perf_counter()
    for i in range(N):
        r = 1 + (i + 1) / N
        A[i] = tasks.logisticMap(r, numIters)
    dt_serial = (time.perf_counter() - t)
    print("Serial:", dt_serial)

    # pool
    t = time.perf_counter()
    with mp.Pool(processes = numWorkers) as pool:
       B = pool.starmap(tasks.logisticMap, ((1 + (i + 1) / N, numIters) for i in range(N)) )
    dt = (time.perf_counter() - t)
    print("Pool:", dt, ", speedup:", dt_serial/dt, "correct: ", (A == B).all())

    # joblib
    t = time.perf_counter()
    C = joblib.Parallel(n_jobs = numWorkers)(joblib.delayed(tasks.logisticMap)((1 + (i + 1) / N), numIters) for i in range(N))
    dt =  (time.perf_counter() - t)
    print("Joblib:", dt, ", speedup:", dt_serial/dt, "correct: ", (A == C).all())

    # pool - output arrays
    t = time.perf_counter()
    blockSize = N // numWorkers # note the reminder!
    with mp.Pool(processes = numWorkers) as pool:
       res = pool.starmap(tasks.logisticMap_outArray, ((N, numIters, i * blockSize, blockSize ) for i in range(numWorkers)) )
    D = numpy.concatenate(res)
    dt = (time.perf_counter() - t)
    print("Pool, array-wise:", dt, ", speedup:", dt_serial/dt, "correct: ", (A == D).all())

    # joblib - output arrays
    t = time.perf_counter()
    blockSize = N // numWorkers
    res = joblib.Parallel(n_jobs = numWorkers)(joblib.delayed(tasks.logisticMap_outArray)(N, numIters, i * blockSize, blockSize ) for i in range(numWorkers))
    E = numpy.concatenate(res)
    dt = (time.perf_counter() - t)
    print("Joblib, array-wise:", dt, ", speedup:", dt_serial/dt, "correct: ", (A == E).all())

    ##############################################################################
    # Problem: apply complex function on a large array
    # - in place transformation (modify original array)
    # - workers share memory but do not interact
    print("\nProblem: complex function")
    clear_all()

    n = 10 ** 8      # 100 million elements
    origA = numpy.random.uniform(0, 1, n)
    fun = lambda x: numpy.arctan(numpy.power(numpy.sin(x) + 1.0, 0.75))  # complex function to make task arithmetic-bound

    # Serial
    A = origA.copy()
    t = time.perf_counter()
    A = fun(A)
    dt_serial = time.perf_counter() - t
    print("Serial:", dt_serial)

    # Parallel
    bufA = mp.Array("d", n)                         # allocate shared buffers (it contains optional lock)
    sharedA = numpy.frombuffer(bufA.get_obj())      # create 1D numpy matrices as views
    sharedA[:] = origA                              # initialize vector (alters underlying buffer)

    test = numpy.frombuffer(bufA.get_obj())         # get another view to see if it works

    # create list of workers
    t = time.perf_counter()
    numWorkers = 4
    sliceSize = n // numWorkers
    workers = [mp.Process(target=tasks.transformComplex, args = (bufA, sid * sliceSize, sliceSize) ) for sid in range(numWorkers)]

    for w in workers:
        w.start()
    for w in workers:
        w.join()

    dt = (time.perf_counter() - t)
    print("Parallel:", dt, ", speedup: ", dt_serial/dt, "correct: ", (A == sharedA).all())

    ##############################################################################
    # Problem: calculating histogram
    # - workers share memory and do interact
    print("\nProblem: histogram")
    sampleSize = 100
    n = 10 ** 7
    A = numpy.random.binomial(sampleSize, 0.5, n)

    histo = numpy.zeros(sampleSize, dtype="i")

    t = time.perf_counter()
    for i in range(n):
        ai = A[i]
        histo[ai] = histo[ai] + 1
    dt_serial = time.perf_counter() - t
    print ("Serial:", dt_serial)

    # Parallel
    bufA = mp.Array("i", n)                       # allocate shared buffers with optional lock
    sharedA = numpy.frombuffer(bufA.get_obj(), dtype="i")    # create 1D numpy matrices as views
    sharedA[:] = A

    bufHisto = mp.Array("i", sampleSize)                       # allocate shared buffers with optional lock
    sharedHisto = numpy.frombuffer(bufHisto.get_obj(), dtype="i")    # create 1D numpy matrices as views
    sharedHisto[:] = 0                                # initialize vector (alters underlying buffer)

    numWorkers = 4
    sliceSize = n // numWorkers

    # variant 1
    t = time.perf_counter()
    workers = [mp.Process(target=tasks.histo, args = (bufA, bufHisto, sid, sliceSize) ) for sid in range(numWorkers)]
    for w in workers: w.start()
    for w in workers: w.join()
    dt = (time.perf_counter() - t)
    print ("Parallel:", dt, ", speedup: ", dt_serial/dt, "correct: ", (histo == sharedHisto).all())

    # variant 2
    sharedHisto[:] = 0
    t = time.perf_counter()
    workers = [mp.Process(target=tasks.histoLock, args = (bufA, bufHisto, sid, sliceSize) ) for sid in range(numWorkers)]
    for w in workers: w.start()
    for w in workers: w.join()
    dt = (time.perf_counter() - t)
    print ("Parallel:", dt, ", speedup: ", dt_serial/dt, "correct: ", (histo == sharedHisto).all())

    # variant 3
    sharedHisto[:] = 0
    t = time.perf_counter()
    workers = [mp.Process(target=tasks.histoLockFast, args = (bufA, bufHisto, sid, sliceSize) ) for sid in range(numWorkers)]
    for w in workers: w.start()
    for w in workers: w.join()
    dt = (time.perf_counter() - t)
    print ("Parallel:", dt, ", speedup: ", dt_serial/dt, "correct: ", (histo == sharedHisto).all())


########################################################################
# This program illustrate many approaches to solve ODEs. In this case
#    we use the simple pendulum as the system.
#
# This module implements all the classes and functions for the problem.
#
#    Marina von Steinkirch, spring/2013
#    (based on Mike Zingale's codes)
#
########################################################################

import math
import numpy as np

class pendulumtHistory:
    """ container to store the integrated history of the pendulum.  """

    def __init__(self):
        self.t = None
        self.theta = None
        self.omega = None
        self.E = None
        self.dE = None

    def energyError(self, EN,E0):
        """ absolute energy error given by the starting and ending point """
        return abs( (EN - E0)/ E0)

class pendulumTrajectory:
    """ hold the initial conditions the pendulum and integrate """

    def __init__(self, CONST_theta0, CONST_l, CONST_g, CONST_m):
        self.theta0_degrees = CONST_theta0                 # initial condition -> theta0 in degree
        self.theta0 = self.theta0_degrees*math.pi/180      # convert theta0 to radians
        self.t0 = 0.0                                       # time starts at zero
        self.omega0 = 0.0                                   # initial condition -> omega0 starts at 0
        self.l = CONST_l                                    # length of pendulum
        self.g = CONST_g                                    # acceleration of gravity
        self.m = CONST_m                                    # mass

    def pendulumPeriod(self):
        """ return the estimative of the period of the pendulum, which
                only depend on the theta0 (initial displacement). """
        return 2.0*math.pi*math.sqrt(self.l/self.g)*(1.0 + self.theta0**2*(1.0/16.0))

    def pendulumEnergy(self):
        """ return the estimative of the energy of the pendulum,
                in function of the initial values. The total energy
                should be conserved so this value should be a
                constant."""
        return abs(0.5*self.m*(self.l)**2*(self.omega0)**2 - self.m*self.g*self.l*(math.cos(self.theta0)))

    def newEnergy(self, the, ome):
        """ return the energy of the pendulum by substituting the time steps,
                we use this to show that Euler is not good for the pendulum
                system since the total energy seems to monotonically increases
                in time. This wont happen on the other methods."""
        return abs(0.5*self.m*(self.l)**2*(ome)**2 - self.m*self.g*self.l*(math.cos(the)))

    def intEuler(self, dt, tmax):
        """ integrate the equations of motion using Euler's method. """

        # initial conditions
        t = self.t0
        theta = self.theta0
        omega = self.omega0
        e = self.pendulumEnergy()

        # store the history for plotting
        tpoints = [t]
        thetapoints = [theta]
        omegapoints = [omega]
        energypoints = [e]

        # loop over the time steps
        while (t < tmax):

            # make sure that the next step doesn't take us past where
            # we want to be, because of roundoff
            if t+dt > tmax:
                dt = tmax-t

            # get the RHS
            thetadot, omegadot = self.rhs(theta, omega)

            # advance
            thetanew = theta + dt*thetadot
            omeganew = omega + dt*omegadot
            t += dt

            # calculates the step energy
            energynew = self.newEnergy(theta,omega)

            # set for the next step
            theta = thetanew
            omega = omeganew

            # store
            tpoints.append(t)
            thetapoints.append(thetanew*180/(math.pi))
            omegapoints.append(omeganew)
            energypoints.append(energynew)

        # return a orbitHistory object with the angular displacement
        H = pendulumtHistory()
        H.t = np.array(tpoints)
        H.theta = np.array(thetapoints)
        H.omega = np.array(omegapoints)
        H.energy = np.array(energypoints)

        return H

    def intEulerCromer(self, dt, tmax):
        """ integrate the equations of motion using Euler-Cromer's method."""

        # initial conditions
        t = self.t0
        theta = self.theta0
        omega = self.omega0
        e = self.pendulumEnergy()

        # store the history for plotting
        tpoints = [t]
        thetapoints = [theta]
        omegapoints = [omega]
        energypoints = [e]

        # loop over the time steps
        while (t < tmax):

            # make sure that the next step doesn't take us past where
            # we want to be, because of roundoff
            if t+dt > tmax:
                dt = tmax-t

            # get the RHS
            thetadot, omegadot = self.rhs(theta, omega)

            # advance
            omeganew = omega + dt*omegadot

            # these line is the only change from Euler
            thetanew = theta + dt*omeganew

            t += dt

            # calculates the step energy
            energynew = self.newEnergy(theta,omega)

            # set for the next step
            theta = thetanew
            omega = omeganew

            # store
            tpoints.append(t)
            thetapoints.append(thetanew*180/(math.pi))
            omegapoints.append(omeganew)
            energypoints.append(energynew)

        # return a orbitHistory object with the angular displacement
        H = pendulumtHistory()
        H.t = np.array(tpoints)
        H.theta = np.array(thetapoints)
        H.omega = np.array(omegapoints)
        H.energy = np.array(energypoints)

        return H

    def intVerlet(self, Dt, tmax):
        """ integrate the equations of motion using Euler's method. """

        # initial conditions
        t = self.t0
        theta = self.theta0
        omega = self.omega0
        energy_ver = self.pendulumEnergy()
        dt = Dt

        while (t < tmax):

            # make sure that the next step doesn't take us past where
            # we want to be, because of roundoff
            if t+dt > tmax:
                dt = tmax-t
            # Take one backward step to start Verlet

            #alpha = - (self.g/self.l)*math.sin(theta)
            #thetaold = theta - omega*dt + 0.5*dt**2*alpha
            # get the RHS
            thetadot, alpha = self.rhs(theta, omega)

            # advance
            thetanew = theta + dt*omega +0.5*dt**2*alpha
            thetadot, alphaplus1 = self.rhs(thetanew, omega)
            omeganew = omega + 0.5*dt*(alpha+alphaplus1)

            # calculate energy  for this time step
            energynew = self.newEnergy(thetanew,omeganew)
            t += dt
            # set for the next step
            theta = thetanew;
            omega = omeganew

        # return a orbitHistory object with the angular displacement
        H = pendulumtHistory()
        H.t = Dt
        H.E = energynew
        H.dE = H.energyError(energynew,energy_ver)

        return H

    def rhs(self, the, ome):
        """ RHS of the equations of motion """
        thetadot = ome
        omegadot = -(self.g/self.l)*math.sin(the)

        return thetadot, omegadot


"""Transformations for points on the sphere.
We can transform from cartesian to spherical
or from spherical to cartesian.
Points are assumed to live on the unit sphere.
The notation is consistent with the one found here:
http://mathworld.wolfram.com/SphericalCoordinates.html
"""
# pylint: disable=C0103
# pylint: disable=E1111
import numpy
from numpy import arctan2, arccos, cos, sin, zeros, reshape, sqrt
from numpy.linalg import norm

def cast2matrix(x, dim):
    """Handles different cases of the input x that might happen to
    be given when xyz2thetaphi or thetaphi2xyz is called.
    These functions require the input to be an (n,dim) matrix (with dim=2 or 3
    depending on which direction).
    Sometimes the user calls it with a vector of (dim,) which will fail.
    Some other weird formats have occured before and we will handle them
    and convert everything (when possible) into a (n,dim) matrix.
    """
    if type(x) == list:  # cast list to numpy array and then call cas2matrix
        return cast2matrix(numpy.array(x), dim)

    if type(x) == numpy.ndarray:
        if len(x.shape) == 1:  # vector to 1x3 matrix
            return reshape(x, (1, dim))
        if len(x.shape) == 2:
            if x.shape[1] == dim:  # is n x 3
                return x
            if x.shape[0] == dim:  # has to be transposed since it is 3 x n
                return x.T
        if len(x.shape) > 2:
            raise ValueError("Numpy ndarrays as input have to be vectors or matrices.")

def xyz2thetaphi(xyz):
    """Transformation from points on the unit sphere given by their
    cartesian representation as (x,y,z) to their
    spherical representation as (theta,phi),
    with the azimuthal angle theta in [0,2pi) and
    the polar angle phi in [0,pi].

    Args:
        xyz: An (n,3) numpy.ndarray of cartesian points living on the unit
        sphere.
    Raises:
        ValueError: If any point does not live on the unit sphere.
        ValueError: If not exactly three points per row are given.
    Returns:
        thetaphi: An (n,2) numpy.ndarray of spherical points living on the unit
        sphere.
    """
    xyz = cast2matrix(xyz, 3)

    n, dim = xyz.shape
    thetaphi = zeros((n, 2))
    for i in range(n):
        x, y, z = xyz[i, :]
        r = x ** 2 + y ** 2 + z ** 2
        if not abs(r - 1.0) < 1.0e-6:
            raise ValueError(
                "Point %i does not live on the unit sphere. "
                "The coordinates are (%d,%d,%d) wit norm %d." % (i, x, y, z, sqrt(r))
            )
        thetaphi[i, 0] = arctan2(xyz[i, 1], xyz[i, 0])
        thetaphi[i, 1] = arccos(xyz[i, 2])
    return thetaphi

def thetaphi2xyz(thetaphi):
    """Transformation from points on the unit sphere given by their
    spherical representation as (theta,phi) to their
    cartesian representation as (x,y,z),
    with the azimuthal angle theta in [0,2pi) and
    the polar angle phi in [0,pi].

    Args:
        thetaphi: An (n,2) numpy.ndarray of spherical points living on the unit
        sphere.
    Raises:
        ValueError: If not exactly two points per row are given.
    Returns:
        xyz: An (n,3) numpy.ndarray of cartesian points living on the unit
        sphere.
    """
    thetaphi = cast2matrix(thetaphi, 2)
    n, dim = thetaphi.shape

    xyz = zeros((n, 3))
    for i in range(n):
        theta, phi = thetaphi[i, :]
        xyz[i, 0] = cos(theta) * sin(phi)
        xyz[i, 1] = sin(theta) * sin(phi)
        xyz[i, 2] = cos(phi)
    return xyz


import numpy as np
import matplotlib.pyplot as plt

def generate_quadratic_data():
    quadratic_a = 2.4
    quadratic_b = -2.
    quadratic_c = 1.

    num_data = 30

    noise = 2 * np.random.randn(num_data)

    sampled_x = np.linspace(-10, 10., num_data)
    sampled_y = quadratic_a * sampled_x * sampled_x + quadratic_b * sampled_x + quadratic_c + noise

    return sampled_x, sampled_y

class LeastSquares:
    def __init__(self, x, y):
        self.x = x
        self.y = y

        self.data_mat = np.vstack(
            [self.x * self.x, self.x,
             np.ones_like(self.x)]).T

        self.theta = np.array([0, 0, 0.])

    def residual(self):
        pred_y = self.data_mat @ self.theta
        r = pred_y - self.y
        return r

    def cost(self):
        r = self.residual()
        return r.T @ r

    def compute_jacobian(self):
        return self.data_mat

    def least_squares_solve(self):
        for i in range(10):
            print('iteration: {} cost: {}'.format(i, self.cost()))
            J = self.compute_jacobian()
            r = self.residual()

            delta = np.linalg.solve(J.T @ J, -J.T @ r)
            self.theta += delta

            if np.linalg.norm(delta) < 1e-8:
                print('converged iteration: {} cost: {}'.format(
                    i, self.cost()))
                break

        return self.theta

def main():
    x_data, y_data = generate_quadratic_data()

    solver = LeastSquares(x_data, y_data)

    theta = solver.least_squares_solve()

    x = np.linspace(-12, 12., 100)
    a, b, c = theta
    print('fitted coefficient (a,b,c):', theta.transpose())
    pred_y = a * x * x + b * x + c

    p1 = plt.plot(x_data, y_data, '*r')
    p2 = plt.plot(x, pred_y, 'g')
    plt.legend((p1[0], p2[0]), ('sampled data', 'fitted function'))
    plt.title('Data points vs Fitted curve')
    plt.show()

if __name__ == "__main__":
    main()


import numpy as np

class LinearRegression:
    """Linear Regression model. Models the relation between some data and a
    response assuming linear relationship and using ordinary least squares to
    estimate the parameters.

    Parameters
    ----------
    dims : `int`, optional
        Number of features of the training data, without taking into
        account the intercept. Defaults to None, leaving the model
        unconstrained with respect to the number of features until the fit
        method is called for the first time)

    """
    def __init__(self, dims=None):
        self.dims = dims
        self.w = None
        self.r2 = None
        self.std_err = None

    def fit(self, X, y):
        """Fits X data using ordinary least squares method.

        Parameters
        ----------
        X : `numpy.ndarray` (n_samples, n_features)
            Training examples from which to
            estimate the parameters of the model.

        y : `numpy.ndarray` (n_samples, n_features)
            The response corresponding to each training example in ``x``.

        Returns
        -------
        weights : `numpy.array` (n_features+1,)
            The estimated of the model parameters given ``x`` and ``y``

        """

        # If dims setted check for compatibility, if not initialize dims
        if self.dims is not None:
            assert X.shape[1] == self.dims, "Number of dimensions mismatch."
        else:
            self.dims = X.shape[1]

        # Add a first dimension of ones corresponding to the intercept
        X = np.hstack((np.ones([X.shape[0], 1]), X))
        pseudo_inv = np.matmul(np.linalg.pinv(np.matmul(X.T, X)), X.T)
        self.w = np.matmul(pseudo_inv, y)

        # Compute fitness metrics
        self._compute_r2(X, y)
        self._estimate_standard_error(X, y)

        return self.w

    def _compute_r2(self, x, y):
        # Computes r2 metric.
        y_pred = np.matmul(x, self.w)
        y_mean = np.mean(y)

        y_stdev = (y-y_mean)**2
        y_pstdev = (y_pred-y_mean)**2
        self.r2 = np.sum(y_pstdev)/np.sum(y_stdev)

    def _estimate_standard_error(self, x, y):
        # Estimates the standar error of the data.
        y_pred = np.matmul(x, self.w)
        self.std_err = np.sqrt(np.sum((y_pred-y)**2)/(x.shape[0]-2))

    def predict(self, X):
        """Predicts the response using the current estimands.

        Parameters
        ----------
        X : `numpy.ndarray` (n_examples, n_features))
            New data to predict.

        Returns
        -------
        predicted : `numpy.ndarray` (n_examples,)
            Predicted values using the last computed weights.

        """

        assert self.w is not None, "Unfitted predictor! Call the fit method."

        X = np.atleast_2d(X)
        assert X.shape[1] == self.dims, "Number of dimensions mismatch."

        # Add a first dimension of ones corresponding to the intercept
        X = np.hstack((np.ones([X.shape[0], 1]), X))

        return np.matmul(X, self.w)


import cv2
import numpy as np
from matplotlib import pyplot as plt
img=cv2.imread("obama.jpg",0)
img=cv2.resize(img,(512,512))
cv2.imshow("Original Image",img)

#flatten the image to get the flattened array
f1=img.flatten()

#calculate the histogram
hist, bins = np.histogram(img, 256, [0,255])

#calculate the cdf cumulative sum
cdf=hist.cumsum()

#places where cdf=0 are ignored and stored as cdf_m
cdf_m=np.ma.masked_equal(cdf,0)

#normalize the cdf to plot with the histogram
cdf_normalized = cdf * float(hist.max()) / cdf.max()
plt.plot(cdf_normalized, color = 'b')
plt.hist(f1,256,[0,256], color = 'r')
plt.xlim([0,256])
plt.legend(('cdf','histogram'), loc = 'upper left')
plt.show()

#iplementing histogram eqaualization transformation with following the formulae
num_cdf_m=(cdf_m - cdf_m.min())*255
den_cdf_m=(cdf_m.max()-cdf_m.min())
cdf_m=num_cdf_m/den_cdf_m

#the masked places of cdf_m are now 0
cdf=np.ma.filled(cdf_m,0).astype('uint8')

#cdf values are assigned in the flattend array
img2=cdf[f1]

#converting 1D image to 2D image to make it original shape
img3=np.reshape(img2, img.shape)
cv2.imshow("Equalized Image",img3)

#calculate the histogram of new image
hist2, bins = np.histogram(img2, 256, [0,255])

#calculate the cdf of equalized histogram
cdf=hist2.cumsum()

#normalize the cdf to plot the histogram
cdf_normalized2 = cdf * float(hist2.max()) / cdf.max()
plt.plot(cdf_normalized2, color = 'b')
plt.hist(img2,256,[0,256], color = 'r')
plt.xlim([0,256])
plt.legend(('cdf','histogram'), loc = 'upper left')
plt.show()

#============================================
#using opencv
#=============================================
equ = cv2.equalizeHist(img)
res = np.hstack((img,equ))
cv2.imshow("Equalization", res)

def equalize_hist_color(img):
    """Equalize the image splitting the image applying cv2.equalizeHist() to each channel and merging the results"""
    channels = cv2.split(img)
    eq_channels = []
    for ch in channels:
        eq_channels.append(cv2.equalizeHist(ch))
        eq_image = cv2.merge(eq_channels)
    return eq_image
img_color=cv2.imread("obama.jpg")
chans = cv2.split(img_color)
colors = ("b", "g", "r")
plt.figure()
plt.title("'Flattened' Color Histogram")
plt.xlabel("Bins")
plt.ylabel("# of Pixels")
features = []
for (chan, color) in zip(chans, colors):
	# create a histogram for the current channel and
	# concatenate the resulting histograms for each
	# channel
	hist = cv2.calcHist([chan], [0], None, [256], [0, 256])
	features.extend(hist)
	# plot the histogram
	plt.plot(hist, color = color)
	plt.xlim([0, 256])
plt.show()
img_color=cv2.resize(img_color,(512,512))
# img_color=cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)
color_equalized_image=equalize_hist_color(img_color)
chans = cv2.split(color_equalized_image)
colors = ("b", "g", "r")
plt.figure()
plt.title("'Flattened' Color Histogram")
plt.xlabel("Bins")
plt.ylabel("# of Pixels")
features = []
for (chan, color) in zip(chans, colors):
	# create a histogram for the current channel and
	# concatenate the resulting histograms for each
	# channel
	hist = cv2.calcHist([chan], [0], None, [256], [0, 256])
	features.extend(hist)
	# plot the histogram
	plt.plot(hist, color = color)
	plt.xlim([0, 256])
plt.show()
result=np.hstack((img_color,color_equalized_image))
cv2.imshow("Equalized colour image",result)
cv2.waitKey(0)
cv2.destroyAllWindows()

#==============================================================================
# We have commented that equalizing the three channels is not a good approach
# because the color shade changes dramatically. This is due to the additive
# properties of the BGR color space. As we are changing both the brightness and
# the contrast in the three channels independently, this can lead to new color
# shades appearing in the image when merging the equalized channels. This issue
# can be seen in the previous screenshot.
# A better approach is to convert the BGR image to a color space containing a
# luminance/intensity channel (Yuv, Lab, HSV, and HSL). Then, we apply
# histogram equalization only on the luminance channel and, finally, perform
# inverse transformation, that is, we merge the channels and convert them back to
# the BGR color space.

def equalize_hist_color_hsv(img_color):
    """Equalize the image splitting the image after HSV conversion and applying cv2.equalizeHist()
    to the V channel, merging the channels and convert back to the BGR color space
    """
    H, S, V = cv2.split(cv2.cvtColor(img_color, cv2.COLOR_BGR2HSV))
    eq_V = cv2.equalizeHist(V)
    eq_image = cv2.cvtColor(cv2.merge([H, S, eq_V]), cv2.COLOR_HSV2BGR)
    return eq_image
hsv_equalized_image=equalize_hist_color_hsv(img_color)
result_hsv=np.hstack((result, hsv_equalized_image))
cv2.imshow("HSV_Histogram_Equalization", result_hsv)
cv2.waitKey(0)

# As can be seen, obtained the results obtained after equalizing only the V channel
# of the HSV image are much better than equalizing all the channels of the BGR
# image. As we commented, this approach is also valid for a color space
# containing a luminance/intensity channel (Yuv, Lab, HSV, and HSL). This will
# be seen in the next section.


import sys

import numpy as np

def gauss(n):
    return 0.5 * n * (n + 1)

def binary_search(pos, score_func):
    xs = np.arange(0, pos.max())
    n = len(xs)
    val = np.infty

    lo, hi = 0, n - 1

    while lo <= hi:
        mid = lo + (hi - lo) // 2

        sl = None if mid == 0 else score_func(pos, xs[mid - 1])
        sr = None if mid == n - 1 else score_func(pos, xs[mid + 1])
        sv = score_func(pos, xs[mid])

        val = min(sv, val)

        if sl < sv:
            hi = mid - 1
        elif sr < sv:
            lo = mid + 1
        else:
            break

    return val

def solve_binary_search(pos):
    """
    Use binary search to find the min. value of the score function.
    """
    print("Using binary search")

    score_func = lambda xs, x: np.abs(xs - x).sum()
    print("Part 1:", binary_search(pos, score_func))

    score_func = lambda xs, x: gauss(np.abs(xs - x)).sum()
    print("Part 1:", binary_search(pos, score_func))

def solve_direct(pos):
    """
    Solve using direct calculation -> median & mean.
    See https://math.stackexchange.com/a/1024462
    """
    print("Using direct calculation")

    x = np.median(pos)
    score = np.abs(pos - x).sum()
    print("Part 1:", int(score))

    # lo <= x <= hi, x int
    lo = np.mean(pos) - 1 / 2
    hi = np.mean(pos) + 1 / 2

    score = min(
        gauss(np.abs(pos - x)).sum() for x in np.arange(np.floor(lo), np.ceil(hi) + 1)
    )
    print("Part 2:", int(score))

def main():
    pos = np.loadtxt(sys.argv[1], delimiter=",", dtype=int)

    solve_binary_search(pos)
    solve_direct(pos)

if __name__ == "__main__":
    main()


import numpy as np
import warnings

def polynomialMatrix(x_vector, grade) :
	a_matrix = []

	for x in x_vector:
		row = np.power(np.full((grade + 1), x), np.flipud(np.arange(grade + 1)))
		a_matrix.append(row)

	a_matrix = np.array(a_matrix)

	return a_matrix

def leastSquare(a_matrix, y_vector) :
	at_matrix = a_matrix.T

	at_a_inverse = np.linalg.inv((at_matrix.dot(a_matrix)))
	result = ((at_a_inverse).dot(at_matrix)).dot(y_vector)

	return result

def polynomialFuncEval(x_range, coef):
	grade = len(coef)
	y_results = np.empty(shape = (0,0))

	for x in x_range :
		x_values = np.power(np.full((grade), x), np.flipud(np.arange(grade)))
		result = x_values.dot(coef)
		y_results = np.append(y_results, result)

	return y_results

def polynomialAproximation(x_range, x_vector, y_vector, grade) :
	a_matrix = polynomialMatrix(x_vector, grade)
	coef = leastSquare(a_matrix, y_vector)
	y_graph = polynomialFuncEval(x_range, coef)
	return y_graph


import numpy as np
import matplotlib.pyplot as plt
from scipy import optimize

m=10
X = np.arange(m)
np.random.seed(42) #use to get the same results every run
rnd = np.random.rand(m)
Y = 2 + np.arange(m)*0.5 + rnd

def cost_function(Teta):
    H = Teta[0]+X * Teta[1]
    E = Y - H
    cost = np.dot(E,E)/(2*m)
    return cost

res = optimize.minimize(cost_function, x0=[0,0])
teta0=res.x[0]
teta1=res.x[1]
print(teta0,teta1)

H = teta0+teta1 * X
plt.plot(X, Y,'o',X,H)
plt.title('fit linear line with optimize.minimize')
plt.show()


import numpy as np

W = np.array([
    [0.6, 0.1, 0.1, 0.1, 0.1],
    [0.1, 0.6, 0.1, 0.1, 0.1],
    [0.1, 0.1, 0.6, 0.1, 0.1],
    [0.1, 0.1, 0.1, 0.6, 0.1],
    [0.1, 0.1, 0.1, 0.1, 0.6]])
print(f"W:\n{W}")

u = np.array([[0.6], [0.5], [0.6], [0.2], [0.1]])
print(f"u:\n{u}")

M = np.array([
    [-0.125, 0, 0.125, 0.125, 0],
    [0, -0.125, 0, 0.125, 0.125],
    [0.125, 0, -0.125, 0, 0.125],
    [0.125, 0.125, 0, -0.125, 0],
    [0, 0.125, 0.125, 0, -0.125]
])
print(f"M:\n{M}")

h = W * u
print(f"h:{h}")

ev, e = np.linalg.eig(M)
print(f"ev:{ev}")
print(f"e:{e}")

v_ss= ((np.dot(h.T, e) / (1 - ev))* e)
print(np.sum(v_ss, axis=0))
# [0.57468284 0.46104003 0.5556308  0.35789389 0.33646672]


from .Activation import activation
import numpy as np

class layer():
    def __init__(self, in_nodes=0, out_nodes=0, activation_type=""):
        self.w = np.random.randn(out_nodes, in_nodes)*0.01
        self.dw = np.zeros((out_nodes, in_nodes))
        self.b = np.zeros((out_nodes, 1))
        self.db = np.zeros((out_nodes, 1))
        self.act_func = activation(activation_type)

    def forward(self, X):
        # X dimensions are (number of input nodes) x (number of examples)
        Z = self.w @ X + self.b  # Z = w * X + b
        weights_sum=np.sum(np.square(self.w))
        output = self.act_func.forward(Z)  # A = activation(Z)
        self.X = X  # cache input to use it in back prop
        return output, weights_sum

    def backward(self, dA, Lambda):
        # dA dimensions are (number of output nodes) x (number of examples)
        m = dA.shape[1]
        dZ = self.act_func.backward(dA) # dZ = dA . g'(Z) (element wise product)
        self.dw = (1/m) * dZ @ self.X.T + (Lambda/m) * self.w   # dw = dZ * X.T+(lambda/m) * w
        self.db = (1/m) * np.sum(dZ, axis=1, keepdims=True) # db = 1/m * sum(dZ)
        grad_input = self.w.T @ dZ
        return grad_input

    def output_dims(self):
        return self.b.shape[0]

    def getParams(self):
        return self.w, self.b

    def getGrads(self):
        return self.dw, self.db

    def setParams(self, w, b):
        self.w = w
        self.b = b

    def getLayerParams(self):
        LayerParams = self.w, self.b, self.act_func.activation_type
        return "layer", LayerParams

    def setLayerParams(self, LayerParams):
        self.w, self.b , activation_type = LayerParams
        self.act_func = activation(activation_type)

'''
in_nodes = 2
out_nodes = 3
activation_type = "Relu"  # (Relu, Sigmoid, Linear)
L = layer(in_nodes, out_nodes, activation_type)
w,b = L.getParams()
w = w + 5
b = np.ones((out_nodes, 1))
L.setParams(w, b)
print('Weights=\n',w,"\nb=\n",b)

# testing forward propagation

X = np.array([
    [0, 1, 2],
    [1, 4, 3]
])
A = L.forward(X)
# print("\n\nfor input=\n",X,"\noutput=\n",A)

# testing back propagation

dw, db = L.getGrads()
print('dw=\n', dw, "\ndb=\n", db)
dA = np.array([
    [1, 9, 7],
    [6, 1, 2],
    [4, 5, 8]
])
print("\n\nfor dA=\n\n", dA)
dw, db = L.getGrads()
dX = L.backward(dA)
print('dw=\n', dw, "\ndb=\n", db, "\ndX=\n", dX)
'''


# A contagious and fatal virus has tragically struck the city of Plaguesville, which has been quarantined until the virus has run its course. The function
# P(t) gives the number of people who are newly infected t days after the outbreak began.
#
# Find first and second derivatives P'(t) and P''(t)
# Solve P(t) = 0, P(t) and P''(t) = 0, then complete the table.
# Use the table to describe the details of the Plagesville tragedy.

from sympy import solve, symbols, lambdify, diff
import matplotlib.pyplot as plt
import numpy as np

g_xlim = [ 1, 100 ]
g_ylim = [ -5, 15 ]

def plot_fun( fun, name, col ):
	x_vals = np.linspace( g_xlim[0], g_xlim[1], 1000, endpoint=True )
	y_vals = fun( x_vals )
	plt.plot( x_vals, y_vals, label = name, color = col )

t = symbols( 't' )
P = 6.4*t**3 - 0.064* t**4

dP = diff( P, t )
ddP = diff( dP, t )

lam_P = lambdify( t, P )
plot_fun( lam_P, 'P', 'B' )

lam_dP = lambdify( t, dP )
plot_fun( lam_dP, 'P\'', 'G' )

lam_ddP = lambdify( t, ddP )
plot_fun( lam_ddP, 'P\'\'', 'R' )

plt.show()

solve( P, t )
solve( dP, t )
solve( ddP, t )

P.subs( { t: 0 } )
dP.subs( { t: 75 } )
ddP.subs( { t: 0 } )


from typing import Iterable, Union

import numpy as np

class DataNormalizer:

    """
    Transform a numeric array by scaling it to a given range. To scale the array
    between an arbitrary set of values [a, b], the transformation used is

    x' = a + (x-min(x)) * (b - a) / (max(x) - min(x))

    To return to the original scale, the inverse transformation is given by

    x = min(x) + (x' - a) * (max(x) - min(x)) / (b - a)

    Parameters
    ----------
    feature_range : Iterable[Union[int, float]], optional
        The desired range of transformed data. By default, (0, 1).

    Attributes
    ----------
    y_min : Union[int, float]
        The minimum value of the numeric array.
    y_range : Union[int, float]
        The range of the numeric array, i.e., the difference between the maximum
        and the minimum value.

    """

    def __init__(self, feature_range: Iterable[Union[int, float]] = (0, 1)):
        self.feature_range = feature_range

    def fit(self, y: np.ndarray) -> object:

        """
        Compute the minimum and the range of the numeric array which is to be
        scaled.

        Parameters
        ----------
        y : np.ndarray
            The numeric array.

        Returns
        -------
        self : object
            Fitted scaler.

        Raises
        ------
        ValueError
            If the first value of the input range is greater or equal to the
            second value.
        ValueError
            If the input range does not contain two values.
        """

        if self.feature_range[0] >= self.feature_range[1]:
            raise ValueError(
                "Minimum of desired feature range must be smaller than maximum."
            )
        if len(self.feature_range) != 2:
            raise ValueError("Range must consist of two elements.")

        self.y_min = np.min(y)
        self.y_range = np.ptp(y)
        return self

    def transform(
        self, y: Union[int, float, np.ndarray], derivative: bool = False
    ) -> np.ndarray:

        """
        Transform the numeric array according to the desired range.

        Parameters
        ----------
        y : Union[int, float, np.ndarray]
            The numeric array.
        derivative : bool
            If True, the input array to be transformed is a derivative with
            respect the array used to fit, by default False.

        Returns
        -------
        np.ndarray
            The scaled numeric array.
        """

        if derivative:
            out = y * (self.feature_range[1] - self.feature_range[0]) / self.y_range
        else:
            out = (
                self.feature_range[0]
                + (y - self.y_min)
                * (self.feature_range[1] - self.feature_range[0])
                / self.y_range
            )
        return out

    def inverse_transform(
        self, y: Union[int, float, np.ndarray], derivative: bool = False
    ) -> np.ndarray:

        """
        Transform the scaled numeric array into the original scale.

        Parameters
        ----------
        y : Union[int, float, np.ndarray]
            The scaled numeric array.
        derivative : bool
            If True, the input array to be transformed is a derivative with
            respect the array used to fit, by default False.

        Returns
        -------
        np.ndarray
            The numeric array.
        """

        if derivative:
            out = self.y_range * y / (self.feature_range[1] - self.feature_range[0])
        else:
            out = self.y_min + self.y_range * (y - self.feature_range[0]) / (
                self.feature_range[1] - self.feature_range[0]
            )
        return out


"""
    Second test of FFTLog algorithm

    FFTLog can be used to do an integral similar to:

        $$ F_\ell(k) = \int \frac{dr}{r} f(r) j_{\ell}(kr)$$

    if we use $f(r) = r$ for ell=0, then we expect:

        $$ \int dr j_{0}(kr) = \frac{\pi}{2k}$$

"""

import numpy as np
import matplotlib.pyplot as plt
from fftlog import fftlog

r = np.logspace(-4., 2., num=2000)
fr = r

nu = 1.01
ell = 0

myfftlog = fftlog(r, fr, nu=nu, N_extrap_low=1500,
                  N_extrap_high=1500, c_window_width=0.25, N_pad=5000)

k, Fk = myfftlog.fftlog(ell)
Fk_true = np.pi / (2*k)

assert np.all(abs(Fk-Fk_true) < 1.0e-8)

fig, ax = plt.subplots(ncols=2, figsize=(12, 4))

ax[0].plot(r, fr)
ax[1].plot(k, Fk, label='FFTLog')
ax[1].plot(k, Fk_true, 'r--', label=r'Truth (i.e., $\pi/(2k)$)')

ax[0].set(xlabel='r', ylabel='f(r)')
ax[1].set(xscale='log', xlabel='k', ylabel='F(k)')
ax[1].legend()
fig.savefig('fftlog_test.png', bbox_inches='tight', dpi=300)


#fourier transform in numpy
import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt

img = cv.imread('Resources/messi5.jpg',0)
# img = np.ones((3,3))
f = np.fft.fft2(img)
fshift = np.fft.fftshift(f)
fshift_magnitude = np.abs(fshift)
magnitude_spectrum = 20*np.log(np.abs(fshift))
plt.subplot(121),plt.imshow(img, cmap = 'gray')
plt.title('Input Image'), plt.xticks([]), plt.yticks([])
plt.subplot(122),plt.imshow(magnitude_spectrum, cmap = 'gray')
plt.title('Magnitude Spectrum'), plt.xticks([]), plt.yticks([])
plt.show()

#low frequency filter
# rows, cols = img.shape
# crow,ccol = rows//2 , cols//2
# fshift[crow-30:crow+31, ccol-30:ccol+31] = 0
# f_ishift = np.fft.ifftshift(fshift)
# img_back = np.fft.ifft2(f_ishift)
# img_back = np.real(img_back)
# plt.subplot(222),plt.imshow(img, cmap = 'gray')
# plt.title('Input Image'), plt.xticks([]), plt.yticks([])
# plt.subplot(223),plt.imshow(img_back, cmap = 'gray')
# plt.title('Image after HPF'), plt.xticks([]), plt.yticks([])
# plt.subplot(224),plt.imshow(img_back)
# plt.title('Result in JET'), plt.xticks([]), plt.yticks([])
# plt.show()

#fourier transform in opencv
# It returns the same result as previous, but with two channels. First channel will have the real part of the result and second channel will have the imaginary part of the result. The input image should be converted to np.float32 first.
# dft = cv.dft(np.float32(img),flags = cv.DFT_COMPLEX_OUTPUT)
# dft_shift = np.fft.fftshift(dft)
# magnitude_spectrum = 20*np.log(cv.magnitude(dft_shift[:,:,0],dft_shift[:,:,1]))
# plt.subplot(121),plt.imshow(img, cmap = 'gray')
# plt.title('Input Image'), plt.xticks([]), plt.yticks([])
# plt.subplot(122),plt.imshow(magnitude_spectrum, cmap = 'gray')
# plt.title('Magnitude Spectrum'), plt.xticks([]), plt.yticks([])
# plt.show()

#low pass filter
# rows, cols = img.shape
# crow,ccol = rows//2 , cols//2
# # create a mask first, center square is 1, remaining all zeros
# mask = np.zeros((rows,cols,2),np.uint8)
# mask[crow-30:crow+30, ccol-30:ccol+30] = 1
# # apply mask and inverse DFT
# fshift = dft_shift*mask
# f_ishift = np.fft.ifftshift(fshift)
# img_back = cv.idft(f_ishift)
# img_back = cv.magnitude(img_back[:,:,0],img_back[:,:,1])
# plt.subplot(121),plt.imshow(img, cmap = 'gray')
# plt.title('Input Image'), plt.xticks([]), plt.yticks([])
# plt.subplot(122),plt.imshow(img_back, cmap = 'gray')
# plt.title('Magnitude Spectrum'), plt.xticks([]), plt.yticks([])
# plt.show()

#performance optimization of dft
rows,cols = img.shape
print("{} {}".format(rows,cols))

nrows = cv.getOptimalDFTSize(rows)
ncols = cv.getOptimalDFTSize(cols)
print("{} {}".format(nrows,ncols))

nimg = np.zeros((nrows,ncols))
nimg[:rows,:cols] = img

# laplacian high pass filter
# simple averaging filter without scaling parameter
# mean_filter = np.ones((100,100))
# # creating a gaussian filter
# x = cv.getGaussianKernel(20,10)
# gaussian = x*x.T
# # different edge detecting filters
# # scharr in x-direction
# scharr = np.array([[-3, 0, 3],
#                    [-10,0,10],
#                    [-3, 0, 3]])
# # sobel in x direction
# sobel_x= np.array([[-1, 0, 1],
#                    [-2, 0, 2],
#                    [-1, 0, 1]])
# # sobel in y direction
# sobel_y= np.array([[-1,-2,-1],
#                    [0, 0, 0],
#                    [1, 2, 1]])
# # laplacian
# laplacian=np.array([[0, 1, 0],
#                     [1,-4, 1],
#                     [0, 1, 0]])
# filters = [mean_filter, gaussian, laplacian, sobel_x, sobel_y, scharr]
# filter_name = ['mean_filter', 'gaussian','laplacian', 'sobel_x', \
#                 'sobel_y', 'scharr_x']
# fft_filters = [np.fft.fft2(x) for x in filters]
# fft_shift = [np.fft.fftshift(y) for y in fft_filters]
# mag_spectrum = [np.log(np.abs(z)+1) for z in fft_shift]
# for i in range(6):
#     plt.subplot(2,3,i+1),plt.imshow(mag_spectrum[i],cmap = 'gray')
#     plt.title(filter_name[i]), plt.xticks([]), plt.yticks([])
# plt.show()


#!/usr/bin/env python
# -*- coding: utf-8 -*-

"""

Maths

    f_lin, f_lin0, f_lin1, gaussian, gaussian2D,
    rms, nanrms, std, nanstd, nanavg, bsplinterpol

"""

import math
import numpy as np
import scipy.interpolate as interpolate

def f_lin(x, A, B):
    '''
    Y = A * x + B
    '''
    return A * x + B

def f_lin0(x, A):
    '''
    Y = A * x
    '''
    return A * x

def f_lin1(x, B):
    '''
    Y = x + B
    '''
    return x + B

def gaussian(x, mu, sigma):
    '''
    Normalized Gaussian function given variable x
    '''
    return 1/(sigma * np.sqrt(2 * np.pi)) * np.exp( - (x - mu)**2 / (2 * sigma**2))

def gaussian2D(x1, x2, mu1, mu2, sig1, sig2, A=1.):
    '''
    2D Gaussian function given iid variables x & y (and amplitude A)
    '''
    return A * np.exp(- (x1 - mu1)**2 / (2 * sig1**2) - (x2 - mu2)**2 / (2 * sig2**2))

def rms(a, ddof=0):
    '''
    Calculate root mean square

    ------ INPUT ------
    a                   an array or a list
    ddof                Delta Degrees of Freedom
    ------ OUTPUT ------
    rms                 root mean square of a
    '''
    n = np.size(a) - ddof
    a = np.array(a)
    ms = np.sum(a*a) / n

    return np.sqrt(ms)

def nanrms(a, ddof=0):
    '''
    Calculate root mean square (NaNs treated as zeros)

    ------ INPUT ------
    a                   an array or a list
    ddof                Delta Degrees of Freedom
    ------ OUTPUT ------
    rms                 root mean square of a
    '''
    n = np.size(a) - ddof
    a = np.array(a)
    ms = np.nansum(a*a) / n

    return np.sqrt(ms)

def std(a, ddof=0):
    '''
    The same as np.std
    '''
    n = np.size(a) - ddof
    a = np.array(a)
    mu = np.mean(a)
    ms = np.sum((a - mu)**2) / n

    return np.sqrt(ms)

def nanstd(a, axis=None, weights=None, MaskedValue=np.nan):
    '''
    Weighted standard deviation with NaNs ignored (NaN convention diff from nanrms *)
    MaskedValue presents if all NaNs
    '''
    ma = np.ma.MaskedArray(a, mask=np.isnan(a))
    if weights is not None:
        wgt = np.ma.MaskedArray(weights, mask=np.isnan(a))
    else:
        wgt = weights

    mask_any = ma.mask.any(axis=axis)
    mask_all = ma.mask.all(axis=axis) # All NaNs

    avg = np.average(ma, axis=axis, weights=wgt) # NaNs ignored
    ## Extend avg shape to that of a
    if axis is None:
        avg_ext = np.tile(avg, a.shape)
    elif axis==0:
        avg_ext = np.repeat(avg[np.newaxis,:,:], a.shape[0], axis=0)
        # avg_ext = np.tile(avg, (a.shape[0],1,1)) # alternative
    else:
        avg_ext = np.repeat(avg, ma.shape[axis], axis=axis-1).reshape(ma.shape)

    ## Deviation array of masked a from avg along axis
    dev_ma = ma - avg_ext

    if weights is not None:
        ## Count nonzero weights
        wgt_nz = np.ma.masked_where(wgt==0, wgt)
        Nwgt = wgt_nz.count(axis=axis)
        # print('Number of nonzero weights along axis {}: \n'.format(axis), Nwgt)
        std = np.sqrt(Nwgt/(Nwgt-1) * np.average(dev_ma**2, axis=axis, weights=wgt))
    else:
        std = np.sqrt(np.average(dev_ma**2, axis=axis, weights=wgt))

    ## Convert output none value convention (Default: NaNs)
    if axis is not None:
        std = std.data
        std[mask_all] = MaskedValue

    return std

def nanavg(a, axis=None, weights=None, MaskedValue=np.nan):
    '''
    Numpy.average with NaNs ignored (weight normalisation included)
    or
    Numpy.nanmean with weights
    '''
    ## NaNs -> --
    ma = np.ma.MaskedArray(a, mask=np.isnan(a))
    if weights is not None:
        wgt = np.ma.MaskedArray(weights, mask=np.isnan(a))
    else:
        wgt = weights

    mask_any = ma.mask.any(axis=axis)
    mask_all = ma.mask.all(axis=axis)

    avg = np.average(ma, axis=axis, weights=wgt)

    ## Check weight sums
    # print('wgt sums: ', np.average(ma, axis=axis, weights=wgt, returned=True)[1])

    ## Convert output none value convention (Default: NaNs)
    if axis is not None:
        avg = avg.data
        avg[mask_all] = MaskedValue

    return avg

def bsplinterpol(x, y, x0):
    '''
    Monte-Carlo propagated error calculator

    ------ INPUT ------
    x                   in base x
    y                   in data y
    x0                  out base x
    ------ OUTPUT ------
    bspl(x0)            B-spline interpol out data
    '''
    mask = []
    for i, yi in enumerate(y):
        if np.isnan(yi)==1 or yi==0:
            mask.append(i)
    if len(x)-len(mask)>4: # number of knots (avoid all NaNs col)
        x = np.delete(x, mask)
        y = np.delete(y, mask)

    t, c, k = interpolate.splrep(x, y, s=0, k=4) # s - smooth
    # print('''\
    # t: {}
    # c: {}
    # k: {}
    # '''.format(t, c, k))
    bspl = interpolate.BSpline(t, c, k, extrapolate=False)

    return bspl(x0)

"""
------------------------------ MAIN (test) ------------------------------
"""
if __name__ == "__main__":

    pass


from scipy.fftpack import dct
import numpy as np
import tensorflow as tf
from tensorflow.keras.layers import Permute

"""
A function other_transform() is implemented in utils.py which takes the input_feature, performs other statistic measures
instead of the global average pool and returns the transformed input.
Measures are:
Standard Deviation- transform='std',
Singular Value Decomposition- transform='svd',
Trace- transform='trace',
Discrete Cosine Transform- transform='dct_dcval'

"""

def other_transform(x_tensor, channel_axis, transform='std'):

    if(channel_axis == -1):
        x_tensor = Permute((3, 1, 2))(x_tensor) # Always Make channels first

    if transform=='std':
            # STANDARD DEVIATION
        std = tf.keras.backend.std(x_tensor, axis=[2,3]) # (batch, channels, widht, height)
        transformed_feature = std/tf.reduce_max(std, axis=None)

    elif transform=='dct_dcval':
        dct = tf.signal.dct(x_tensor, norm='ortho')
        transformed_feature = tf.keras.layers.GlobalMaxPool2D(data_format='channels_first')(dct) # dc term
        # DC term or the 1st term is always the largest in value, thus dct[0][0] === GlobalMaxpool2d

    elif transform=='svd':
        # SVD DECOMPOSITION
        s = tf.linalg.svd(x_tensor, compute_uv=False)
        transformed_feature = tf.reduce_max(s, axis=-1)
        transformed_feature = transformed_feature/tf.reduce_max(transformed_feature, axis=None) # Max value scaling.

    elif transform=='trace':
        # Sum of Diagonal
        trace = abs(tf.linalg.trace(x_tensor))
        transformed_feature = trace/tf.reduce_max(trace, axis=None) # Max value scaling.

    else:
        raise Exception("'{}' statistic measure is not implemented!".format(transform))

    return transformed_feature


# Muller's Recurrence
# Code from the website:
# https://scipython.com/blog/mullers-recurrence

# Want to tweak and experiment, but this is the start.

import numpy as np
from fractions import Fraction

def E_float(y, z):
    """Naive implementation of Muller's recurrence."""
    return 108 - (815 - 1500 / z) / y

def E_float2(y, _):
    """Numerically stable implementation of Muller's recurrence."""
    return 8 - 15/y

def E_fraction(y, z):
    """Muller's recurrence implemented with rational arithmetic."""
    return 108 - Fraction((815 - Fraction(1500, z)), y)

def do_recurrence(E, x):
    for n in range(2, N+1):
        x[n] = E(x[n-1], x[n-2])

N = 80
x_fraction = [0]*(N+1)
x_fraction[0], x_fraction[1] = 4, Fraction(17, 4)
do_recurrence(E_fraction, x_fraction)

x_float = [0]*(N+1)
x_float[0], x_float[1] = 4, 4.25
do_recurrence(E_float, x_float)

x_float2 = [0]*(N+1)
x_float2[0], x_float2[1] = 4, 4.25
do_recurrence(E_float2, x_float2)

print(' n {:>22s} {:^22s} {:^22s}'.format('rational arithmetic',
                    'floating point 1', 'floating point 2'))
print('--', '-'*22, '-'*22, '-'*22)
for n in range(N+1):
    print('{:2d} {:22.15f} {:22.15f} {:22.15f}'.format(n, float(x_fraction[n]),
                                             x_float[n], x_float2[n]))


from sklearn import datasets
from sklearn.decomposition import PCA
from sklearn.cluster import KMeans
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
from scipy.cluster.hierarchy import linkage
from scipy.cluster.hierarchy import fcluster
from scipy.cluster.hierarchy import dendrogram

import os
fig_folder = os.getcwd()

#Pull in iris dataset.
iris_dataset_url = 'https://raw.githubusercontent.com/pydata/pandas/master/pandas/tests/data/iris.csv'
iris = pd.read_csv(iris_dataset_url)

print(iris.head())
print(iris.corr().round(2))

#Feature names
ColNs = ['SepalLength', 'SepalWidth', 'PetalLength', 'PetalWidth']

X = iris[ColNs]
X_sd = iris[ColNs]

#Standardize X_sd
for n in ColNs:
    val = (X[n] - np.mean(X[n])) / np.std(X[n])
    X_sd.loc[:, n] = val

#Determine PCA directions
pca = PCA(n_components=4).fit(X_sd)
Dirs = pca.components_

#Create summary output.
CompNs = ["Comp " + str(i) for i in range(1, 5)]
IdxNs = ['Standard Deviation', 'Proportion of Variance', 'Cumulative Proportion']
Summary = pd.DataFrame(index=IdxNs, columns=CompNs)
ProjectedData = np.dot(X_sd, np.transpose(Dirs))
Sds = np.std(ProjectedData, 0)
Vars = Sds ** 2
Summary.loc['Standard Deviation', :] = Sds
Summary.loc['Proportion of Variance', :] = Vars / sum(Vars)
Summary.loc['Cumulative Proportion', :] = np.cumsum(Summary.loc['Proportion of Variance', :])

print(Summary)

#Graph of cumulative variance explained
fig1 = plt.figure(1)
plt.plot(Vars)
plt.xticks(range(4), CompNs)
plt.draw()

#2D Biplot
Ns = np.unique(iris['Name'])
Cs = ['blue', 'red', 'green']
print('names')
print(Ns)
print('colors')
print(Cs)

fig2, ax = plt.subplots(num=2)

for i in range(4):
    y1 = Dirs[0, i]*3
    y2 = Dirs[1, i]*3
    ax.arrow(0, 0, y1, y2, head_width=0.05)
    ax.annotate(ColNs[i], (y1, y2))

for i in range(3):
    sel = list(iris.loc[iris['Name'] == Ns[i]].index)
    m = ProjectedData[sel, :]
    ax.scatter(m[:, 0], m[:, 1], c=Cs[i], s=100)

ax.set_xlabel('Comp 1')
ax.set_ylabel('Comp 2')
plt.draw()
plt.savefig(os.path.join(fig_folder, 'pcaIris2dBiplotPython.pdf'))

#3D Biplot
fig3 = plt.figure(3)
ax = fig3.add_subplot(111, projection='3d')

for i in range(3):
    sel = [x for x in range(150) if iris.loc[x, 'Name'] == Ns[i]]
    m = ProjectedData[sel, :]
    ax.scatter(m[:, 0], m[:, 1], m[:, 2], c=Cs[i], s=100)
    ax.set_xlabel('Comp 1')
    ax.set_ylabel('Comp 2')
    ax.set_zlabel('Comp 3')
for i in range(4):
    x = Dirs[0, i]*3
    y = Dirs[1, i]*3
    z = Dirs[2, i]*3
    ax.plot([0, x], [0, y], [0, z], c='black')
    ax.text(x, y, z, ColNs[i], color='black', size=20)
plt.draw()
plt.savefig(os.path.join(fig_folder, 'pcaIris3dBiplotPython.pdf'))

#KMeans clustering (applied to unstandardized data)
KM3 = KMeans(n_clusters=3)
KM3.fit(X)
labels = KM3.labels_
iris['KMeans'] = labels

#3D plot of PCA-projected points classified according to KMeans
fig4 = plt.figure(4)
ax = fig4.add_subplot(111, projection='3d')
for i in range(3):
    sel = [x for x in range(150) if labels[x] == i]
    m = ProjectedData[sel, :]
    ax.scatter(m[:, 0], m[:, 1], m[:, 2], c=Cs[i], s=100)
    ax.set_xlabel('Comp 1')
    ax.set_ylabel('Comp 2')
    ax.set_zlabel('Comp 3')
ax.set_title('Clusters according to K-Means')
plt.draw()

#Hierarchical clustering (applied to unstandardized data)
linkage_matrix = linkage(X, "ward")

#3D plot of PCA-projected points classified according to hierarchical clustering
fig5 = plt.figure(5)
ddata = dendrogram(linkage_matrix)
plt.draw()

iris['Ward'] = fcluster(linkage_matrix, 9, 'distance')

#Function to replicate R's table function
def table(x, y):
    xuni = np.unique(x)
    yuni = np.unique(y)
    res = pd.DataFrame(index=xuni, columns=yuni)
    for x1 in xuni:
        for y1 in yuni:
            res.loc[x1, y1] = sum((x == x1) & (y == y1))
    return res

#Print tables to inspect how well classification has worked.
print('Kmeans purity matrix')
print(table(iris['KMeans'], iris['Name']))

print('Ward purity matrix')
print(table(iris['Ward'], iris['Name']))

plt.show(block=True)


"""
Author : Aditya Jain
Contact: https://adityajain.me
"""

import numpy as np

def adjusted_rand_score(labels_true, labels_pred):
	"""
	Rand index adjusted for chance.

	The Rand Index computes a similarity measure between two clusterings
	by considering all pairs of samples and counting pairs that are
	assigned in the same or different clusters in the predicted and
	true clusterings.

	ARI = (RI - Expected_RI) / (max(RI) - Expected_RI)

	Note : https://davetang.org/muse/2017/09/21/adjusted-rand-index/

	Parameters
	----------

	labels_true : numpy array,  true cluster labels

	label_pred : numpy array, predicted cluster label

	Returns
	-------
	adjusted rand index

	"""
	def factorail(n):
		if n<=1: return 1
		return n*factorail(n-1)
	def comb(n,r):
		if n<r: return 0
		return factorail(n)/(factorail(r)*factorail(n-r))

	n_2 = comb( len(labels_true), 2 )
	contigency_matrix = np.zeros( (len(np.unique(labels_true)),len(np.unique(labels_pred))) )
	for i in range(len(contigency_matrix)):
		for j in range(len(contigency_matrix[0])):
			contigency_matrix[i][j] = ((labels_true==i) & (labels_pred==j)).sum()
	rows_sum = np.sum(contigency_matrix, axis=1)
	cols_sum = np.sum(contigency_matrix, axis=0)

	ri = sum([ comb(n,2) for n in contigency_matrix.ravel()])
	rows_comb = sum([ comb(n,2) for n in rows_sum])
	cols_comb = sum([ comb(n,2) for n in cols_sum])

	numerator = (ri - (rows_comb*cols_comb)/n_2)
	denominator = (0.5*(rows_comb+cols_comb)) - ((rows_comb*cols_comb)/n_2)
	return numerator/denominator


"""
This is an example of using scipy.maxentropy to solve Jaynes' dice problem

See Golan, Judge, and Miller Section 2.3
"""

from scipy import maxentropy
import numpy as np

samplespace = [1., 2., 3., 4., 5., 6.]
def sump(x):
    return x in samplespace

def meanp(x):
    return np.mean(x)
# Set the constraints
# 1) We have a proper probability
# 2) The mean is equal to...
F = [sump, meanp]
model = maxentropy.model(F, samplespace)

# set the desired feature expectations
K = np.ones((5,2))
K[:,1] = [2.,3.,3.5,4.,5.]

model.verbose = False

for i in range(K.shape[0]):
    model.fit(K[i])

    # Output the distribution
    print("\nFitted model parameters are:\n" + str(model.params))
    print("\nFitted distribution is:")
    p = model.probdist()
    for j in range(len(model.samplespace)):
        x = model.samplespace[j]
        print("y = %-15s\tx = %-15s" %(str(K[i,1])+":",str(x) + ":") + \
                " p(x) = "+str(p[j]))

    # Now show how well the constraints are satisfied:
    print()
    print("Desired constraints:")
    print("\tsum_{i}p_{i}= 1")
    print("\tE[X] = %-15s" % str(K[i,1]))
    print()
    print("Actual expectations under the fitted model:")
    print("\tsum_{i}p_{i} =", np.sum(p))
    print("\tE[X]  = " + str(np.sum(p*np.arange(1,7))))


import numpy as np
import tensorflow as tf
def realImagVectorsToMagAngVectors(realVec, imagVec, deg=True):
    """
    Receives two real valued vectors x and y, which
    represents the complex vector x+1j*y.

    :param realVec: numpy array / list of real values
    :param imagVec: numpy array / list of imaginary values
    :param deg: Boolean, set to truth if degrees are
                the units of the angle/ If set to False,
                then radians will be used for the output
                angle vector.
    :return: two real valued vectors that represents the Magnitude(x+1j*y) and Angle(x+1j*y)
             same type as the input arrays.

    """
    assert(type(realVec) == type(imagVec))
    if type(realVec)==list:
        retType = list
        imagVec = np.array(imagVec)
        realVec = np.array(realVec)
    else:
        retType = np.array
    assert(realVec.shape == imagVec.shape)

    complex_vec = realVec + 1j * imagVec
    magnitude_vec = np.absolute(complex_vec)
    angle_vec = np.angle(complex_vec, deg=deg)
    return retType(magnitude_vec), retType(angle_vec)

def complex_matmul_using_real_ops(Areal, Aimag, Breal, Bimag, adjoint_a=False, adjoint_b=False):
    """
     P=A*B matrix product OP. where instead of complex matrices we use
     real valued representation as follows:

        A=Areal + 1j * Aimag
        B=Breal + 1j * Bimag
        P=Preal + 1j * Pimag

    :param Areal: tensorflow tensor of real valued numbers
    :param Aimag:
    :param Breal:
    :param Bimag:
    :return: tuple(Preal, Pimag)
    """

    if adjoint_a:
        Areal = tf.transpose(Areal)
        Aimag = tf.transpose(tf.negative(Aimag))

    if adjoint_b:
        Breal = tf.transpose(Breal)
        Bimag = tf.transpose(tf.negative(Bimag))

    Preal = tf.subtract(tf.matmul(Areal,Breal), tf.matmul(Aimag,Bimag))
    Pimag = tf.add(tf.matmul(Areal, Bimag), tf.matmul(Aimag, Breal))

    return Preal, Pimag

def realImagFormatToComplex(realImagFormattedVec):
    """
    A reformatting function which converts a real valued array into complex vector
    Let the values in the input array be [r1, i1, r2, i2,..., rN, iN]
    then the output array will be [r1+j*i1,r 2+j*i2,..., rN+j*iN]
    :param realImagFormattedVec:
    :return: returns a 1D numpy array of a length N containing complex numbers.
    """
    twoN = len(realImagFormattedVec)
    real_vals = np.take(realImagFormattedVec, range(0,twoN,2))
    imag_vals = np.take(realImagFormattedVec, range(1, twoN, 2))
    return real_vals + 1j * imag_vals

def complexFormatToRealImag(complexVec):
    """
    A reformatting function which converts a complex vector into real valued array.
    Let the values in the input array be [r1+j*i1,r 2+j*i2,..., rN+j*iN]
    then the output array will be [r1, i1, r2, i2,..., rN, iN]
    :param complexVec: complex numpy ndarray
    :return: returns a 1D numpy array of a length N containing complex numbers.
    """
    N = len(complexVec)
    ret = np.empty((2*N,), dtype=np.real(complexVec).dtype)
    ret[0::2] = complexVec.real
    ret[1::2] = complexVec.imag
    return ret


import numpy as np

def RotationX(theta):
    return np.array([[1, 0, 0],
                     [0, np.cos(theta), np.sin(theta)],
                     [0, -np.sin(theta), np.cos(theta)]])

def RotationY(theta):
    return np.array([[np.cos(theta), 0, -np.sin(theta)],
                     [0, 1, 0],
                     [np.sin(theta), 0, np.cos(theta)]])

def RotationZ(theta):
    return np.array([[np.cos(theta), np.sin(theta), 0],
                     [-np.sin(theta), np.cos(theta), 0],
                     [0, 0, 1]])

def DCMfromEulerXYZ(attitude_xyz):
    Rx = RotationX(attitude_xyz[0])
    Ry = RotationY(attitude_xyz[1])
    Rz = RotationZ(attitude_xyz[2])
    R = np.matmul(Rx, np.matmul(Ry, Rz))
    return R

def QuaternionFromEulerXYZ(attitude_xyz):
    c1 = np.cos(0.5 * attitude_xyz[0])
    s1 = np.sin(0.5 * attitude_xyz[0])
    c2 = np.cos(0.5 * attitude_xyz[1])
    s2 = np.sin(0.5 * attitude_xyz[1])
    c3 = np.cos(0.5 * attitude_xyz[2])
    s3 = np.sin(0.5 * attitude_xyz[2])

    q0 = (c1 * c2 * c3) + (s1 * s2 * s3)
    q1 = (s1 * c2 * c3) - (c1 * s2 * s3)
    q2 = (c1 * s2 * c3) + (s1 * c2 * s3)
    q3 = (c1 * c2 * s3) - (s1 * s2 * c3)

    return np.array([q0, q1, q2, q3])

def EulerXYZfromQuaternion(quat):
    q0 = quat[0]
    q1 = quat[1]
    q2 = quat[2]
    q3 = quat[3]

    r11 = (q0 * q0) + (q1 * q1) - (q2 * q2) - (q3 * q3)
    r12 = 2.0 * ((q1 * q2) + (q0 * q3))
    r13 = 2.0 * ((q1 * q3) - (q0 * q2))
    r23 = 2.0 * ((q2 * q3) + (q0 * q1))
    r33 = (q0 * q0) - (q1 * q1) - (q2 * q2) + (q3 * q3)

    phi = np.arctan2(r23, r33)
    theta = -np.arcsin(r13)
    psi = np.arctan2(r12, r11)

    return np.array([phi, theta, psi])

def QuaternionNorm(quat):
    q0 = quat[0]
    q1 = quat[1]
    q2 = quat[2]
    q3 = quat[3]
    return np.sqrt((q0 * q0) + (q1 * q1) + (q2 * q2) + (q3 * q3))

def QuaternionNormalise(quat):
    norm = QuaternionNorm(quat)
    return quat / norm

def Rad_to_Deg(rad):
    return np.array(rad) * (180.0 / np.pi)

def Deg_to_Rad(deg):
    return np.array(deg) * (np.pi / 180.0)

def QuatToDCM(quat):
    q0 = quat[0]
    q1 = quat[1]
    q2 = quat[2]
    q3 = quat[3]

    q0_2 = q0 * q0
    q1_2 = q1 * q1
    q2_2 = q2 * q2
    q3_2 = q3 * q3

    q1q2 = q1 * q2
    q0q3 = q0 * q3
    q1q3 = q1 * q3
    q0q2 = q0 * q2
    q2q3 = q2 * q3
    q0q1 = q0 * q1

    m11 = q0_2 + q1_2 - q2_2 - q3_2
    m12 = 2.0 * (q1q2 + q0q3)
    m13 = 2.0 * (q1q3 - q0q2)
    m21 = 2.0 * (q1q2 - q0q3)
    m22 = q0_2 - q1_2 + q2_2 - q3_2
    m23 = 2.0 * (q2q3 + q0q1)
    m31 = 2.0 * (q1q3 + q0q2)
    m32 = 2.0 * (q2q3 - q0q1)
    m33 = q0_2 - q1_2 - q2_2 + q3_2

    return np.array([[m11, m12, m13],
                     [m21, m22, m23],
                     [m31, m32, m33]])

def QuaternionRates(quat, omega_body):
    q0 = quat[0]
    q1 = quat[1]
    q2 = quat[2]
    q3 = quat[3]
    w = np.array([[-q1, -q2, -q3],
                  [q0, q3, -q2],
                  [-q3, q0, q1],
                  [q2, -q1, q0]])
    return 0.5 * np.matmul(w, omega_body)

def EulerAngleRatesXYZ(attitude, omega_body):
    phi = attitude[0]
    theta = attitude[1]
    E = np.array([[1, np.tan(theta) * np.sin(phi), np.tan(theta) * np.cos(phi)],
                  [0, np.cos(phi), -np.sin(phi)],
                  [0, np.sin(phi) / np.cos(theta), np.cos(phi) / np.cos(theta)]])
    return np.matmul(E, omega_body)

def EulerIntegration(X, Xdot, dt):
    return X + (Xdot * dt)

def EulerFromQuaternion(quat):
    w = quat[0]
    x = quat[1]
    y = quat[2]
    z = quat[3]

    t0 = 2.0 * (w * x + y * z)
    t1 = 1.0 - 2.0 * (x * x + y * y)
    roll_x = np.arctan2(t0, t1)

    t2 = 2.0 * (w * y - z * x)
    t2 = 1.0 if t2 > 1.0 else t2
    t2 = -1.0 if t2 < -1.0 else t2
    pitch_y = np.arcsin(t2)

    t3 = 2.0 * (w * z + x * y)
    t4 = 1.0 - 2.0 * (y * y + z * z)
    yaw_z = np.arctan2(t3, t4)

    return (roll_x, pitch_y, yaw_z)


'''
Purpose of the RSA cryptosystem is to have a secure way of transmitting data

Programmed by Aladdin Persson <aladdin.persson at hotmail dot com>
*   2019-08-26 Initial programming

'''

from math import gcd
from sympy import isprime
import random
from euclid_gcd import extended_euclidean

def generate_pq(bits):
    # Randomly generate two primes p,q
    p = random.getrandbits(bits)
    q = random.getrandbits(bits)

    # Check if p,q is prime
    p_isprime = isprime(p)
    q_isprime = isprime(q)

    # Keep generating until both are primes
    while (not (p_isprime and q_isprime)):
        if (not p_isprime):
            p = random.getrandbits(bits)
        if (not q_isprime):
            q = random.getrandbits(bits)

        p_isprime = isprime(p)
        q_isprime = isprime(q)

    return p, q

def generate_e(totient):
    # Generate e such that 1 < e < phi(n)
    # phi(n) in this case is totient

    while True:
        # Should be (2,totient) so if it is stuck in infinite loop then restart or replace 80000 -> totient
        # Reason why I want e to be a low value is to make encryption faster
        e = random.randint(2,50000)

        if gcd(e, totient) == 1:
            return e

def generate_d(e, totient):
    _, e_inverse, _ = extended_euclidean(e, totient)
    d = (e_inverse % totient)

    return d

def generate_all_values():
    num_bits = 1024
    p,q = generate_pq(num_bits)
    totient = (p-1) * (q-1)
    e = generate_e(totient)
    d = generate_d(e,totient)

    print('Generated value n: ' + str(p*q))
    print('Generated e and d: ' + str(e) + ' and ' + str(d))

    return p*q,e,d

def encrypt(message, n, e):
    encrypted = ''
    for letter in message:
        pad = 3 - len(str(ord(letter)))

        if pad > 0:
            new_letter = '0' * pad + str(ord(letter))
        else:
            new_letter = ord(letter)

        encrypted += str(new_letter)

    encrypted = pow(int(encrypted), e, n)
    return encrypted

def decrypt(encrypted, n, d):
    decrypted_message = ''
    decrypted_code = str(pow(encrypted, d, n))

    if len(decrypted_code) % 3 != 0:
        decrypted_code = '0' + decrypted_code

    while len(decrypted_code):
        decrypted_message += chr(int(decrypted_code[0:3]))
        decrypted_code = decrypted_code[3:]

    return decrypted_message

def example():
    # An example of a test case where we generate all necessary primes, encrypt and then decrypt the message.
    # Only to show how all parts of the code is working. This is not how it's going to be used in practice.
    hidden_message = "i really love peanuts"
    n,e,d = generate_all_values()
    encrypted_message = encrypt(hidden_message, n, e)
    decrypted_message = decrypt(encrypted_message, n, d)

    print('\n')
    print('Original message: ' + hidden_message)
    print('Encrypted message: ' + str(encrypted_message))
    print('Decrypted message: ' + decrypted_message)

def main():
    # Write the values of your RSA encryption (Note: Never give the 'd' to someone that doesn't want it)
    n = 354089397494626050014776605732143027269473328409397973403863001639624332101789181044818951483060155060788030618162673282176493895463414816015601230408140046833172059490430968956729878861381343553446553025440156523477822105773362480000716985478565013956749662865189691539813391686696182702224364834273144673717742246537383454469146642154754778836797926780437490677663302034284308892191362266103193070200405420180296005388479418941723827243187899338980201782128797489464650981164232057548015010630986959083998487019465357524040595865260220030689502065850060761344148196291328192760801074939658292752592564874822996765430361631210613041006858858506787439506504448316606509551260553919757840169593791152166515571202450662850988377002989153080277915454500432640601643512909764636398157415600050468972065216354878984114648007494687081718749734915103155014825420081658864982423629447913147575382146725524407739875786801876011026010419782863232303861065841801863420557617962438178979549855959377311548527613240676904989886382444381261628076009466895878852398923237601309285642954207693266358989851324012643315688180744573155217352955083176785543099571257338683938756048920161738393295253775030232399282686809347027784971441882883201432807953
    e = 9361
    #d =

    enc_or_dec = input("Would you like to encrypt or decrypt a message (input: 'enc' or 'dec'): ")

    if enc_or_dec.lower() == 'enc':
        hidden_message = input("What is your hidden message?: ")
        print('Encrypted message: ' + str(encrypt(hidden_message, n, e)))

    elif enc_or_dec.lower() == 'dec':
        encrypted = input('What is your encrypted message?: ')
        print(encrypted)
        print('Decrypted message: ' + str(decrypt(int(encrypted), n, d)))

    else:
        print("Not sure what you typed")

main()


from linear_algebra import matrix_size, dot_product, gramian
import numpy as np

class TestGramian:
    def test_gramian(self):
        matrix = [[3, -2], [-3, 2]]

        result = gramian(matrix)

        assert result.tolist() == [[18, -12], [-12, 8]]

    def test_gramian_3_by_3(self):
        matrix = np.array([[80, 60, 0], [150, 120, 100], [255, 220, 200]])

        result = gramian(matrix)

        assert result.tolist() == [
            [93925, 78900, 66000],
            [78900, 66400, 56000],
            [66000, 56000, 50000]]

class TestDotProduct:
    def test_dot_product(self):
        vector1 = [[1], [4], [7]]
        vector2 = [[2], [5], [8]]

        result = dot_product(vector1, vector2)

        assert result == 78

class TestMatrixSize:
    def test_matrix(self):
        matrix = [[1, 2], [4, 5], [7, 8]]
        rows, columns = matrix_size(matrix)

        assert rows == 3
        assert columns == 2

    def test_row_vector2(self):
        matrix = [[1, 2, 3]]
        rows, columns = matrix_size(matrix)

        assert rows == 1
        assert columns == 3

    def test_column_vector(self):
        matrix = [[1], [2], [3]]
        rows, columns = matrix_size(matrix)

        assert rows == 3
        assert columns == 1

    def test_empty(self):
        matrix = []
        rows, columns = matrix_size(matrix)

        assert rows == 0
        assert columns == 0


# https://towardsdatascience.com/how-to-use-and-create-a-z-table-standard-normal-table-240e21f36e53
# https://www.hackerrank.com/challenges/s10-normal-distribution-1/problem
"""Lesson"""
# https://www.hackerrank.com/challenges/s10-normal-distribution-1/tutorial

import numpy as np
from scipy.integrate import quad
from math import pi, sqrt, exp, erf

def z_score(x,mu,sd):
    # z-score
    """
    z = (x - mu) / S.D
    """
    # x is the individual variable
    # mu is the mean of the distribution
    # sd is the standard deviation of the distribution
    return (x-mu)/ sd

def normalProbabilityDensityFunction(x):
    # z-table
    """
    probability density function :
    f ( x | mean, S.D) = (1 / sqrt(2pi) ) ** ((-1/2)* ((x-mean)/s.d))**2)
    """
    # x is the z-score calculated
    return (1.0 / np.sqrt(2*np.pi)) * (np.exp((-x**2) / 2.0))

def pdf(x,m,s):
    # mean -> m, standard deviation -> s
    return (1.0 / (sqrt(2*pi)*s))* exp((-1/2)*((x-m)/s)**2)

def cdf(x,m,s):
    # f(x) = 1/2(1+erf((x-mean)/sd(sqrt(2))))
    # where erf(z) = 2/sqrt(pi)(integration:0->z(e^(-x**2)) dx)
    z = (x-m)/(s*sqrt(2))
    return (1+erf(z))/2

# print(quad(normalProbabilityDensityFunction,np.NINF,19.5)[0])

# let X be the random variable for normal distribution then probability density function will be
# pmf => P(X=x) = (1.0/ sqrt(2*3.14)*sd)* (2.71828)**((-1/2)*(xi-mu)/sd)
# empirical rule says
# 68% falls under one sd
# 95% falls under two sd
# 97.7% falls under three sd

def pdf(x,m,s):
    # mean -> m, standard deviation -> s
    return (1.0 / (sqrt(2*pi)*s))* exp((-1/2)*((x-m)/s)**2)

# Given that,
mean = 20
SD =  2

# Q1. P(x<19.5) = ?
x = 19.5

# z-table lookup
q1, _ = quad(normalProbabilityDensityFunction, np.NINF, z_score(x,mean,SD))

# probability of the area under the 19 hours is
print(round(q1 * 100,3))

#================================================================#

# Q2. P(20 < x < 22) = P(x<22) - P(x<20)
x1 = 20
x2 = 22

# P(x<20) = ?
a1, _ = quad(normalProbabilityDensityFunction, np.NINF, z_score(x1,mean,SD))

# P(x<22) = ?
a2, _ = quad(normalProbabilityDensityFunction, np.NINF, z_score(x2,mean,SD))

# Probability of the area between 20 and 22
q2 = a2 - a1
print(round(q2*100, 3))


## import numpy, panda, statsmodels
import pandas as pd
import numpy as np
import scipy
import statsmodels.api as sm
import matplotlib as mpl
mpl.use('TkAgg') # backend adjustment
import matplotlib.pyplot as plt
from math import sqrt

## read csv dataset
url = "https://raw.githubusercontent.com/sashaboulouds/statistics/master/data/CAPM_Exercise_formated.csv"
capm = pd.read_csv(url, sep=";", header=0)
df = pd.DataFrame(capm)

## set data
y = pd.DataFrame(capm, columns = ['Return AMZN'])
x = pd.DataFrame(capm, columns = ['Returns S&P'])

## model
x_model = sm.add_constant(x)
model1 = sm.OLS(y, x_model)
result1 = model1.fit()
print result1.summary()

## plot
plt.scatter(x, y, color='black')
plt.savefig('plot6_1.png')
plt.scatter(x, y, color='black')
slope = result1.params['Returns S&P']
intercept = result1.params['const']
abline = slope * x['Returns S&P'] + intercept
plt.plot(x, abline, color='blue')
plt.savefig('plot6_2.png')
plt.clf()

## Z
Z = (result1.params['Returns S&P']-1)/(result1.bse[1])
scipy.stats.norm.cdf(Z)

'''
x<-CAPM_Exercise$`Returns S&P`
y<-CAPM_Exercise$`Return AMZN`
regression<-lm(y~x)
plot(x,y)
summary(regression)
abline(regression,col="blue")
Z<-(summary(regression)$coefficients[2,1]-1)/summary(regression)$coefficients[2,2]
pnorm(Z)
'''

import numpy,import scipy,import statsmodels


import numpy
import matplotlib.pylab as plt
import fourier_transform as ft

######### Sampling signal points ################

# Sampling frequency, in Hz
f_s = 100
number_of_points = 512

start = 0
stop = (number_of_points - 1)/f_s
t = numpy.linspace(start, stop, number_of_points)

f_1 = 20
f_2 = 30
f_3 = 40
x = numpy.sin(2 * numpy.pi * 20 * t) \
    + numpy.sin(2 * numpy.pi * f_2 * t) \
    + numpy.sin(2 * numpy.pi * f_3 * t)
x_1 = x[0:128]

######### FFT using rectangular window ##############

plt.rcParams['axes.unicode_minus']=False
plt.rcParams['font.family'] = 'sans-serif'
plt.rcParams['font.sans-serif'] = ['SimHei', 'Helvetica', 'Calibri']
plt.rcParams['font.size'] = 20
plt.rcParams['figure.figsize'] = 50, 10

plt.plot(x_1)
plt.title('128 个样本信号')
plt.savefig(f'../figures/boxcar-128.svg', bbox_inches='tight')
plt.close()

ft.save_spectrum("boxcar-128", x_1)

plt.plot(x)
plt.title('512 个样本信号')
plt.savefig(f'../figures/boxcar-512.svg', bbox_inches='tight')
plt.close()

ft.save_spectrum("boxcar-512", x)


"""
Vicsek fractal
https://en.wikipedia.org/wiki/Vicsek_fractal
"""
import sys
import numpy as np
import matplotlib.pyplot as plt

def vicsek0():
    T = np.array([[0, 1, 0], [1, 0, 1], [0, 1, 0]])
    plt.matshow(T, cmap=plt.get_cmap('hot'))
    plt.title("Viscek fractal")
    plt.show()

def vicsek1():
    T1 = np.array([[0, 1, 0], [1, 0, 1], [0, 1, 0]])
    T2 = np.ones((3, 3))
    T3 = np.hstack((T2, T1, T2))
    T4 = np.hstack((T1, T2, T1))
    T5 = np.vstack((T4, T3, T4))
    plt.matshow(T5, cmap=plt.get_cmap('hot'))
    plt.title("Viscek fractal")
    plt.show()

def vicsek(n):
    T1 = np.array([[0, 1, 0], [1, 0, 1], [0, 1, 0]])
    for i in range(1, n + 1):
        T2 = np.ones((3**i, 3**i))
        T3 = np.hstack((T2, T1, T2))
        T4 = np.hstack((T1, T2, T1))
        T1 = np.vstack((T4, T3, T4))
    plt.matshow(T1, cmap='hot')
    plt.title("Viscek fractal")
    plt.show()

if __name__ == "__main__":
    n = int(sys.argv[1]) if len(sys.argv) == 2 else 4
    vicsek(n)


import pandas as pd
import numpy as np

import matplotlib

import matplotlib.pyplot as plt

import cufflinks as cf

import plotly
import plotly.offline as py
import plotly.graph_objs as go

## VECTOR MODULE

class Vector:
    def __init__(self, x , y , z):
        self.i = x
        self.j = y
        self.k = z

    def __str__(self):
        return "x = {}  ;    y = {} ;   z = {}".format(self.i,self.j,self.k)

    def __add__(self,other):

        return Vector(self.i + other.i , self.j + other.j , self.k + other.k)

    def __radd__(self, other):

        return self.__add__(other)

    def __sub__(self,other):

        return Vector(self.i - other.i , self.j - other.j , self.k - other.k)

    def __mul__(self,other):

        return Vector(self.i * other , self.j * other , self.k * other)

    def __rmul__(self,other):

        return self.__mul__(other)

    def __matmul__(self,other):

        return self.i * other.i + self.j * other.j + self.k * other.k

    def __neg__(self):

        return(Vector(-self.i,-self.j,-self.k))

    def contents(self):
        print("x = ",self.i , ";    y = ",self.j , ";   z = ",self.k )

    def array(self):
        array = np.array([self.i , self.j , self.k])
        return array

    def unit(self):
        d = np.sqrt((self.i)**2 + (self.j)**2 + (self.k)**2)
        if d > 0:
            uniter = Vector(self.i/d,self.j/d,self.k/d)
        else:
            uniter = Vector(self.i,self.j,self.k)

        return uniter

    def mag(self):
        d = np.sqrt((self.i)**2 + (self.j)**2 + (self.k)**2)

        return d

    def Cr_Pr(self):
        a = np.zeros((3,3))
        a[0,1] = -self.k
        a[0,2] =  self.j
        a[1,0] =  self.k
        a[1,2] = -self.i
        a[2,0] = -self.j
        a[2,1] =  self.i
        return a

## FUNCTIONS

# CONVERT NP ARRAY TO VECTOR
def array2vec(A):

    A = np.array(A)
    Avec = Vector(A[0],A[1],A[2])

    return Avec

# FIND EUCLIDIAN DISTANCE BETWEEN TWO (POSITION) VECTORS

def Distance(A , B):
    d = np.sqrt((A.i-B.i)**2 + (A.j-B.j)**2 + (A.k-B.k)**2)

    return d

# DOT PRODUCT OF TWO VECTORS

def Dot(A,B):
    d = A.i*B.i + A.j*B.j + A.k*B.k

    return d

# CROSS PRODUCT OF TWO QUARTERNIONS

def Cross(A,B):
    a = A.array()
    b = B.array()

    c = [a[1]*b[2] - a[2]*b[1],
         a[2]*b[0] - a[0]*b[2],
         a[0]*b[1] - a[1]*b[0]]

    C = array2vec(c)

    return C

# RETURN ANGLE BETWEEN TWO VECTORS

def Angle(A,B):
    costheta = Dot(A.unit(),B.unit())
    Theta = np.arccos(costheta)
    Theta = np.rad2deg(Theta)

    return Theta

def AnglewrtAxis(Ax,v0,v):
    AngleMag = Angle(v0,v)
    cosAng = v0.unit()@v.unit()
    #print(v0,v,cosAng,'cosine')
    if cosAng**2 < 0.9999:
        AngleSign = np.sign(Cross(v0,v).unit()@Ax)
        Anglet = AngleMag*AngleSign
    elif cosAng > 0:
        Anglet = 0
    elif cosAng < 0:
        Anglet = 180

    return Anglet

# CONVERT VECTOR TO UNIT VECTOR

def Unit(A):
    d = Dot(A,A)
    A = array2vec(((A.array)/d))

    return A

# ADD TWO VECTORS

def Add(A,B):
    C = Vector(0,0,0)
    C.i = A.i + B.i
    C.j = A.j + B.j
    C.k = A.k + B.k

    return C

# SUBTRACT TWO VECTORS

def Subtract(A,B):
    C = Vector(0,0,0)
    C.i = A.i - B.i
    C.j = A.j - B.j
    C.k = A.k - B.k

    return C

# SCALAR MULTIPLICATION

def ScalarMul(A,B):
    C = Vector(0,0,0)
    C.i = A*B.i
    C.j = A*B.j
    C.k = A*B.k

    return C

# SCALAR TRIPLE PRODUCT

def ScTrProduct(a,b,c):
    Prod = Dot(a,Cross(b,c))
    return Prod

# PROJECT POSITION VECTOR ON A PLANE DEFINED BY NORMAL AND CONSTITUENT POINT

def Project(DC,A,B):
    DC = DC.unit()

    C1 = B
    C2mul = Dot(A,DC) - Dot(B,DC)

    C2 = ScalarMul(C2mul,DC)
    print(type(C2),type(C1))

    Cone = Add(C1,C2)
    Ctwo = Subtract(C1,C2)

    coned = Distance(Cone,A)
    ctwod = Distance(Ctwo,A)

    if coned < ctwod:
        C = Cone
    elif coned > ctwod:
        C = Ctwo
    elif coned == ctwod:
        C = Cone

    return C

# TRANSFORM A VECTOR BY THETA ALONG AXIS

def TransformVector(Theta, Vector, AxisVector):
    Vector = Vector.array()
    Axis = AxisVector.array()

    Theta = np.radians(Theta)
    Vectormag = np.linalg.norm(Vector)
    Vector = Vector / Vectormag
    Axis = Axis / np.linalg.norm(Axis)

    l = Axis[0]
    m = Axis[1]
    n = Axis[2]

    CrossMat = np.array([[0 , - n , m ] , [n,0 , -l] , [-m , l , 0]])

    Matrix1 = np.cos(Theta)*np.eye(3) + np.sin(Theta)*CrossMat
    Matrix2 = (1 - np.cos(Theta))*np.outer(Axis,Axis)
    Matrix = Matrix1 + Matrix2

    VectorOut = np.around(Vectormag*np.matmul(Matrix,np.array(Vector)), decimals = 8)

    VectorOut = array2vec(VectorOut)

    return VectorOut


# USAGE
# python image_drawing_practice.py

# import the necessary packages
import argparse
import re
import cv2
import numpy as np
from skimage.exposure import rescale_intensity

def convolve(image, kernel):
    # grab the spatial dimensions of the image, along with the spatial
    # dimensions of the kernel
    (iH, iW) = image.shape[:2]
    (kH, kW) = kernel.shape[:2]

    # allocate memmory for the output image, taking care to "pad" the
    # borders of the input image so the spatial size (i.e., width and height) are not reduced
    pad = (kW - 1) // 2
    image = cv2.copyMakeBorder(image, pad, pad, pad, pad,
        cv2.BORDER_REPLICATE)
    output = np.zeros((iH, iW), dtype="float32")

    # loop over the input image, "sliding" the kernel across each
    # (x, y)-coordinate from left-to-right and top to bottom
    for y in np.arange(pad, iH + pad):
        for x in np.arange(pad, iW + pad):
            # extract the ROI of the image by extracting the *center* region
            # of the current (x, y)-coordinates dimensions
            roi = image[y - pad:y + pad + 1, x - pad:x + pad + 1]

            # perform the actual convolution by taking the element-wise
            # mulplicate  between the ROI and the kernel, the summing the
            # matrix
            k = (roi * kernel).sum()

            # store the colvolded value in the output (x, y)-coordinate of
            # the output image
            output[y - pad, x - pad] = k
    # rescale the output image to be in the range [0, 255]
    output = rescale_intensity(output, in_range=(0, 255))
    output = (output * 255).astype("uint8")

    #return the output image
    return output

# construct the argument parser and the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-i", "--image", type=str, required=True,
    help="path to the input image")
args = vars(ap.parse_args())

# construct average blurring kernels used to smooth an image
smallBlur = np.ones((7, 7), dtype="float") * (1.0 / (7 * 7))
largeBlur = np.ones((21, 21), dtype="float") * (1.0 / (21 * 21))

# construct a sharpening filter
sharpen = np.array((
    [0 , -1, 0],
    [-1, 5, -1],
    [0, -1, 0]), dtype="int")

# construct the Laplacian kernel to detect edge-like regions on an image
laplacian = np.array((
    [0, 1, 0],
    [1, -4, 1],
    [0 ,1 ,0]), dtype="int")

# construct the Sobel x-asis kernel
sobelX = np.array((
    [-1, 0, 1],
    [-2, 0 , 2],
    [-1, 0, 1]), dtype="int")

# construct the Sobel y-axis kernel
sobelY = np.array((
    [-1, -2, -1],
    [0, 0 ,0],
    [1, 2, 1]), dtype="int")

# construct the kernel bank,  a list of kernels we're going to apply using out
# custom 'convolve' function and OpenCV's 'filter2D' function
kernelBank = (
    ("small_blur", smallBlur),
    ("large_blur", largeBlur),
    ("sharpen", sharpen),
    ("laplacian", laplacian),
    ("sobel_x", sobelX),
    ("sobel_y", sobelY)
)

# load the input image and convert it to grayscale
image = cv2.imread(args["image"])
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

# loop over the kernels
for (kernelName, kernel) in kernelBank:
    # apply the kernel to the grayscale image using both our custom 'convolve'
    # fundtion and OpenCV;s 'filter2D' funstion
    print("[INFO] applying {} kernel".format(kernelName))
    convoleOutput = convolve(gray, kernel)
    opencvOutput = cv2.filter2D(gray, -1, kernel)

    # show the output images
    cv2.imshow("original", gray)
    cv2.imshow("{} - convole".format(kernelName), convoleOutput)
    cv2.imshow("{} - opencv".format(kernelName), opencvOutput)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

OpenCV 102/Basic Image Processing Operations/convolutions-opencv/convolutions_practice.py


# -*- coding: utf-8 -*-
"""Linear_Regression_Prac_1.ipynb

Automatically generated by Colaboratory.

Original file is located at
    https://colab.research.google.com/drive/1JzxOyr6U473zZweo3pKUeL1Vlu6DlYzZ

## Linear Regression Prac 1

This notebook will make use of Scikit-learn's Linear Regression class. The documentation is found here: https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html

Credits:

The notebooks on linear regression are modifications of various sources which include:
https://scikit-learn.org/stable/auto_examples/linear_model/plot_ols.html
https://medium.com/analytics-vidhya/linear-regression-using-python-ce21aa90ade6
https://www.kdnuggets.com/2019/03/beginners-guide-linear-regression-python-scikit-learn.html/2

## Various Python imports
"""

import matplotlib.pyplot as plt
import numpy as np
from sklearn import datasets
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, mean_absolute_error
from sklearn.model_selection import train_test_split

"""## Load the dataset"""

diabetes = datasets.load_diabetes()

"""## Take a look at what has been downloaded"""

diabetes

"""## Print out the feature names"""

diabetes['feature_names']

"""## Take a look at the shape of the data"""

diabetes['data'].shape

"""## Take a look at the shape of the targets (what we are predicting for)"""

diabetes['target'].shape

"""## Select a training feature

In this case we will be using a single feature for training. The algorithm will then try map this feature to the targets. In this case we are randomly select the feature number 5, however, this might not be the best choice! Feature '5' corresponds to the feature 's2' in our dataset (look above at our mapping).

Typically, features are denoted as X and targets as Y (some authors will use 'y' instead).
"""

X = diabetes.data[:, 1]
y = diabetes['target']

"""## Check shape!"""

X.shape

X = X.reshape(-1,1)

X.shape

"""## Split into training and testing"""

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=4278) # 70% training and 30% test

"""## Check the training features shape"""

X_train.shape

"""## Check the testing features shape"""

X_test.shape

"""## Define a linear regression model

Read the API: https://scikit-learn.org/stable/modules/linear_model.html
"""

model = LinearRegression()

"""## Look at what has just being initialised"""

model

"""## Train the model/fit the model"""

model.fit(X_train, y_train)

"""## Print the coefficient"""

model.coef_

"""## Print the intercept"""

model.intercept_

"""## The model

The model, given the above output, is as follows:

y = 149 + 381x

y = 149 + 381d1, d1 = feature 5

y = 149 d0 + 381 d1, d0 = 1, d1 = feature 5

## Apply the model to the testing features and get the predictions
"""

Y_pred = model.predict(X_test)

"""## Determine the performance of the model on the testing data

There are various options, but commonly used one include:

* mean squared error
* mean absolute error
* root mean squared error

There can be implemented using scikit-learn.

### Mean squared error
"""

mean_squared_error(y_test, Y_pred)

"""### Mean absolute error"""

mean_absolute_error(y_test, Y_pred)

"""### Root mean squared error"""

np.sqrt(mean_squared_error(y_test, Y_pred))

"""## Plot the line and the testing data"""

plt.scatter(X_test, y_test,  color='gray')
plt.title('Linear Regression')
plt.plot(X_test, Y_pred, color='red', linewidth=1)
plt.show()

"""## Task: determine which feature will result in the best performing model."""