#!/usr/bin/env python
# coding: utf-8

# # NAM 2019 pMuTT Workshop
#
# Instructions and materials for the "Theory, Applications, and Tools for Kinetic Modeling" workshop can be found on [our documentation page](https://vlachosgroup.github.io/pMuTT/nam_2019.html).
#

# # Table of Contents
#
# | **1\. [Virtual Kinetic Laboratory Ecosystem](#section_1)**
#
# | **2\. [Useful Links](#section_2)**
#
# | **3\. [Constants](#section_3)**
#
# |-- **3.1. [Access common constants in appropriate units](#section_3_1)**
#
# |-- **3.2. [Convert between units](#section_3_2)**
#
# |-- **3.3. [Convert between equivalent quantities](#section_3_3)**
#
# | **4\. [Exercise 1](#section_4)**
#
# | **5\. [Creating statistical mechanical objects using StatMech](#section_5)**
#
# |-- **5.1. [Supported StatMech models](#section_5_1)**
#
# |--|-- **5.1.1 [Translations](#section_5_1_1)**
#
# |--|-- **5.1.2. [Vibrations](#section_5_1_2)**
#
# |--|-- **5.1.3. [Rotations](#section_5_1_3)**
#
# |--|-- **5.1.4. [Electronic](#section_5_1_4)**
#
# |--|-- **5.1.5. [Miscellaneous](#section_5_1_5)**
#
# |-- **5.2. [Initializing StatMech modes individually](#section_5_2)**
#
# |-- **5.3. [Initializing StatMech modes using presets](#section_5_3)**
#
# | **6\. [Plot Thermodynamic Quantities](#section_6)**
#
# | **7\. [Exercise 2](#section_7)**
#
# | **8\. [Creating empirical objects](#section_8)**
#
# |-- **8.1. [Inputting a NASA polynomial directly](#section_8_1)**
#
# |-- **8.2. [Fitting an empirical object to a StatMech object](#section_8_2)**
#
# | **9\. [Input/Output](#section_9)**
#
# |-- **9.1. [Input via Excel](#section_9_1)**
#
# |-- **9.2. [Output via Thermdat](#section_9_2)**
#
# | **10\. [Reactions](#section_10)**
#
# | **11\. [Exercise 3](#section_11)**
#
# | **12\. [Solutions](#section_12)**
#
# |-- **12.1. [Solution 1](#section_12_1)**
#
# |-- **12.2. [Solution 2](#section_12_2)**
#
# |-- **12.3. [Solution 3](#section_12_3)**

#

# # 1. Virtual Kinetic Laboratory Ecosystem
#
#
#
#
#
#
# - Estimates thermochemical and kinetic parameters using statistical mechanics, transition state theory
# - Writes input files for kinetic models and eases thermodynamic analysis
# - Implemented in Python
#   - Easy to learn
#   - Heavily used in scientific community
#   - Object-oriented approach is a natural analogy to chemical phenomenon
# - Library approach allows users to define the starting point and end point
#
#
#

#

# # 2. Useful Links
#
# - [Documentation](https://vlachosgroup.github.io/pMuTT/): find the most updated documentation
# - [Issues](https://github.com/VlachosGroup/pmutt/issues): report bugs, request features, receive help
# - [Examples](https://vlachosgroup.github.io/pMuTT/examples.html): see examples

#

# # 3. Constants
#
# The [constants module](https://vlachosgroup.github.io/pMuTT/constants.html) has a wide variety of functions for constants and unit conversion.

#

# ## 3.1. Access common constants in appropriate units
# Below, we access Planck's constant in J s.

# In[1]:

from pmutt import constants as c

h1 = c.h('eV s', bar=True)
print('h = {} eV s'.format(h1))

#

# ## 3.2. Convert between units
# Below, we convert 12 atm of pressure to psi.

# In[2]:

from pmutt import constants as c

P_atm = 12. # atm
P_psi = c.convert_unit(num=P_atm, initial='atm', final='psi')

print('{} atm = {} psi'.format(P_atm, P_psi))

#

# ## 3.3. Convert between equivalent quantities
# Below, we convert 1000 wavenumbers (cm-1) to frequency.

# In[3]:

from pmutt import constants as c

wave_num = 1000. # cm-1
freq = c.wavenumber_to_freq(wave_num) # Hz

print('{} cm-1 = {} Hz'.format(wave_num, freq))

#

# # 4. Exercise 1
#
# Using `pmutt.constants`, calculate the dimensionless enthalpy (H/RT) using the following information:
# - H = 0.5 eV
# - T = 77 F

# In[4]:

# Fill in your answer for Exercise 1 here

#

# # 5. Creating statistical mechanical objects using StatMech
#
# Molecules show translational, vibrational, rotational, electronic, and nuclear modes.
#
#

#

# ## 5.1. Supported StatMech modes
#
#
#
# The StatMech object allows us to specify translational, vibrational, rotational, electronic and nuclear modes independently, which gives flexibility in what behavior you would like. Below are the available modes.

#

# ### 5.1.1. Translations
# - [``FreeTrans``](https://vlachosgroup.github.io/pMuTT/statmech.html#freetrans) - Translations assuming no intermolecular interactions

#

# ### 5.1.2. Vibrations
# - [``HarmonicVib``](https://vlachosgroup.github.io/pMuTT/statmech.html#harmonicvib) - Harmonic vibrations
# - [``QRRHOVib``](https://vlachosgroup.github.io/pMuTT/statmech.html#harmonicvib) - Quasi rigid rotor harmonic oscillator. Low frequency modes are treated as rigid rotations.
# - [``EinsteinVib``](https://vlachosgroup.github.io/pMuTT/statmech.html#einsteinvib) - Each atom in the crystal vibrates as independent 3D harmonic oscillators
# - [``DebyeVib``](https://vlachosgroup.github.io/pMuTT/statmech.html#debyevib) - Improves upon ``EinsteinVib`` by considering simultaneous vibrations. Improves accuracy at lower temperatures.

#

# ### 5.1.3. Rotations
# - [``RigidRotor``](https://vlachosgroup.github.io/pMuTT/statmech.html#rigidrotor) - Molecule can be rotated with no change in bond properties

#

# ### 5.1.4. Electronic
# - [``GroundStateElec``](https://vlachosgroup.github.io/pMuTT/statmech.html#groundstateelec) - Electronic ground state of the system
# - [``LSR``](https://vlachosgroup.github.io/pMuTT/statmech.html#linear-scaling-relationships-lsrs) - Linear Scaling Relationship to estimate binding energies using reference adsorbate

#

# ### 5.1.5. Miscellaneous
# - [``EmptyMode``](https://vlachosgroup.github.io/pMuTT/statmech.html#empty-mode) - Default mode if not specified. Does not contribute to any properties
# - [``ConstantMode``](https://vlachosgroup.github.io/pMuTT/statmech.html#constant-mode) - Specify arbitrary values to thermodynamic quantities
#
# Using a ``StatMech`` mode gives you access to all the common thermodynamic properties.
#
#
#
# For this example, we will use a hydrogen molecule as an ideal gas:
# - translations with no interaction between molecules
# - harmonic vibrations
# - rigid rotor rotations
# - ground state electronic structure
# - no contribution from nuclear modes.
#
#

#

# ## 5.2. Initializing StatMech modes individually

# In[5]:

from ase.build import molecule
from pmutt.statmech import StatMech, trans, vib, rot, elec

H2_atoms = molecule('H2')

'''Translational'''
H2_trans = trans.FreeTrans(n_degrees=3, atoms=H2_atoms)

'''Vibrational'''
H2_vib = vib.HarmonicVib(vib_wavenumbers=[4342.]) # vib_wavenumbers in cm-1

'''Rotational'''
H2_rot = rot.RigidRotor(symmetrynumber=2, atoms=H2_atoms)

'''Electronic'''
H2_elec = elec.GroundStateElec(potentialenergy=-6.77,spin=0) # potentialenergy in eV

'''StatMech Initialization'''
H2_statmech = StatMech(name='H2',
                       trans_model=H2_trans,
                       vib_model=H2_vib,
                       rot_model=H2_rot,
                       elec_model=H2_elec)

'''Calculate thermodynamic properties'''
H_statmech = H2_statmech.get_H(T=298., units='kJ/mol')
S_statmech = H2_statmech.get_S(T=298., units='J/mol/K')
print('H_H2(T=298 K) = {:.1f} kJ/mol'.format(H_statmech))
print('S_H2(T=298 K) = {:.2f} J/mol/K'.format(S_statmech))

#

# ## 5.3. Initializing StatMech modes using presets
#
# Commonly used models can be accessed via [``presets``](https://vlachosgroup.github.io/pMuTT/statmech.html#presets). The currently supported models are:
#
# - [``idealgas``](https://vlachosgroup.github.io/pMuTT/statmech.html#ideal-gas-idealgas) - Ideal gases
# - [``harmonic``](https://vlachosgroup.github.io/pMuTT/statmech.html#harmonic-approximation-harmonic) - Typical for surface species
# - [``electronic``](https://vlachosgroup.github.io/pMuTT/statmech.html#electronic-electronic) - Only has electronic modes
# - [``placeholder``](https://vlachosgroup.github.io/pMuTT/statmech.html#placeholder-placeholder) - No contribution to any property
# - [``constant``](https://vlachosgroup.github.io/pMuTT/statmech.html#constant-constant) - Use arbitrary constants to thermodynamic properties
#

# In[6]:

from ase.build import molecule
from pmutt.statmech import StatMech, presets

H2_statmech = StatMech(atoms=molecule('H2'),
                       vib_wavenumbers=[4342.], # cm-1
                       symmetrynumber=2,
                       potentialenergy=-6.77, # eV
                       spin=0.,
                       **presets['idealgas'])

'''Calculate thermodynamic properties'''
H_statmech = H2_statmech.get_H(T=298., units='kJ/mol')
S_statmech = H2_statmech.get_S(T=298., units='J/mol/K')
print('H_H2(T=298 K) = {:.1f} kJ/mol'.format(H_statmech))
print('S_H2(T=298 K) = {:.2f} J/mol/K'.format(S_statmech))

#

# # 6. Plot Thermodynamic Quantities
# Use [`pmutt.plot_1D`](https://vlachosgroup.github.io/pMuTT/visual.html#plot-1d) and [`pmutt.plot_2D`](https://vlachosgroup.github.io/pMuTT/visual.html#plot-2d) to plot any function with respect to 1 or 2 variables.

# In[7]:

import numpy as np
from matplotlib import pyplot as plt
from pmutt import plot_1D, plot_2D

T = np.linspace(300., 500.)

f1, ax1 = plot_1D(H2_statmech,
                  x_name='T', x_values=T,
                  methods=('get_H', 'get_S', 'get_G'),
                  get_H_kwargs={'units': 'kcal/mol'},
                  get_S_kwargs={'units': 'cal/mol/K'},
                  get_G_kwargs={'units': 'kcal/mol'})
f1.set_size_inches(6, 6)
f1.set_dpi(200)
plt.show()

#

# # 7. Exercise 2
#
# 1. Create a ``StatMech`` object for ideal gas-phase H2O. The necessary inputs are given below.
#
# |            Parameter           |             Value            |
# |--------------------------------|------------------------------|
# |              atoms             |       `molecule('H2O')`      |
# |      Potential Energy (eV)     |            -6.7598           |
# |         Symmetry number        |               2              |
# |              Spin              |               0              |
# | Vibrational Wavenumbers (cm-1) | 3825.434, 3710.264, 1582.432 |
#
# 2. Calculate the Gibbs energy in eV for H2O at T = 500 K and P = 2 bar.
#
# 3. Create a ``StatMech`` object for a Cu crystal using the ``DebyeVib`` model for the vibration mode and ``GroundStateElec`` model for electronic mode. The necessary inputs are given below.
#
# | Parameter              | Value      |
# |------------------------|------------|
# | Debye Temperature (K)  | 310        |
# | Interaction energy (eV)| 0          |
# | Potential energy (eV)  | -14.922356 |
#
# 4. Plot the H (in eV) and S (in eV/K) for Cu between T = 300 - 700 K.

# In[8]:

# Fill in your answer for Exercise 2 here

# 1.

# 2.

# 3.

# 4.

#

# # 8. Creating empirical objects
# Currently, pMuTT supports [NASA polynomials](https://vlachosgroup.github.io/pMuTT/empirical.html#nasa) and [Shomate polynomials](https://vlachosgroup.github.io/pMuTT/empirical.html#shomate). They can be initialized in three ways:
# - passing in the polynomials directly
# - from a model (e.g. ``StatMech``, ``Shomate``) (``from_model``)
# - from heat capacity, enthalpy and entropy data (``from_data``)
#
#

#

# ## 8.1. Inputting a NASA polynomial directly
#
# The H2 NASA polynomial from the [Burcat database](http://combustion.berkeley.edu/gri_mech/version30/files30/thermo30.dat) is represented as:
#
# ```
# H2                TPIS78H   2               G   200.000  3500.000  1000.000    1
#  3.33727920E+00-4.94024731E-05 4.99456778E-07-1.79566394E-10 2.00255376E-14    2
# -9.50158922E+02-3.20502331E+00 2.34433112E+00 7.98052075E-03-1.94781510E-05    3
#  2.01572094E-08-7.37611761E-12-9.17935173E+02 6.83010238E-01                   4
# ```
#
# This can be translated to pMuTT syntax using:

# In[9]:

from pmutt.empirical.nasa import Nasa

# Initialize NASA polynomial
H2_nasa = Nasa(name='H2',
               elements={'H': 2},
               phase='G',
               T_low=200., T_mid=1000., T_high=3500.,
               a_low=[2.34433112E+00, 7.98052075E-03, -1.94781510E-05,
                      2.01572094E-08, -7.37611761E-12, -9.17935173E+02,
                      6.83010238E-01],
               a_high=[3.33727920E+00, -4.94024731E-05, 4.99456778E-07,
                       -1.79566394E-10, 2.00255376E-14, -9.50158922E+02,
                       -3.20502331E+00])

# Calculate thermodynamic quantities using the same syntax as StatMech
H_H2 = H2_nasa.get_H(units='kcal/mol', T=298.)
print('H_H2(T=298 K) = {} kcal/mol'.format(H_H2))

# Show thermodynamic quantities vs. T
T = np.linspace(200., 3500.)
f2, ax2 = plot_1D(H2_nasa,
                  x_name='T', x_values=T,
                  methods=('get_H', 'get_S', 'get_G'),
                  get_H_kwargs={'units': 'kcal/mol'},
                  get_S_kwargs={'units': 'cal/mol/K'},
                  get_G_kwargs={'units': 'kcal/mol'})
f2.set_size_inches(6, 6)
f2.set_dpi(200)
plt.show()

#

# ## 8.2. Fitting an empirical object to a StatMech object
# Empirical objects can be made directly using ``StatMech`` objects and the ``from_model`` method.

# In[10]:

H2_nasa = Nasa.from_model(name='H2',
                          T_low=200.,
                          T_high=3500.,
                          model=H2_statmech)

# Compare the statistical mechanical model to the empirical model
f3, ax3 = H2_nasa.plot_statmech_and_empirical(Cp_units='J/mol/K',
                                              H_units='kJ/mol',
                                              S_units='J/mol/K',
                                              G_units='kJ/mol')
f3.set_size_inches(6, 8)
f3.set_dpi(200)
plt.show()

#

# # 9. Input/Output
# pMuTT has more IO functionality than below. See this page for [supported IO functions](https://vlachosgroup.github.io/pMuTT/io.html).

#

# ## 9.1. Input via Excel
#
# Encoding each object in Python can be tedious. You can read several species from Excel spreadsheets using [``pmutt.io.excel.read_excel``](https://vlachosgroup.github.io/pmutt/io.html?highlight=read_excel#pmutt.io.excel.read_excel). Note that this function returns a list of dictionaries. This output allows you to initialize whichever object you want using kwargs syntax. There are also [special rules that depend on the header name](https://vlachosgroup.github.io/pMuTT/io.html#special-rules).
#
# Below, we show an example importing species data from a spreadsheet and creating a series of NASA polynomials.

# In[11]:

import os
from pprint import pprint
from pathlib import Path
from pmutt.io.excel import read_excel
from pmutt.empirical.nasa import Nasa

# Find the location of Jupyter notebook
# Note that normally Python scripts have a __file__ variable but Jupyter notebook doesn't.
# Using pathlib can overcome this limiation
try:
    notebook_folder = os.path.dirname(__file__)
except NameError:
    notebook_folder = Path().resolve()
os.chdir(notebook_folder)

# Read the data from Excel
ab_initio_data = read_excel(io='./input/NH3_Input_Data.xlsx', sheet_name='species')
pprint(ab_initio_data)

# In[12]:

# Create NASA polynomials using **kwargs syntax
nasa_species = []
for species_data in ab_initio_data:
    single_nasa_species = Nasa.from_model(T_low=100.,
                                          T_high=1500.,
                                          **species_data)
    nasa_species.append(single_nasa_species)

# Print out a table using enthalpy, entropy, Gibbs energy at 298 K for each species
print('Name         Enthalpy (kcal/mol)   Entropy (cal/mol/K)   Gibbs energy (kcal/mol)')
print('--------------------------------------------------------------------------------')
for single_nasa_species in nasa_species:
    name = single_nasa_species.name
    H = single_nasa_species.get_H(units='kcal/mol', T=298.)
    S = single_nasa_species.get_S(units='cal/mol/K', T=298.)
    G = single_nasa_species.get_G(units='kcal/mol', T=298.)
    print('{:12}       {:10.1f}       {:15.1f}       {:15.1f}'.format(name, H, S, G))

#

# ## 9.2. Output via Thermdat
# The thermdat format uses NASA polynomials to represent several species. It has a very particular format so doing it manually is error-prone. You can write a list of ``Nasa`` objects to thermdat format using [``pmutt.io.thermdat.write_thermdat``](https://vlachosgroup.github.io/pmutt/io.html#pmutt.io.thermdat.write_thermdat).
#
# Below, we write a thermdat file using the species imported from the spreadsheet.

# In[13]:

from pmutt.io.thermdat import write_thermdat

write_thermdat(filename='./output/thermdat', nasa_species=nasa_species)

# Similarly, a list of ``Nasa`` objects can be read from a thermdat using [``pmutt.io.thermdat.read_thermdat``](https://vlachosgroup.github.io/pMuTT/io.html#pmutt.io.thermdat.read_thermdat).

# In[14]:

from pmutt.io.thermdat import read_thermdat

nasa_species = read_thermdat('./output/thermdat')

#

# # 10. Reactions
#
#
#
# ``Reaction`` objects can be created by putting together ``Nasa``, ``Shomate`` and ``StatMech`` objects.
#
#
#
#
# The ``from_string`` method is the easiest way to create a ``Reaction`` object. It requires the relevant species to be in a dictionary and a string to describe the reaction.
#
#
#
# We will demonstrate its use for the formation of NH3.

# In[15]:

from pmutt.empirical.nasa import Nasa
from pmutt.empirical.shomate import Shomate
from pmutt.reaction import Reaction

# Create species. Note that you can mix different types of species
species = {
    'H2': StatMech(name='H2', atoms=molecule('H2'),
                   vib_wavenumbers=[4342.], # cm-1
                   symmetrynumber=2,
                   potentialenergy=-6.77, # eV
                   spin=0.,
                   **presets['idealgas']),
    'N2': Nasa(name='N2', T_low=300., T_mid=643., T_high=1000.,
               a_low=[3.3956319945669633, 0.001115707689025668,
                      -4.301993779374381e-06, 6.8071424019295535e-09,
                      -3.2903312791047058e-12, -191001.55648623788,
                      3.556111439828502],
               a_high=[4.050329990684662, -0.0029677854067980108,
                       5.323485005316287e-06, -3.3518122405333548e-09,
                       7.58446718337381e-13, -191086.2004520406,
                       0.6858235504924011]),
    'NH3': Shomate(name='NH3', T_low=300., T_high=1000.,
                   a=[18.792357134351683, 44.82725349479501,
                      -10.05898449447048, 0.3711633831565547,
                      0.2969942466370908, -1791.225746924463,
                      203.9035662274934, 1784.714638346206]),
}

# Define the formation of water reaction
rxn = Reaction.from_string('1.5H2 + 0.5N2 = NH3', species)

# Calculate forward change in enthalpy
H_rxn_fwd = rxn.get_delta_H(units='kcal/mol', T=300.)
print('Delta H_fwd(T = 300 K) = {:.1f} kcal/mol'.format(H_rxn_fwd))

# Calculate reverse change in enthalpy
H_rxn_rev = rxn.get_delta_H(units='kcal/mol', T=300., rev=True)
print('Delta H_rev(T = 300 K) = {:.1f} kcal/mol'.format(H_rxn_rev))

# Calculate enthalpy of reactants
H_react = rxn.get_H_state(units='kcal/mol', T=300., state='reactants')
print('H_reactants(T = 300 K) = {:.1f} kcal/mol'.format(H_react))

#

# ## 11. Exercise 3
#
# 1. Use [``pmutt.io.thermdat.read_thermdat``](https://vlachosgroup.github.io/pMuTT/io.html#pmutt.io.thermdat.read_thermdat) to read the thermdat from './output/thermdat'.
#
# 2. Convert the list of ``Nasa`` to a dictionary of ``Nasa`` using ``pmutt.pmutt_list_to_dict``. The syntax to use the function is shown below.
#
# ```
# from pmutt import pmutt_list_to_dict
#
# species_dict = pmutt_list_to_dict(species_list)
# ```
#
# 3. Create a ``Reaction`` from the string: ``NH3(S) + RU(S) = TS1_NH3(S) = NH2(S) + H(S)`` and the dictionary from step 2.
#
# 4. Calculate the forward reaction enthalpy in kcal/mol at 298 K using [``Reaction.get_delta_H``](https://vlachosgroup.github.io/pMuTT/reactions.html#pmutt.reaction.Reaction.get_E_act).
#
# 5. Calculate the forward activation energy in kcal/mol at 298 K using [``Reaction.get_E_act``](https://vlachosgroup.github.io/pMuTT/reactions.html#pmutt.reaction.Reaction.get_E_act).

# In[16]:

# Fill in your answer for Exercise 3

# 1.

# 2.

# 3.

# 4.

# 5.

#

# # 12. Solutions

#

# ## 12.1. Solution to Exercise 1
# [Link to Exercise 1](#section_4)

# In[17]:

from pmutt import constants as c

# Define information given
H = 0.5 # eV
T = 77 # F

# Calculate H/RT
HoRT = H/c.R('eV/K')/c.convert_unit(77, initial='F', final='K')
print('H/RT = {}'.format(HoRT))

#

# ## 12.2. Solution to Exercise 2
# [Link to Exercise 2](#section_7)

# In[18]:

from ase.build import molecule
from pmutt import plot_1D
from pmutt.statmech import StatMech, presets
from pmutt.statmech.vib import DebyeVib
from pmutt.statmech.elec import GroundStateElec

# 1. Create H2O molecule
H2O_statmech = StatMech(atoms=molecule('H2O'),
                        potentialenergy=-6.7598,
                        symmetrynumber=2,
                        spin=0,
                        vib_wavenumbers=[3825.434, 3710.264, 1582.432],
                        **presets['idealgas'])

# 2. Calculate Gibbs energy of H2O at T = 500 K and P = 2 bar
T = 500. # K
P = 2. # bar
G_H2O = H2O_statmech.get_G(units='eV', T=T, P=P)
print('G_H2O(T = 500 K, P = 2 bar) = {} eV'.format(G_H2O))

# 3. Create Cu crystal
Cu_vib = DebyeVib(debye_temperature=310., interaction_energy=0.)
Cu_elec = GroundStateElec(potentialenergy=-14.922356)
Cu_statmech = StatMech(vib_model=Cu_vib, elec_model=Cu_elec)

# 4. Plot the 1D profile for H and S between 300 K and 700 K
T = np.linspace(300., 700.) # K
f2, ax2 = plot_1D(Cu_statmech,
                  x_name='T', x_values=T,
                  methods=('get_H', 'get_S'),
                  get_H_kwargs={'units': 'eV'},
                  get_S_kwargs={'units': 'eV/K'})
f2.set_size_inches(6, 6)
f2.set_dpi(200)
plt.show()

#

# ## 12.3. Solution to Exercise 3
# [Link to Exercise 3](#section_11)

# In[19]:

from pmutt.io.thermdat import read_thermdat
from pmutt import pmutt_list_to_dict
from pmutt.reaction import Reaction

# 1. Read the thermdat file
species_list = read_thermdat('./output/thermdat')

# 2. Convert the list of Nasa to a dictionary of Nasa
species_dict = pmutt_list_to_dict(species_list)

# 3. Create a Reaction from the specified string
rxn = Reaction.from_string('NH3(S) + RU(S) = TS1_NH3(S) = NH2(S) + H(S)', species_dict)

# 4. Calculate the reaction enthalpy
H_rxn = rxn.get_delta_H(units='kcal/mol', T=298.)
print('H_rxn = {:.1f} kcal/mol'.format(H_rxn))

# 5. Calculate the forward activation energy
Ea = rxn.get_E_act(units='kcal/mol', T=298.)
print('Ea_fwd = {:.1f} kcal/mol'.format(Ea))


# -*- coding: utf-8 -*-
# ---
# jupyter:
#   jupytext:
#     formats: ipynb,py
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.6.0
#   kernelspec:
#     display_name: deep_ml_curriculum
#     language: python
#     name: deep_ml_curriculum
# ---

# # Introduction to Dask
# There are many occasions when we have to work with datasets that are so big we can't just load all of it into memory. If we have to work with such a data, what is the solution? One of the libraries in python for this type of problems is __Dask__. Dask is a library for parallel computing. It helps us perform common pandas and numpy opperations on large datasets. In this tutorial we will learn about some of the features of Dask and how it can help us.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import dask
import dask.dataframe as dd
import os
import logging
import psutil

# For the beginning to see the difference in the performance of Dask vs Pandas, let's read a 70 MB csv file with both libraries. Of course 70 MB is not considered a large file and we can easily fit it into memory, but it is large enough to see the advantage of using Dask.

path = "../../data/processed/MNIST/train.csv"

# We are also defining a function to report memory usage, so we can see how each method affect the memory.

def memory_usage():
    """String with current memory usage in MB. Requires `psutil` package."""
    pid = os.getpid()
    mem_bytes = psutil.Process(pid).memory_info().rss
    print('[Process {} uses {:.1f}MB]'.format(pid, mem_bytes / 1024 / 1024))
    return mem_bytes / 1024 / 1024

memory_usage()

# You should be able to see the amount of used memory.

# Now let's load the data with pandas and see how long it takes to load and how much memory it takes.

# %%time
df1 = pd.read_csv(path)

memory_usage()

# The memory usage has gone up by about ~250 MB.

# Now do the same with Dask.

# %%time
df2=dd.read_csv(path)

memory_usage()

# Dask read the file in a fraction of a second and used only about 3 MB of memory. How is that possible?
#
# It's because Dask doesn't load the data into memory. The data is still on the disk. It only reads the data when it needs to perform calculations.
#

# Now, let's calculate the mean for the first 100 columns.

# %%time
df1.iloc[:,:100].mean()

# As you can see pandas does the calculations in a fraction of a second.

# Let's try Dask:

# %%time
avg = df2.iloc[:,100:200].mean()

# Dask is also did the operation very quickly. Let's have a look at the output.

avg

# It's not returning any numbers. What is happening?

# The reason is Dask has not calculated the result yet. It only creates a dependency graph (also called Directed Acyclec Graph - DAG), which is basically how the calculations will take place. We need to execute the graph to see the result.

# %%time
avg.compute()

# Now we can see the results. Also, we can see that this step is the most time consuming step of all. It's because this is where Dask actually goes to disk and reads the data. If you add up the time for all the steps (reading the file and performing calculations) you will see that both take almost the same amount of time to do the operation, with pandas being slightly faster. This shows that Dask is not doing any magic. It's doing the same steps but it's doing it without using as much memory. However, Dask can perform operations in parallel using multiple cpu cores and even multiple machines.

# We mentioned that Dask creates a dependency graph before doing the calculations. We can have a look at this graph and see how it is taking place:

#
To see the graph you need a library called GraphViz. If this library is not installed on your system you will not be able to see the graph.

# Here is one we prepared earlier:
#
# ![](img/dask_graphviz.png)

# +
# Try commenting this out
# # avg.visualize()
# -

# You can also create a progress bar for each computation:

from dask.diagnostics import ProgressBar
avg = df2.iloc[:, 100:200].mean()
task = avg.mean()

with ProgressBar():
    task.compute()

# [Dask dataframe](https://docs.dask.org/en/latest/dataframe-api.html) is very similar to pandas dataframe. It's mostly a subset of the pandas api. Even though the functionalities are limited but you will find many methods from pandas dataframe in Dask as well.
#
# A few operations are missing, and some are especially expensive: those that read the whole array. An example is sort.

task = df2.rolling(window=10).mean().max()

with ProgressBar():
    ma_max = task.compute()

ma_max

task = df2[["label", "pixel100", "pixel300", "pixel500"]].groupby("label").mean()

with ProgressBar():
    result = task.compute()

result

df2

#

#
Exercise

#
# Use Dask dataframe of MNIST (df2) and follow these steps:
#
# 1. Add a new column called `sum` to the dataframe which contains sum of all the pixels
# 2. Use groupby to find the mean value for `sum` for each label
#
#
#
#   → Hints
#
#   * Columns 1 onwards are the pixels. You can access them with `pixels=df2.iloc[:, 1:]`
#   * Instead of `df['sum']=pixels.sum()` try `df['sum']=pixels.sum(axis=1)` because we want to sum along columns, not rows
#   * If the dask output is confusing, try with df1 first
#   * to groupby use `df2.groupby('label').?`, where you replace the `?` with the aggregation operation
#
#
#
#

#

#
#
#     → Solution
#
#
#   ```python
#     # With pandas
#     pixels = df1.loc[:, ['pixel' in c for c in df1.columns]]
#     df1['sum']=pixels.sum(axis=1)
#     task = df1[['label','sum']].groupby('label').mean()
#     print(result)
#
#     # With dask
#     pixels = df2.loc[:, ['pixel' in c for c in df2.columns]]
#     df2['sum']=pixels.sum(axis=1)
#     task = df2[['label','sum']].groupby('label').mean()
#     with ProgressBar():
#         result=task.compute()
#     print(result)
#   ```
#
#
#
#
#

# +
# With pandas
# pixels = df1.loc[:, ['pixel' in c for c in df1.columns]]
pixels = df1.loc[:, 'pixel1':'pixel783']
pixels
df1['sum']=pixels.sum(axis=1)
df1['sum']
# task = df1[['label','sum']].groupby('label').mean()
df1[['label','sum']].groupby('label').mean()

gg = df1[['label','sum']].groupby('label')
list(gg)[0]
# df1.loc[:, 'pixel1':'pixel783'].csum
# print(result)

# # With dask
pixels = df2.loc[:, ['pixel' in c for c in df2.columns]]
pixels
# df2['sum']=pixels.sum(axis=1)
# task = df2[['label','sum']].groupby('label').mean()
# with ProgressBar():
#     result=task.compute()
# print(result)
# -

# ## When to use Dask DataFrame?
#
# Lets visit [the dask page](https://docs.dask.org/en/latest/dataframe.html#common-uses-and-anti-uses) to look at when we should use it
#
# It is harder so only if you dataset is larger than memory.
#
# If fact also consider:
# - a database (if you have lots of structured queries)
# - https://downloadmoreram.com/ ;p
# - dask array

# ## Dask Array
# Dask is not just used to replace pandas. There are also multiple numpy functions which can be replaced by Dask. Dask array is Dask equivalent of a numpy array. By doing so, we can perform the computations in parallel and get the results faster.

from dask import array

big_array = array.random.normal(size=(10000000, 100), chunks=200000)

big_array

# This data takes 8 GB if we wanted to store it in RAM. But Dask only generates the numbers in chunks when it needs them. So at each steps it has to deal with a chunk which is ~160 MB in this case.

task = (big_array * big_array).mean(axis=1)
with ProgressBar():
    res = task.compute()

# We can set the chunk size:

big_array = array.random.normal(size=(10000000, 100), chunks=(2 ** 19, 100))
big_array

task = (big_array * big_array).mean(axis=1)
with ProgressBar():
    res = task.compute()

# We can also apply most of common numpy functions to the array.

task = np.sin(big_array).mean(axis=0)
with ProgressBar():
    res = task.compute()

task

res

# ### Exercise
#
# - Create two Dask random arrays of size 10,000,000-by-100.
# - Find the difference between the two `y = ..`
# - and pass it to `array.linalg.norm` using argument `axis=1`.
# - Calculate the result and create a histogram of it.

from matplotlib.pyplot import hist

#

#
Exercise

#
#   Description:
#
# - Create two Dask random arrays of size 10,000,000-by-100.
# - Find the difference between the two `y = ..`
# - and pass it to `array.linalg.norm` using argument `axis=1`.
# - Calculate the result and create a histogram of it.
#
#
#
#   → Hints
#
#       Replace the question marks `?`
#
# ```python
# a = array.random.normal(size=(10000000, 100), chunks=200000)
# b = array.random.normal(size=(10000000, 100), chunks=200000)
# r = array.linalg.norm(?, axis=1)
# r.?
# ```
#
#
#
#

#

#
#
#     → Solution
#
#
#   ```python
#     x1 = array.random.random(size=(10000000,100))
#     x2 = array.random.random(size=(10000000,100))
#     y = x2-x1
#     d = array.linalg.norm(y,axis=1)
#     with ProgressBar():
#         result = d.compute()
#     plt.hist(result,bins=100);
#   ```
#
#
#
#

# ## Delayed
# Dask delayed is a method for parallelising code where you can't write your code directly as dataframe or array operation. `Dask.delayed` is an easy-to-use tool to quickly parallelise these tasks.

# Consider the following functions. The first one takes an input, waits for one second and returns the value. The second function takes two inputs, waits for one second and returns the sum. We are using these functions to represent tasks that are time consuming.

# +
from time import sleep

def task1(x):
    sleep(1)
    return x

def task2(x, y):
    sleep(1)
    return x + y

# -

# Now, if we pass two values separately into the first function and then pass the results into the second function, we will have the following code:

# %%time
x1 = task1(1)
x2 = task1(2)
y = task2(x1,x2)

# Since each of these functions are taking one second; therefore, the entire block takes three seconds. But the calculation for `x1` is totally independent of the calculation for `x2`. If we were able to do these operation simultaneously we could save time. This is where `Dask.delayed` comes into play. We need to convert the functions into `delayed` functions so Dask can handle parallelisation.

task1_delayed = dask.delayed(task1)
task2_delayed = dask.delayed(task2)

# And now instead of the original function we use the delayed functions:

# %%time
x1 = task1_delayed(1)
x2 = task1_delayed(2)
y = task2_delayed(x1,x2)

# %%time
y.compute()

# And we saved one second! `x1` and `x2` where calculated in parallel, and then `y` was calculated using `x1` and `x2`.

# We can directly create delayed functions using `dask.delayed` decorator.

# +
@dask.delayed
def task1(x):
    sleep(1)
    return x

@dask.delayed
def task2(x, y):
    sleep(1)
    return x + y

# -

# %%time
x1 = task1(1)
x2 = task1(2)
y = task2(x1,x2)
y.compute()

# # Xarray
#
# Xarray is pandas for N-dimensional data. It also has a [dask backend](http://xarray.pydata.org/en/stable/dask.html)

# +
# %matplotlib inline
import numpy as np
import pandas as pd
import xarray as xr
import matplotlib.pyplot as plt

ds = xr.tutorial.open_dataset('rasm').load().chunk(dict(time=10))
ds

# +
# You can use isel instead of iloc. You always need to specify the dimension
ds.isel(time=10)['Tair'].plot.pcolormesh(
        vmin=-30, vmax=30, cmap='Spectral_r',
        add_colorbar=True, extend='both')

ds.isel(time=10)
plt.title('Seasonal Surface Air Temperature')
# -

# You can also resample by date
res = ds.resample(time='A').mean().isel(x=200, y=200)['Tair']
# The result is a dask array
res

# But you can use .compute
res.compute()

# You can see all the datetime methods
print(ds.time.dt)
dir(ds.time.dt)

# These are seasons specified by the months inside
# JJA = Jun, Jul, Aug.
ds.time.dt.season

#

#
Exercise

#
#   1. Look at the output of `ds.time.dt.season`
#   2. Try grouping by season and getting the mean
#   3. Plot each season (use the plotting code from above)
#
#
#
#   → Hints
#
#   * You do a for loop over groups `for season, ds_season in ds.groupby(ds.time.dt.season):`
#   * You need to remove the time dimension, with `.mean('time')`
#   * Use  `mean['Tair'].plot.pcolormesh()` to plot
#
#
#
#

#

#
#
#     → Solution
#
#
#   ```python
#     for season, ds_season in ds.groupby(ds.time.dt.season):
#         mean = ds_season.mean('time')
#         mean['Tair'].plot.pcolormesh(
#             vmin=-30, vmax=30)
#         plt.title(season)
#         plt.show()
#   ```
#
#
#
#

# # Introduction to Numba

# ## What is Numba?
#
# Numba is a **just-in-time**, **type-specializing**, **function compiler** for accelerating **numerically-focused** Python.  That's a long list, so let's break down those terms:
#
#  * **function compiler**: Numba compiles Python functions, not entire applications, and not parts of functions.  Numba does not replace your Python interpreter, but is just another Python module that can turn a function into a (usually) faster function.
#  * **type-specializing**: Numba speeds up your function by generating a specialized implementation for the specific data types you are using.  Python functions are designed to operate on generic data types, which makes them very flexible, but also very slow.  In practice, you only will call a function with a small number of argument types, so Numba will generate a fast implementation for each set of types.
#  * **just-in-time**: Numba translates functions when they are first called.  This ensures the compiler knows what argument types you will be using.  This also allows Numba to be used interactively in a Jupyter notebook just as easily as a traditional application
#  * **numerically-focused**: Currently, Numba is focused on numerical data types, like `int`, `float`, and `complex`.  There is very limited string processing support, and many string use cases are not going to work well on the GPU.  To get best results with Numba, you will likely be using NumPy arrays.
#

# ### First Steps
#
# Let's write our first Numba function and compile it for the **CPU**.  The Numba compiler is typically enabled by applying a *decorator* to a Python function.  Decorators are functions that transform Python functions.  Here we will use the CPU compilation decorator:

#
# The length of the hypotenuse of a triangle is
#
# $r = \sqrt{x^2 + y^2}.$
#
# However, the squares of very large or small values of x and y may exceed the range of machine precision when calculated on a computer, leading to an inaccurate result caused by arithmetic underflow and/or arithmetic overflow.
#
# $ hypot = |x| \sqrt{1 + \left(\tfrac{y}{x}\right)^2}$
#
#

# +
from numba import jit
import math

@jit
def hypot(x, y):
    # Implementation from https://en.wikipedia.org/wiki/Hypot
    x = abs(x)
    y = abs(y)
    t = min(x, y)
    x = max(x, y)
    t = t / x
    return x * math.sqrt(1 + t * t)

# -

# The above code is equivalent to writing:
# ``` python
# def hypot(x, y):
#     x = abs(x);
#     y = abs(y);
#     t = min(x, y);
#     x = max(x, y);
#     t = t / x;
#     return x * math.sqrt(1+t*t)
#
# hypot = jit(hypot)
# ```
# This means that the Numba compiler is just a function you can call whenever you want!
#
# Let's try out our hypotenuse calculation:

# %%time
hypot(3.0, 4.0)

# The first time we call `hypot`, the compiler is triggered and compiles a machine code implementation for float inputs.  Numba also saves the original Python implementation of the function in the `.py_func` attribute, so we can call the original Python code to make sure we get the same answer:

# %%time
hypot.py_func(3.0, 4.0)

# ### Benchmarking
#
# An important part of using Numba is measuring the performance of your new code.  Let's see if we actually sped anything up.  The easiest way to do this in the Jupyter notebook is to use the `%timeit` magic function.  Let's first measure the speed of the original Python:

# %timeit hypot.py_func(3.0, 4.0)

# The `%timeit` magic runs the statement many times to get an accurate estimate of the run time.

# %timeit hypot(3.0, 4.0)

# Numba did a pretty good job with this function.  It's 3x faster than the pure Python version.
#
# Of course, the `hypot` function is already present in the Python module:

# %timeit math.hypot(3.0, 4.0)

# Python's built-in is even faster than Numba!  This is because Numba does introduce some overhead to each function call that is larger than the function call overhead of Python itself.  Extremely fast functions (like the above one) will be hurt by this.
#
# (However, if you call one Numba function from another one, there is very little function overhead, sometimes even zero if the compiler inlines the function into the other one.)

# ### How does Numba work?
#
# The first time we called our Numba-wrapped `hypot` function, the following process was initiated:
#
# ![Numba Flowchart](img/numba_flowchart.png "The compilation process")
#
# We can see the result of type inference by using the `.inspect_types()` method, which prints an annotated version of the source code:

hypot.inspect_types()

# Note that Numba's type names tend to mirror the NumPy type names, so a Python `float` is a `float64` (also called "double precision" in other languages).  Taking a look at the data types can sometimes be important in GPU code because the performance of `float32` and `float64` computations will be very different on CUDA devices.  An accidental upcast can dramatically slow down a function.

# ### When Things Go Wrong
#
# Numba cannot compile all Python code.  Some functions don't have a Numba-translation, and some kinds of Python types can't be efficiently compiled at all (yet).  For example, Numba does not support `FrozenSet` (as of this tutorial):

# +
@jit
def cannot_compile(x):
    return "a" in x

cannot_compile(frozenset(("a", "b", "c")))

# -

# Wait, what happened??  By default, Numba will fall back to a mode, called "object mode," which does not do type-specialization.  Object mode exists to enable other Numba functionality, but in many cases, you want Numba to tell you if type inference fails.  You can force "nopython mode" (the other compilation mode) by passing arguments to the decorator:

# +
@jit(nopython=True)
def cannot_compile(x):
    return "a" in x

try:
    cannot_compile(frozenset(("a", "b", "c")))
except Exception as e:
    logging.exception(e)

# -

# Now we get an exception when Numba tries to compile the function, with an error that says:
# ```
# - argument 0: cannot determine Numba type of <class 'frozenset'>
# ```
# which is the underlying problem. Numba doesn't know about frozenset. There are classes that we use regularly in our code but they might not be defined in Numba. An example of a common class that you cannot use in Numba is pandas data frames.
Now the question is: what does Numba support? Some of the types/classes that are supported by Numba are listed below:
# * Numbers (integers, floats, etc)
# * Numpy arrays
# * Strings
# * Lists and tuples (note that a list/tuple of numbers or strings is supported but a list of lists is not)

# So, if we want the last example to be compiled successfully by Numba jit, we need to use a tuple or a list.

# +
@jit(nopython=True)
def can_compile(x):
    return "a" in x

can_compile(("a", "b", "c"))
# -

# ### Exercise
# Gregory–Leibniz infinite series converges to $\pi$:
# $$\pi = \frac{4}{1} - \frac{4}{3} + \frac{4}{5} - \frac{4}{7} + \frac{4}{9} - \frac{4}{11} + \frac{4}{13} - \cdots$$
#
# Write a Numba function which calculates the sum of first $n$ terms in this series. Then test its speed agains normal Python function for $ n = 1000000$.

# +
# Code Here
# -

# Solution
#
# ```Python
#     @jit
#     def gl_pi(n):
#         pi = 0
#         for i in range(n):
#             if i%2 ==0:
#                 pi += 4/(2*i+1)
#             else:
#                 pi -= 4/(2*i+1)
#         return pi
# ```
#
# Numba function speed test:
# ```Python
#     %timeit gl_pi(1000000)
# ```
#
# Normal Python function speed test:
# ```Python
#     %timeit gl_pi.py_func(1000000)
# ```
#
#

# # References
# The following sources where used for creation of this notebook:
# - https://github.com/NCAR/ncar-python-tutorial
# - https://github.com/stevesimmons/pydata-ams2017-pandas-and-dask-from-the-inside
# - https://github.com/numba/euroscipy2019-numba

# # Further Reading
# - [Dask documentation](https://docs.dask.org/en/latest/)
# - [Why Dask?](https://docs.dask.org/en/latest/why.html)
# - [Distributed Machine Learning with Python and Dask](https://towardsdatascience.com/distributed-machine-learning-with-python-and-dask-2d6bae91a726)
# - [Speeding up your Algorithms — Dask](https://towardsdatascience.com/speeding-up-your-algorithms-part-4-dask-7c6ed79994ef)
# - [Speeding Up your Algorithms — Numba](https://towardsdatascience.com/speed-up-your-algorithms-part-2-numba-293e554c5cc1)


import numpy as np
import pygame as pg
import argparse
from os import path

class Piano():
	def __init__(self, samplerate=44100, amplitude=8192, decay=3, note_value=2.5):
		"""
		Initialize pygame mixer, read file with keybindings, generate sound waves
		"""
		pg.mixer.init(samplerate, channels=2)

		self.SAMPLERATE = samplerate
		self.AMPLITUDE = amplitude
		self.DECAY = decay
		self.NOTE_VALUE = note_value

		self.notes = [
					'C3', 'C#3', 'D3', 'D#3', 'E3', 'F3', 'F#3', 'G3', 'G#3', 'A3', 'A#3', 'B3',
					'C4', 'C#4', 'D4', 'D#4', 'E4', 'F4', 'F#4', 'G4', 'G#4', 'A4', 'A#4', 'B4',
					'C5', 'C#5', 'D5', 'D#5', 'E5', 'F5', 'F#5', 'G5', 'G#5', 'A5', 'A#5', 'B5'
				]

		base_freq = np.array([[261.63, 523.25, 788.35, 1051.027, 2118.91]]).T
		ampl = np.array([[amplitude, amplitude/2, amplitude/4, amplitude/8, amplitude/16]]).T
		sounds = list()
		for n in range(-12, 24):
			freq = base_freq * 2 ** (n / 12)
			data = np.linspace(0, self.NOTE_VALUE, int(self.SAMPLERATE*self.NOTE_VALUE))
			data = (ampl * np.sin(2 * np.pi * freq * data) / np.exp(self.DECAY*data))
			data = np.sum(data, axis=0).astype(np.int16)
			data = data.reshape(-1, 1)
			data = np.concatenate([data, data], axis=1)
			sounds.append(pg.sndarray.make_sound(data))
		with open(path.join('misc', 'pianokeys.mapping')) as f:
			self.keys = f.read().split('\n')
		self.key_note = dict(zip(self.keys, sounds))
		self.pressed = {key: False for key in self.keys}

	def play_note(self, key):
		self.key_note[key].stop()
		self.key_note[key].play()

	def start_playing(self, notes_file=''):
		pg.init()

		icon = pg.image.load(path.join('misc', 'icon.png'))
		pg.display.set_icon(icon)
		pg.display.set_caption("Virtual Piano")

		if not notes_file:
			SIZE = (973, 282)
			self.notes_file = notes_file
		else:
			SIZE = (973, 482)
			pg.font.init()

		self.screen = pg.display.set_mode(SIZE)
		self.screen.fill((20, 11, 10))
		bg = pg.image.load(path.join('misc', "keys.png")).convert()
		self.screen.blit(bg, (0, SIZE[1]-282))

		if notes_file:
			self.WIDTH = 400
			self.HEIGHT = 150
			self.LEFT = (SIZE[0]-self.WIDTH)//2
			self.TOP = (SIZE[1]-282-self.HEIGHT)//2

			self.RECT = pg.Rect(self.LEFT, self.TOP, self.WIDTH, self.HEIGHT)
			pg.draw.rect(self.screen, (245, 241, 230), self.RECT)

			self.CURRENT_COLOR = (0, 0, 0)
			self.OTHER_COLOR = (122, 122, 122)
			self.FONT = pg.font.Font(None, 22)

		pg.display.flip()

		if not notes_file:
			while True:
				event = pg.event.wait()
				if event.type in [pg.KEYDOWN, pg.KEYUP]:
					if event.key == pg.K_ESCAPE:
						pg.quit()
						quit()
					current_key = pg.key.name(event.key)

				if event.type == pg.KEYDOWN and current_key in self.key_note.keys():
					self.play_note(current_key)
		else:
			sheet = self.load_notes(notes_file)
			self.SPACE_BETWEEN = self.WIDTH // len(sheet[0]) - 7

			current_idx = 0
			self.display_notes(sheet, current_idx)

			while True:
				event = pg.event.wait()
				if event.type in [pg.KEYDOWN, pg.KEYUP]:
					if event.key == pg.K_ESCAPE:
						pg.quit()
						quit()
					current_key = pg.key.name(event.key)

				if event.type == pg.KEYDOWN and current_key in self.key_note.keys():
					self.play_note(current_key)
					self.pressed[current_key] = True

				if event.type == pg.KEYUP and current_key in self.key_note.keys():
					self.pressed[current_key] = False
					if not any(self.pressed.values()):
						current_idx = self.display_notes(sheet, current_idx + 1)

	def load_notes(self, notes_file):
		parsed = list()
		note_key = dict(zip(self.notes, self.keys))
		with open(notes_file) as file:
			lines = file.read().split('\n')[:-1]
			for line in lines:
				parsed_line = list()
				for chord in line.split():
					res = ''
					for note in chord.split('.'):
						res += note_key[note]
					parsed_line.append(res)
				parsed.append(parsed_line)
		return parsed

	def display_notes(self, notes, current):
		pg.draw.rect(self.screen, (245, 241, 230), self.RECT)
		if len(notes):
			if current >= len(notes[0]):
				current = 0
				notes.pop(0)
			if not len(notes):
				pg.display.update(self.RECT)
				return 0

			left = self.LEFT + self.SPACE_BETWEEN
			for i in range(len(notes[0])):
				color = self.OTHER_COLOR
				if current == i: color = self.CURRENT_COLOR
				note = self.FONT.render(notes[0][i], True, color)
				self.screen.blit(note, (left, self.TOP + self.HEIGHT//3))
				left += self.SPACE_BETWEEN + 5*len(notes[0][i])

			if len(notes) > 1:
				left = self.LEFT + self.SPACE_BETWEEN
				for i in range(len(notes[1])):
					note = self.FONT.render(notes[1][i], True, self.OTHER_COLOR)
					self.screen.blit(note, (left, self.TOP + self.HEIGHT//3*2))
					left += self.SPACE_BETWEEN + 5*len(notes[1][i])

			pg.display.update(self.RECT)

			return current
		else:
			pg.display.update(self.RECT)
			return 0

if __name__ == '__main__':
	parser = argparse.ArgumentParser(description='Virtual piano with Python')

	parser.add_argument(
		'-n', '--notes',
        default='',
        help='(Optional) File with notes to play')

	args = parser.parse_args()

	piano = Piano()
	piano.start_playing(args.notes)


# -*- coding: utf-8 -*-
"""
Spyder Editor

This is a temporary script file.
"""
"""-------------------------------------------------------------------------------------------------------------------------------------------------------------- """

"""
자료구조 : 1. sequence 자료구조 (순서있음)
          2. 딕셔너리, 세트 (순서없음)

인덱싱 : 리스트에서 하나의 요소를 인덱스 연산자를 통하여 참조(접근)하는 것

슬라이싱 : 리스트 안에서 범위를 지정하여서 원하는 요소들을 선택하는 연산

리스트 : 여러 개의 데이터가 저장되어 있는 자료구조
리스트가 필요한 이유 : 여러 개의 데이터가 저장되어 있는 자료구조
"""

# 문자열 인덱싱 & 슬라이싱

text = "IT Will is power."
print(text[:-2]) # IT Will is powe
print(text[:8], text[-1])

# 인덱싱
flist = ["apple", "banana", "tomato", "peach", "pear" ]
print(flist[0], flist[3], flist[-1])

# 슬라이싱
a = [0,1,4,9,16,25,36,49]
a[3:6]

# 리스트 (append로 추가해서 리스트를 만들어보는 것)

scores = [ ]
for i in range(10):
    scores.append(int(input("성적을 입력하시오:")))
print(scores)

scores[0] = 80

scores[i] = 10;
scores[i+2] = 20;

# 리스트의 요소갯수만큼 리스트가 반복되어 출력
for element in scores:
    print(scores)

# 복잡한 리스트
list1 = [12,"dog",180.14] # 혼합자료형
list2 = [["Seoul", 10], ["Paris", 12], ["London", 50]] # 내장 리스트

# 리스트 기초연산
marvel_heroes = [ "스파이더맨", "헐크", "아이언맨" ]
dc_heroes = [ "슈퍼맨", "배트맨", "원더우먼" ]

heros = marvel_heroes + dc_heroes
heros
# 리스트에 곱하기
values = [1,2,3]*3
values # [1, 2, 3, 1, 2, 3, 1, 2, 3]
len(values)

# 요소추가하기
developteam = []
developteam.append("이유현")
developteam.append("성민승")
developteam.append("김동완")

developteam

# 리스트 안의 내용 조회
if "이유현" in developteam:
    print("내부인원")

# 리스트 인덱스 확인
developteam.index("이유현") # 0
developteam.index("김동완") # 2

# 리스트 최소값 최대값
values = [ 100, 20, 31, 45, 15, 6, 7, 8, 9, 10 ]
min(values)
max(values)

# 리스트에서 sort 쓰기 정렬~~~
values.sort()
value2=sorted(values)
print(value2)

# 리스트 내의 조건문
list1 = [3,4,5]
list2 = [x*2 for x in list1]
print(list2)

# 2차원 리스트
s = [
[ 1, 2, 3, 4, 5 ] ,
[ 6, 7, 8, 9, 10 ],
[11, 12, 13, 14, 15 ]
]

print(s)

# 동적으로 2차원 리스트 생성

rows = 3
cols = 5

s = []
for row in range(rows):
    s += [[3]*cols]

print("s=",s) # s= [[3, 3, 3, 3, 3], [3, 3, 3, 3, 3], [3, 3, 3, 3, 3]]

rows = len(s)
cols = len(s[0])
cols
for r in range(rows):
    for c in range(cols):
        print(s[r][c], end=",")
    print()

"""
    tuple!!
    튜플은 변경될 수 없는 리스트 + 순서가 없다!!

    tuple('listname')
    : 리스트를 튜플로 변경한다.
"""
# tuple을 변경하려고 해보자

t1 = (1,2,3,4,5);
t2 = (1,2,3,4,5);

t1[0] =100; # TypeError: 'tuple' object does not support item assignment

# 튜플 대입 연산
student1 = ("철수",19,"CS")
(name,age,major) = student1
name # '철수'

"""
    set!!
    세트는 중복되지 않은 항목들이 모인것 + 순서가 없다!!

"""

numbers = {1,2,2,3,3,3,4}

numbers # {1, 2, 3, 4}

# 요소 추가
numbers.add(5)

"""

    dictionary
    딕셔너리는 키(key)와 값(value)의 쌍을 저장할 수 있는 객체

"""

# 형태
dictionary = {'name':'이유현','phone':'01091597160','score':100}
dictionary['score']
# 추가하기

dictionary['speed'] = '1000'
print(dictionary)

# 항목 순회하며 출력하기.

for item in dictionary.items():
    print(item)
"""
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
    plot 문제로 정리하기
    1. matplotlib
        1) 한글 및 음수 부호 지원
        2) 기본차트 시각화 - 기본 선 스타일과 색상, x축 y축 스타일&색상, color와 marker 이용
        3) 산점도, 히스토그램, 상자그래프
        3) 이산형 변수 시각화 - 가로막대 그래프, 세로막대 차트, 원차트
        4) subplot 차트
        5) 시계열차트
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
"""

data3= np.random.randn(50) # 난수
data4= np.random.randn(50).cumsum() # 난수

chart.plot(data3, color='r', label='step',
drawstyle="steps-post")

chart.plot(data4, color='g', label='line')
plt.legend(loc='best')
plt.ylabel('y label')
plt.xlabel('x label')
plt.title('chart title')
plt.show()

import matplotlib.pyplot as plt
import seaborn as sn

# 문3) seaborn의  titanic 데이터셋을 이용하여 다음과 같이 단계별로 시각화하시오.
titanic = sn.load_dataset('titanic')
print(titanic.info())

#  <단계1> 'total_bill','tip','sex','size' 칼럼으로 서브셋 만들기
titanic_df = titanic[['survived','pclass', 'age','fare']]
print(titanic_df.info())
#sn.pairplot(data=DataFrame, hue='집단변수', kind='scatter')

# <단계2> 성별(sex) 칼럼을 집단변수로 산점도행렬 시각화
sn.pairplot(data=titanic_df, hue='survived')
plt.show()

# <단계3> 산점도행렬의 시각화 결과 해설하기
'''
pclass : 3등석 일수록 사망비율 매우 높음
pclass vs fare : 1등석 일수록 고 요금
age : 25~50세 사이에서 가장 높은 빈도, 사망과 생존 비율 비슷
age vs fare : 대체적으로 나이가 많고, 요금이 낮은 경우 사망 비율 높음
fare : 비용이 저렴한 경우가 상대적으로 많은 분포
fare vs age : 대체적으로 요금이 낮고, 나이가 50대 이상인 경우 사망 비율 높음
'''

# survived vs age
# 연령대 생존비율 : 20~40
titanic[titanic['survived'] == 1].age.plot(kind = 'hist', color = 'blue')
# 연령대 사망비율 : 20~40
titanic[titanic['survived'] == 0].age.plot(kind = 'hist', color = 'green')

# 문4) seaborn의 tips 데이터셋을 이용하여 다음과 같이 단계별로 시각화하시오.
tips = sn.load_dataset('tips')
print(tips.info())

# <단계1> 'total_bill','tip','sex','size' 칼럼으로 서브셋 만들기
tips_df = tips[['total_bill','tip','sex','size']]

# <단계2> 성별(sex) 칼럼을 집단변수로 산점도행렬 시각화
sn.pairplot(data=tips_df, hue='sex')
plt.show()

# <단계3> 산점도행렬의 시각화 결과 해설하기
"""
total_bill : 총금액 15~20 사이가 가장 높은 빈도, 금액이 클 수록 남자 지불
total_bill vs tip  : 대체적으로 비례관계, 총금액과 팁이 많은 경우 남자 지불
tip : 팁은 1~5 사이가 가장 높은 빈도, 팁 금액이 클 수록 남자 지불
total_bill vs size : 행사규모가 작은 경우 여성 지불
size : 행사규모 2가 가장 높은 빈도, 특히 4일때 남자 지불
size vs total_bill : 대체적으로 비례관계, 규모가 큰 경우 총금액이 많음

"""
import pandas as pd # object
import numpy as np # dataset
import matplotlib.pyplot as plt # plt.show()

# 1. 기본 차트 시각화
ser = pd.Series(np.random.randn(10)) # 1d
print(ser)

# 1차원 객체 : 기본차트 - 선 그래프
ser.plot(color='g')
plt.show()

# 2차원 객체
df = pd.DataFrame(np.random.randn(10, 4),
                  columns=('one','two','three','fore'))

print(df)

# 기본차트 : 선 그래프
df.plot()
plt.show()

# 막대차트 : 세로
df.plot(kind='bar', title = 'bar chart')
plt.show()

# 막대차트 : 가로
df.plot(kind='barh', title = 'bar chart')
plt.show()

# 막대차트 : 가로, 누적형
df.plot(kind='barh', title = 'barh chart', stacked=True)
plt.show()

# 2. dataset 이용
import os

os.chdir('C:/ITWILL/4_Python-II/data')
tips = pd.read_csv('tips.csv')
print(tips.info())

# 교차분할표 : 집단변수 이용
# 요일(day):행 vs 규모(size):열
tips['day'].unique() # ['Sun', 'Sat', 'Thur', 'Fri']
tips['size'].unique()# [2, 3, 4, 1, 6, 5]

tab = pd.crosstab(index=tips['day'], columns=tips['size'])
print(tab)

# 테이블 정보
tab.shape # (4, 6)
tab.index # 행 이름
tab.columns # 열 이름

#tab.index = 수정 이름
type(tab) # pandas.core.frame.DataFrame

help(tab.plot)

# size : 1, 6 제외 -> subset
#obj.loc[행, 열]
new_tab = tab.loc[:, 2:5]
print(new_tab)

new_tab.plot(kind='barh', stacked=True,
         title = 'day and size')
plt.show()

"""
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
plot 예제 끝~~
"""

"""
---------------------------------------------------------------------------------------------------------------------------------------------------------------------

1. Group by : 머 대강 칼럼들을 특정 변수의 조건에 맞게 그룹화 시키는거다
2. apply :
3. Pivot table ---> df.pivot(index='x',columns='y',values='z')
        - 좌측 index(행)를 어떤 칼럼으로 설정할지
        - 칼럼(열)쪽을 y칼럼으로 세우고
        - 안에 채울 values 값 z로 설정

---------------------------------------------------------------------------------------------------------------------------------------------------------------------
"""

"""

NUMPY!!!!!!!!!!!!!
    1. 배열 생성 array()
    - arange() : 배열 객체 반환
    - linspace() : 시작점과 끝점을 균일 간격으로 나눈 점들을 생성
    - reshape() : 행수와 열수를 조절
    2. 특수 행렬 생성 zeros() : 0으로 채워진 배열, ones() : 1로 채워진 배열 ,eye() : 항등행렬

    3. 난수 생성  numpy.random.normal(size = 개수)
    4. 산술연산

"""

"""
# Series Pandas의 Series는 1차원 데이터를 다루는 데 효과적인 자료구조
values 속성을 호출하면 데이터의 배열 원소가 리턴
index 속성을 호출하면 인덱스의 정보가 리턴
각각의 데이터는 [인덱스]를 이용해서 접근이 가능
numpy 함수 사용 가능
dict 객체로 생성 가능
"""
# code 1

from pandas import Series, DataFrame
import numpy as np
import pandas as pd

price = Series([4000, 3000, 3500, 2000])
print(price)
print(price.index)
print(price.values)
print('====================')
fruit = Series([4000, 3000, 3500, 2000],
index=['apple', 'mellon','orange', 'kiwi'])
print(fruit)
print(fruit[0]) # 순번 이용 데이터 접근
print(fruit['apple']) # 인덱스 이용 데이터 접근
print(fruit[fruit>3000]) # 부울리언 식
print("=================")

# code 2
from pandas import Series, DataFrame
import numpy as np
import pandas as pd

good1 = Series([4000,3500,None,2000],index = ['apple','mango','orange','kiwi'])

good2 = Series([3000,3000,3500,2000],index = ['apple','mango','orange','kiwi'])

print(pd.isnull(good1)) # NaN값 검출 none이면 True를 반환
print (good1+good2)

# DataFrame

"""
# DataFrame은 행과 열로 구성된 2차원 데이터를 다루는 데 효과적인 자료구조
일반적으로 딕셔너리 활용해서 생성
입력가능한 데이터
1. 2차원 ndarray
2. 리스트, 튜플,dict, Series의 dict
3. dict, Series의 list
4. 리스트, 튜플의 리스트
5. 칼럼 뽑는 방법
    1) data.iloc[[행]],[열]  # 행은 index

"""
# code 1
from pandas import Series, DataFrame

items = {'code': [1,2,3,4,5,6],
'name': ['apple','watermelon','oriental melon', 'banana', 'lemon', 'mango'],
'manufacture': ['korea', 'korea', 'korea','philippines','korea', 'taiwan'],
'price':[1500, 15000,1000,500,1500,700]}

data = DataFrame(items)
print(data)

# 특정 컬럼만 뽑기
data1 = DataFrame(items, columns = ['code', 'price'])
print(data1)
print(data.loc[0]) # 0 행 출력 가로로 나옴
print(data.loc[:0]) # 0 행 출력 세로로 나옴
print(data.loc[[2],['name']])

# 데이터 프레임 변경해보기
data.index = np.arange(1,7,1) # 1~6까지 인덱스 설정
data.columns
data = data.reindex(['1','2','3','4','5','7'],columns = ['code', 'name', 'manufacture', 'price'])
print(data.index)
print(data);

"""-------------------------------------------------------------------------------------------------------------------------------------------------------------- """
# 정규분포 생성 알고리즘
""" 정규분포란? 가우시안 정규 분포 :
    자연 현상에서 나타나는 숫자를 확률 모형으로 모형화할 때 가장 많이 사용되는 모형
    표준편차 1배안에 전체 데이터의 약 70% 이상이 몰려있고
    1.96배 안에 95% 이상이 분포된 경우
    """
from pandas import Series, DataFrame
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import font_manager, rc
from scipy import stats
import scipy as sp
font_name = font_manager.FontProperties(fname="c:/Windows/Fonts/malgun.ttf").get_name()
rc('font', family=font_name)
mu = 0
std = 1
rv = sp.stats.norm(mu, std)
xx = np.linspace(-5, 5, 100)
plt.plot(xx, rv.pdf(xx))
plt.ylabel("확률")
plt.title("정규분포곡선")
plt.show()
x = rv.rvs(100) # rvs 메서드로 시뮬레이션해 샘플을 얻는것

print(x)

# 검정통계 , 유의확률(p-value)
"""
    검정(testing)은  데이터  뒤에  숨어있는  확률  변수의  분포와  모수에  대한  가설의  진 위를  정량적(quantitatively)으로  증명하는  작업

    실제 모집단에서 표본 몇 십 개 또는 몇 백 개를 추출해서 그것의 분산(표본분산)이나 평균(표본평균)을 사용해야 하는 경우가 대 부분입니다.
   표본수가  크지  않을  때  표본분산(표본표준편차)을  사용한  테스트는  t-분포를  이용 한다고  해서  T-test라고  합니다.
   가설  증명  즉  검정의  기본적인  논리는  다음과  같습니다.
   만약  가설이  맞다면  즉, 모수  값이  특정한  조건을  만족한다면  해당  확률  변수로부터  만 들어진  표본(sample) 데이터들은  어떤  규칙을  따르게  된다.
    해당 규칙에 따라 표본 데이터 집합에서 어떤 숫자를 계산하면 계산된 숫자는 특정한 확률 분포를 따르게 된다. 이 숫자를 검정 통계치(test statistics)라고 하며 확률 분포를 검정 통계 분포(test statistics distribution)라고 한다. 검정 통계 분포의 종류 및 모수의 값은 처음에 정한 가설에 의해 결정된다. 이렇게 검정 통계 분포를 결정하는 최초의 가 설을 귀무 가설(Null hypothesis)이라고 한다.
   데이터에  의해서  실제로  계산된  숫자, 즉, 검정  통계치가  해당  검정  통계  분포에서  나올 수  있는  확률을  계산한다. 이를  유의  확률(p-value)라고  한다.
   만약  유의  확률이  미리  정한  특정한  기준  값보다  작은  경우를  생각하자. 이  기준  값을 유의  수준(significance level)이라고  하는  데  보통  1% 혹은  5% 정도의  작은  값을  지정한 다. 유의  확률이  유의  수준으로  정한  값(예  1%)보다도  작다는  말은  해당  검정  통계  분포 에서  이  검정  통계치가  나올  수  있는  확률이  아주  작다는  의미이므로  가장  근본이  되는 가설  즉, 귀무  가설이  틀렸다는  의미이다. 따라서  이  경우에는  귀무  가설을  기각(reject) 한다.
   만약 유의 확률이 유의 수준보다 크다면 해당 검정 통계 분포에서 이 검정 통계치가 나 오는 것이 불가능하지만은 않다는 의미이므로 귀무 가설을 기각할 수 없다. 따라서 이 경우에는 귀무 가설을 채택(accept)한다.

    linspace 함수는 numpy모듈의 1차원 배열 만드는 함수
    x = np.linspace(start,stop,num) # num은 요소의 개수 그 사이에 몇개를 만들것인가.
    np.random.randit : 균일 분포의 정수 난수 1개 생서
    np.random.rand : 0부터 1사이의 난수 matrix array 생성
    np.random.randn : 가우시안 표준 정규 분포에서 난수 matrix array 생성
    plt.fill_between() : 두 수평 방향의 곡선 사이를 채웁니다.
    plt.fill_betweenx() : 두 수평 방향의 곡선 사이를 채웁니다.
    plt.fill() : 다각형 영역을 채웁니다.

    파이썬 플랏 상세 설명 : https://blog.naver.com/nach3012/222419686483

"""

import numpy as np
import matplotlib.pyplot as plt
from matplotlib import font_manager, rc
from scipy import stats
import scipy as sp

font_name = font_manager.FontProperties(fname="c:/Windows/Fonts/malgun.ttf").get_name()
rc('font', family=font_name)

xx1 = np.linspace(-4, 4, 100)
xx2 = np.linspace(-4, -2, 100)
xx3 = np.linspace(2, 4, 100)

plt.fill_between(xx1, sp.stats.norm.pdf(xx1), facecolor='green', alpha=0.1)
plt.fill_between(xx2, sp.stats.norm.pdf(xx2), facecolor='blue', alpha=0.35)
plt.fill_between(xx3, sp.stats.norm.pdf(xx3), facecolor='blue', alpha=0.35)
plt.text(-3, 0.1, "p-value=%5.3f" % (2*sp.stats.norm.cdf(-2)), horizontalalignment='center')
plt.title("유의확률:0.046")
plt.show()


%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt


# -*- coding: utf-8 -*-
"""
Created on Wed Jan 22 21:08:13 2020

@author: ZHOU_YuZHAO
"""

import numpy as np
import csv

class Read():
    def readFromFile(self,filename):
        data = []
        with open(filename, "r") as csvfile:
            # The data is returned with each row of data as a list
            reader = csv.reader(csvfile)
            for row in reader:
            # Output the row list
                line = []
                for x in row:
                    line.append(int(x))
                data.append(line)

        return np.array(data)


#
#  File:
#    PyEarthScience_read_ASCII_and_plot_counts_per_country.py
#
#  Synopsis:
#    Read ASCII data and draw countries of global map with colors depending on their values.
#
#  Category:
#    map plot
#    labelbar
#    read ASCII file
#    text
#
#  Based on DKRZ's NCL example:
#    NCL_read_ASCII_and_plot_counts_per_country.ncl
#
#  Author:
#    Karin Meier-Fleischer
#
#  Date of initial publication:
#    December, 2018
#
#  Description:
#    Read ASCII data and draw countries of global map with colors depending on their values.
#    Add a labelbar to the plot.
#
#  Input file:
#    country_gesamt.txt
#
#  Effects illustrated:
#    o  Create a global map
#    o  Using area specifiers
#    o  Define color map
#    o  Read ASCII file
#    o  Create a labelbar
#    o  Add text
#
#  Output:
#     Two visualizations are produced.
#
'''
  PyNGL Example: 	PyEarthScience_read_ASCII_and_plot_counts_per_country.py

  -  Create a global map
  -  Using area specifiers
  -  Define color map
  -  Read ASCII file
  -  Create a labelbar
  -  Add text

'''
from __future__ import print_function
from io import StringIO
import numpy as np
import Ngl,Nio

#-----------------------------------------------------------------------------------
#-- read ASCII input file  (country; value)
#-----------------------------------------------------------------------------------
fin     = open('country_gesamt.txt', newline='')
input   = fin.read()
rows    = input.strip('\n')

s       = StringIO(rows)        #-- in-memory stream for text
data    = np.genfromtxt(s, dtype=['U30','i8'], names=['country','value'], delimiter=';')
                                #-- genfromtxt 'U' must be used for strings in Python3
states  = list(data['country']) #-- in contrast to NCL it must be a list
values  = data['value'][:]      #-- data array

#-----------------------------------------------------------------------------------
#-- define levels and labels
#-----------------------------------------------------------------------------------
levels = [1,2,5,10,50,100,200,500,1000,2000]                             #-- value levels
labels = ["1","2",">5",">10",">50",">100",">200",">500",">1000",">2000"] #-- labelbar labels

#-----------------------------------------------------------------------------------
#-- define color map (index 0/1: foreground/background)
#-----------------------------------------------------------------------------------
cmap = np.array([[1.0,      1.0,      1.0], \
                 [0.0,      0.0,      0.0], \
                 [0.997785, 0.999139, 0.846059], \
                 [0.910127, 0.964937, 0.695640], \
                 [0.769320, 0.909419, 0.706959], \
                 [0.521292, 0.812964, 0.731073], \
                 [0.304483, 0.732118, 0.761430], \
                 [0.141961, 0.597647, 0.756078], \
                 [0.122107, 0.483137, 0.712711], \
                 [0.131949, 0.382745, 0.665467], \
                 [0.138408, 0.297578, 0.624990], \
                 [0.031373, 0.113725, 0.345098]],'f')

colors  = np.arange(2,len(cmap)+1,1)                #-- array for color indices
nlevels = len(levels)                               #-- number of levels
icols   = np.ones(len(values),int)                  #-- assign array for color indices

#-- set the color indices array
for i in range(0,len(values)):
   if(values[i] == levels[0]):
      icols[i] = 0

   if(values[i] > levels[nlevels-1]):
      icols[i] = len(colors)

   for j in range(0,nlevels-1):
      if(values[i] > levels[j] and values[i] <= levels[j+1]):
         icols[i] = colors[j]

   print("State: %2.2i   IPs:  %4.2i  %s" % (i,values[i],data['country'][i]))

#-----------------------------------------------------------------------------------
#-- open a workstation and set workstation resources
#-----------------------------------------------------------------------------------
wkres                 = Ngl.Resources()
wkres.wkWidth         = 1024                        #-- wk width
wkres.wkHeight        = 1024                        #-- wk height
wks = Ngl.open_wks("png","plot_counts_per_country_map_blue",wkres)

#-- set new color map (overwrite default colormap)
rlist            = Ngl.Resources()
rlist.wkColorMap = cmap
Ngl.set_values(wks,rlist)

#-----------------------------------------------------------------------------------
#-- set resources
#-----------------------------------------------------------------------------------
res                       =  Ngl.Resources()
res.nglMaximize           =  True                   #-- maximize plot
res.nglFrame              =  False                  #-- don't advance the frame yet

res.vpXF                  =  0.01                   #-- x-position
res.vpYF                  =  0.95                   #-- y-position
res.vpWidthF              =  0.98                   #-- width
res.vpHeightF             =  0.95                   #-- height

res.pmTickMarkDisplayMode = "Always"                #-- turn on map tickmarks

res.mpDataSetName         = "Earth..4"              #-- new database
res.mpDataBaseVersion     = "MediumRes"             #-- Medium resolution database
res.mpOutlineOn           =  True                   #-- turn on map outlines
res.mpFillOn              =  True                   #-- turn on map fill
res.mpOutlineBoundarySets = "National"              #-- draw only national bounds
res.mpOceanFillColor      = "white"                 #-- set ocean fill color to white
res.mpLandFillColor       = "white"                 #-- set land fill color to white
res.mpInlandWaterFillColor= "white"                 #-- set inland water fill color to white
res.mpFillAreaSpecifiers  =  states                 #-- fill listed states
res.mpSpecifiedFillColors =  icols                  #-- use generated color array
res.mpMinLatF             = -60                     #-- don't plot Antarctica

res.tmXBLabelFontHeightF  =  0.012                  #-- change XB label font size
res.tmYLLabelFontHeightF  =  0.012                  #-- change YL label font size
res.tmXBMajorLengthF      =  0.008                  #-- change XB the tickmark length
res.tmYLMajorLengthF      =  0.008                  #-- change YL the tickmark length

res.tiMainString          = "Counts per country"    #-- title string
res.tiMainFont            = "helvetica"             #-- title string font
res.tiMainFontHeightF     =  0.025                  #-- set title string font size

#-- create the map
map = Ngl.map(wks,res)

#-----------------------------------------------------------------------------------
#-- add custom label bar to the plot
#-----------------------------------------------------------------------------------
vpx  = Ngl.get_float(map,"vpXF")                    #-- retrieve viewport x-position
vpy  = Ngl.get_float(map,"vpYF")                    #-- retrieve viewport y-position
vpw  = Ngl.get_float(map,"vpWidthF")                #-- retrieve viewport width
vph  = Ngl.get_float(map,"vpHeightF")               #-- retrieve viewport height

lbx, lby = vpx, vpy-vph-0.04

lbres                    =  Ngl.Resources()
lbres.vpWidthF           =  vpw                     #-- width of labelbar
lbres.vpHeightF          =  0.08                    #-- height of labelbar
lbres.lbOrientation      = "horizontal"             #-- labelbar orientation
lbres.lbLabelFontHeightF =  0.012                   #-- labelbar label font size
lbres.lbAutoManage       =  False                   #-- we control label bar
lbres.lbFillColors       =  colors                  #-- box fill colors
lbres.lbPerimOn          =  False                   #-- turn off labelbar perimeter
lbres.lbMonoFillPattern  =  True                    #-- turn on solid pattern
lbres.lbLabelAlignment   = "BoxCenters"             #-- write labels below box edges

#-- create the labelbar
pid = Ngl.labelbar_ndc(wks, nlevels, labels, lbx, lby, lbres)

#-----------------------------------------------------------------------------------
#-- add x-axis title and copyright string
#-----------------------------------------------------------------------------------
txres               =  Ngl.Resources()
txres.txJust        = "CenterCenter"
txres.txFontHeightF =  0.014

Ngl.text_ndc(wks,"ESGF users",0.5,0.14,txres)

txres.txJust        = "CenterRight"
txres.txFontHeightF =  0.010
Ngl.text_ndc(wks,"~F35~c ~F21~~N~DKRZ",vpx+vpw,0.10,txres)

#-----------------------------------------------------------------------------------
#-- advance the frame
#-----------------------------------------------------------------------------------
Ngl.frame(wks)