![Callysto.ca Banner](https://github.com/callysto/curriculum-notebooks/blob/master/callysto-notebook-banner-top.jpg?raw=true)

To run this notebook press the >> Button and confirm "Restart and Run all".

![](./images/RunB.png)

from IPython.display import HTML

HTML('''
The raw code for this IPython notebook is by default hidden for easier reading.
To toggle on/off the raw code, click here.''')

%%html

# Modules

import string
import numpy as np
import pandas as pd
import qgrid as q
import matplotlib.pyplot as plt

# Widgets & Display modules, etc..

from ipywidgets import widgets as w
from ipywidgets import Button, Layout
from IPython.display import display, Javascript, Markdown

# grid features for interactive grids

grid_features = { 'fullWidthRows': True,
                  'syncColumnCellResize': True,
                  'forceFitColumns': True,
                  'rowHeight': 40,
                  'enableColumnReorder': True,
                  'enableTextSelectionOnCells': True,
                  'editable': True,
                  'filterable': False,
                  'sortable': False,
                  'highlightSelectedRow': True}

from ipywidgets import Button , Layout , interact,widgets
from IPython.display import Javascript, display

# Function: executes previous cell on button widget click event and hides achievement indicators message

def run_current(ev):

    display(Javascript('IPython.notebook.execute_cell_range(IPython.notebook.get_selected_index()+0,IPython.notebook.get_selected_index()+1)'))

# Counter for toggling achievement indicator on/off

button_ctr = 0

# Achievement Indicators

line_1 = "#### Achievement Indicators"
line_2 = "**General Outcome:**"
line_3 = "* The general outcome of this notebook is to use experimental or theoretical probabilities to represent and solve problems involving uncertainty."
line_4 = "**Specific Outcome 4:**"
line_5 = "* Express probabilities as ratios, fractions and percents."
line_6 = "**Specific Outcome 5:**"
line_7 = "* Identify the sample space (where the combined sample space has 36 or fewer elements) for a probability experiment involving two independent events.*"
line_8 = "**Specific Outcome 6:**"
line_9 = "* Conduct a probability experiment to compare the theoretical probability (determined using a tree diagram, table or other graphic organizer) and experimental probability of two independent events*"

# Use to print lines, then save in lines_list

def print_lines(n):

    lines_str = ""

    for i in range(1,n+1):
        lines_str = lines_str + "line_"+str(i)+","

    lines_str = lines_str[:-1]

    print(lines_str)

lines_list = [line_1,line_2,line_3,line_4,line_5,line_6,line_7,line_8,line_9]

# Show/Hide buttons

ai_button_show = widgets.Button(button_style='info',description="Show Achievement Indicators", layout=Layout(width='25%', height='30px') )
ai_button_hide = widgets.Button(button_style='info',description="Hide Achievement Indicators", layout=Layout(width='25%', height='30px') )

display(Markdown("For instructors:"))

button_ctr += 1

if(button_ctr % 2 == 0):

    for line in lines_list:
        display(Markdown(line))

    display(ai_button_hide)
    ai_button_hide.on_click( run_current )

else:

    display(ai_button_show)
    ai_button_show.on_click( run_current )

# Statistics and Probability

## Chance and Uncertainty

#### Grade 11  Math

Overview

In this notebook we will explore basic notions and properties about expressing and manipulating probabilities.
The general outcome of this notebook is to use experimental or theoretical probabilities to represent and solve problems involving uncertainty.

Probabilities as ratios, fractions and percents

A natural question to ask is: how do we measure the probability associated to an event of a given probability experiment, i.e. for a given event, what is the probability it occurs?  We define this now, illustrate with a simple example involving dice, and we will then provide an interactive exercise.

Definition. The probability of an event is the ratio between the size of the event (as a collection of outcomes) and the size of the sample space.

The sample space of rolling a single dice is given by $\lbrace$1,2,3,4,5,6 $\rbrace$ and had sample size 6.  If we assume each face is equally likely to occur (i.e. the dice is unbiased), then the probability of getting each face is the ratio or fraction $\dfrac{1}{6}$.

We will denote the probability of getting each number as $P(i)$, where $i$ can either be 1,2,3,4,5,6.
Then, the probability of getting event 1, denoted $P(1)$ is equal to $\dfrac{1}{6}$; more precisely: $P(1)=P(2)=P(3)=P(4)=P(5)=P(6)=\dfrac{1}{6}$.

This is equivalent to stating that the probability of getting any given face is 1 in 6, or $1:6$. Using ratios we have $P(i) = 1:6$, where $i = 1,2,3,4,5,6$. We can also express probabilities using percents. The total number of outcomes is considered 100%. Since there are 6 possible outcomes and assuming equal probability, $P(i) = 100 / 6  = 16.67 \%$.

We summarize in the table below.

|Event  $i$                  |1 |2 |3 |4 |5 |6 |
|----------------------------|--|--|--|--|--|--|
|Probability $P(i)$ as a fraction|$\dfrac{1}{6}$|$\dfrac{1}{6}$|$\dfrac{1}{6}$|$\dfrac{1}{6}$|$\dfrac{1}{6}$|$\dfrac{1}{6}$|
|Probability $P(i)$ as a ratio            |1:6|1:6|1:6|1:6|1:6|1:6|
|Probability $P(i)$ as a percent|16.67%|16.67%|16.67%|16.67%|16.67%|16.67%|

import numpy as np
import matplotlib
from matplotlib.patches import Rectangle
import matplotlib.pyplot as plt
import matplotlib.cm as cm
import random
import matplotlib.gridspec as gridspec
import ipywidgets
from ipywidgets import interact,interact_manual,widgets
%matplotlib inline

### Interactive Example: Probabilities as Fractions, Ratios and Percents

The widget below illustrates a probability experiment using roulettes of various sizes. The basic experiment consists in spinning  a roulette, divided in a given number of compartments, each of the same size and associated with a unique number that identifies it. The outcome of the experiment is the compartment of the roulette whose number shows in red. Basically, this is an experiment similar to rolling a dice, but where we control the number of faces (number of compartments in the roulette). We assume the roulette is unbiased: each compartment has the same chance to appear in red whenspinning the roulette.

On the upper side of the widget you find a drop down menu indicating the size of the sample space, which is the number of compartments of the roulette. In this case, we are considering roulettes whose outcomes are integers from 1 to the size of the sample space. We consider roulettes with sample spaces of size 2, 4, 6 and 8.

Below the drop down menu you will find find a red button. Click it to play.

On the left hand side of the widget you will see a roulette with numbers in black and one number in red. The number in red corresponds to the outcome of the experiment.

On the right hand side you will find a printed message explaining what the probability of the event associated to the obtained outcome is.

Play multiple times to simulate what would happen if you spun the roulette. Change the size of the sample space to learn what the different probabilities associated to each event in the roulette are.

###
def roulette(number_parititions,value):
    if value==True or value==False:
        lucky_number_one = random.choice(np.arange(1,2*number_parititions+1))

        axalpha = 0.05
        figcolor = 'white'
        dpi = 80
        fig = plt.figure(figsize=(15,10), dpi=dpi,facecolor='black')
        plt.subplot(211)

        plt.subplot(212)

        fig.patch.set_edgecolor(figcolor)
        fig.patch.set_facecolor(figcolor)
        ax = plt.subplot(121,projection='polar',facecolor="red")

        ax.patch.set_alpha(axalpha)
        ax.set_axisbelow(True)

        ax1 = plt.subplot(122)
        ax1.grid(False)
        ax1.set_xlim(0.1,0.9)
        ax1.set_ylim(0.4,0.8)

        ax1.set_xticklabels([])
        ax1.set_yticklabels([])
        ax1.axis("off")

        arc = 2. * np.pi
        N = number_parititions
        theta = np.arange(0.0, arc, arc/N)

        if number_parititions == 4:
            radii_ar = [1.0 for i in range(number_parititions)]
            width_ar = [1.0 for i in range(number_parititions)]

            radii = 10 * np.array(radii_ar)
            width = np.pi/4  * np.array(width_ar)

            bars = ax.bar(theta, radii, width=width, bottom=0.0)
            for r, bar in zip(radii, bars):
                bar.set_facecolor("pink")
                bar.set_alpha(0.6)

            ax.text(0, 7, "1", fontsize=20,transform=ax.transData._b,)
            ax.text(5, 5, "2", fontsize=20,transform=ax.transData._b)
            ax.text(7, 0, "3", fontsize=20,transform=ax.transData._b)
            ax.text(4, -5.5, "4", fontsize=20,transform=ax.transData._b)
            ax.text(0, -7, "5", fontsize=20,transform=ax.transData._b)
            ax.text(-5, -5, "6", fontsize=20,transform=ax.transData._b)
            ax.text(-5, 5, "8", fontsize=20,transform=ax.transData._b)
            ax.text(-7, 0, "7", fontsize=20,transform=ax.transData._b)

            if lucky_number_one==1:
                ax.text(0, 7, "1", fontsize=20,transform=ax.transData._b,color="red")
            elif lucky_number_one==2:
                ax.text(5, 5, "2", fontsize=20,transform=ax.transData._b,color="red")
            elif lucky_number_one==3:
                ax.text(7, 0, "3", fontsize=20,transform=ax.transData._b,color='red')
            elif lucky_number_one==4:
                ax.text(4, -5.5, "4", fontsize=20,transform=ax.transData._b,color='red')
            elif lucky_number_one==5:
                ax.text(0, -7, "5", fontsize=20,transform=ax.transData._b,color="red")
            elif lucky_number_one==6:
                ax.text(-5, -5, "6", fontsize=20,transform=ax.transData._b,color="red")
            elif lucky_number_one==7:
                ax.text(-7, 0, "7", fontsize=20,transform=ax.transData._b,color='red')
            elif lucky_number_one==8:
                ax.text(-5, 5, "8", fontsize=20,transform=ax.transData._b,color='red')
            ax1.text(0.1,0.77,"Probability of each event",fontsize=25)
            ax1.text(0.1,0.5,"P(1) = 1/8 = 1:8 = 12.5%\n\nP(2) = 1/8 = 1:8 = 12.5%\n\nP(3) = 1/8 = 1:8 = 12.5%\n\nP(4) = 1/8 = 1:8 = 12.5%\n\nP(5) = 1/8 = 1:8 = 12.5%\n\nP(6) = 1/8 = 1:8 = 12.5%\n\nP(7) = 1/8 = 1:8 = 12.5%\n\nP(8) = 1/8 = 1:8 = 12.5%"\
                      ,fontsize=20)
            ax1.text(0.1,0.45,"Probability Outcome",fontsize=25)
            ax1.text(0.1,0.42,"The result after spinning is " + str(lucky_number_one),fontsize=20)

        elif number_parititions == 3:

            radii_ar = [1.0 for i in range(number_parititions)]
            width_ar = [1.3 for i in range(number_parititions)]

            radii = 10 * np.array(radii_ar)
            width = np.pi/4  * np.array(width_ar)

            bars = ax.bar(theta, radii, width=width, bottom=0.0)
            for r, bar in zip(radii, bars):
                bar.set_facecolor("pink")
                bar.set_alpha(0.6)
            ax.text(-4, 7, "1", fontsize=20,transform=ax.transData._b)
            ax.text(3, 6.5, "2", fontsize=20,transform=ax.transData._b)
            ax.text(7, 0, "3", fontsize=20,transform=ax.transData._b)
            ax.text(3, -6.5, "4", fontsize=20,transform=ax.transData._b)
            ax.text(-4, -7, "5", fontsize=20,transform=ax.transData._b)
            ax.text(-8, 0, "6", fontsize=20,transform=ax.transData._b)

            if lucky_number_one==1:
                ax.text(-4, 7, "1", fontsize=20,transform=ax.transData._b,color='red')
            elif lucky_number_one==2:
                ax.text(3, 6.5, "2", fontsize=20,transform=ax.transData._b,color='red')
            elif lucky_number_one==3:
                ax.text(7, 0, "3", fontsize=20,transform=ax.transData._b,color='red')
            elif lucky_number_one==4:
                ax.text(3, -6.5, "4", fontsize=20,transform=ax.transData._b,color='red')
            elif lucky_number_one==5:
                ax.text(-4, -7, "5", fontsize=20,transform=ax.transData._b,color='red')
            elif lucky_number_one==6:
                ax.text(-8, 0, "6", fontsize=20,transform=ax.transData._b,color='red')

            ax1.text(0.1,0.74,"Probability of each event",fontsize=25)
            ax1.text(0.1,0.55,"P(1) = 1/6 = 1:6 = 16.67%\n\nP(2) = 1/6 = 1:6 = 16.67%\n\nP(3) = 1/6 = 1:6 = 16.67\n\nP(4) = 1/6 = 1:6 = 16.67%\n\nP(5) = 1/6 = 1:6 = 16.67%\n\nP(6) = 1/6 = 1:6 = 16.67%"\
                      ,fontsize=20)
            ax1.text(0.1,0.5,"Probability Outcome",fontsize=25)
            ax1.text(0.1,0.48,"The result after spinning is " + str(lucky_number_one),fontsize=20)

        elif number_parititions == 2:
            radii_ar = [1.0 for i in range(number_parititions)]
            width_ar = [1.7 for i in range(number_parititions)]

            radii = 10 * np.array(radii_ar)
            width = np.pi/4  * np.array(width_ar)

            bars = ax.bar(theta, radii, width=width, bottom=0.0)
            for r, bar in zip(radii, bars):
                bar.set_facecolor("pink")
                bar.set_alpha(0.6)
            ax.text(0, 8, "1", fontsize=20,transform=ax.transData._b)
            ax.text(7, 0, "2", fontsize=20,transform=ax.transData._b)
            ax.text(0, -8, "3", fontsize=20,transform=ax.transData._b)
            ax.text(-8, 0, "4", fontsize=20,transform=ax.transData._b)

            if lucky_number_one==1:
                ax.text(0, 8, "1", fontsize=20,transform=ax.transData._b,color='red')
            elif lucky_number_one==2:
                ax.text(7, 0, "2", fontsize=20,transform=ax.transData._b,color='red')
            elif lucky_number_one==3:
                ax.text(0, -8, "3", fontsize=20,transform=ax.transData._b,color='red')
            elif lucky_number_one==4:
                ax.text(-8, 0, "4", fontsize=20,transform=ax.transData._b,color='red')

            ax1.text(0.1,0.74,"Probability of each event",fontsize=25)
            ax1.text(0.1,0.6,"P(1) = 1/4 = 1:4 = 25%\n\nP(2) = 1/4 = 1:4 =25%\n\nP(3) = 1/4 = 1:4 =25%\n\nP(4) = 1/4 = 1:4 =25%",fontsize=20)
            #ax1.text(0.1,0.5,"This is equivalent to 50%.",fontsize=20)
            ax1.text(0.1,0.54,"Probability Outcome",fontsize=25)
            ax1.text(0.1,0.52,"The result after spinning is " + str(lucky_number_one),fontsize=20)

        elif number_parititions == 1:
            radii_ar = [1.0 for i in range(number_parititions)]
            width_ar = [4 for i in range(number_parititions)]

            radii = 10 * np.array(radii_ar)
            width = np.pi/4  * np.array(width_ar)

            bars = ax.bar(theta, radii, width=width, bottom=0.0)
            for r, bar in zip(radii, bars):
                bar.set_facecolor("pink")
                bar.set_alpha(0.6)
            ax.text(8, 0, "1", fontsize=20,transform=ax.transData._b)
            ax.text(-8, 0, "2", fontsize=20,transform=ax.transData._b)

            if lucky_number_one==1:
                ax.text(8, 0, "1", fontsize=20,transform=ax.transData._b,color='red')
            elif lucky_number_one==2:
                ax.text(-8, 0, "2", fontsize=20,transform=ax.transData._b,color='red')

            ax1.text(0.1,0.7,"Probability of each event",fontsize=25)
            ax1.text(0.1,0.6,"P(1) = 1/2 = 1:2 = 50%\n\nP(2) = 1/2 = 1:2 =50%",fontsize=20)
            ax1.text(0.1,0.54,"Probability Outcome",fontsize=25)
            ax1.text(0.1,0.52,"The result after spinning is " + str(lucky_number_one),fontsize=20)

        ax.tick_params(labelbottom=False, labeltop=False,
                   labelleft=False, labelright=False)

        ax.grid(False)
        ax.set_yticks(np.arange(1, 9, 2))
        ax1.set_yticks([])
        ax1.set_xticks([])

        ax1.set_title("Probabilities as fractions, ratios and percents",fontsize=30)
        plt.show()

style = {'description_width': 'initial'}
lucky = interact(roulette,number_parititions = widgets.Dropdown(
    options={'Two': 1, 'Four': 2, 'Six': 3,'Eight':4},
    value=4,
    description='Size of sample space:',
    style=style
),
    value = widgets.ToggleButton(
        value=True,
        description='Click to Play!',
        disabled=False,
        button_style='danger', # 'success', 'info', 'warning', 'danger' or ''
        tooltip='Description',
        icon='check'
    ))

### Question 1

Using fractions, what is the probability of the event 1, denoted $P(1)$, if the sample size of the roulette is 4?

from ipywidgets import interact_manual,widgets

s = {'description_width': 'initial'}
@interact(answer =widgets.Select(
                    options=["Select option","1/2",\
                            "4","1/3",\
                             "1/4"],
                    value='Select option',
                    description="Probability as fraction",
                    disabled=False,
                    style=s
))

def reflective_angle_question(answer):
    if answer=="Select option":
        print("Click on the correct probability expressed as a fraction.")

    elif answer=="1/4":
        print("Correct!\nWith a sample space of size 4, each with equal likelihood, P(1)=1/4.")
    elif answer != "1/4" or answer != "Select Option":
        print("Hint: What is P(1) if P(i) = 1/4 for all i = 1,2,3,4?")

### Question 2

What is P(1) if the size of the sample space is 4, but this time expressed as percent?

from ipywidgets import interact_manual,widgets

s = {'description_width': 'initial'}
@interact(answer =widgets.Select(
                    options=["Select option","25%",\
                            "100%","40%",\
                             "10%"],
                    value='Select option',
                    description="Probability as percent",
                    disabled=False,
                    style=s
))

def reflective_angle_question(answer):
    if answer=="Select option":
        print("Click on the correct probability expressed in percent.")

    elif answer=="25%":
        print("Correct!\nWith four probability events, each with equal likelihood, P(1) = 25%.")
    elif answer != "25%" or answer != "Select Option":
        print("Hint: The total number of outcomes is 4, which corresponds to 100%. What is 100/4?")

### Question 3

Using the widget created above, change the Size of sample space to 8. What is $P(7)$ as a ratio?

from ipywidgets import interact_manual,widgets

s = {'description_width': 'initial'}
@interact(answer =widgets.Select(
                    options=["Select option","1:6",\
                            "1:4","1:8",\
                             "1:7"],
                    value='Select option',
                    description="Probability as ratio",
                    disabled=False,
                    style=s
))

def reflective_angle_question(answer):
    if answer=="Select option":
        print("Click on the correct probability expressed as a percentage.")

    elif answer=="1:8":
        print("Correct!\nWith eight probability events, each with equal likelihood, \nP(7) =  1:8.")
    elif answer != "1:8" or answer != "Select Option":
        print("Hint: 1 in every 8 corresponds to the event 7. What is P(7) in ratio?")

Independent Events & Sample Space

In this section, we define the concept of **independent probability experiments** as well as the corresponding sample space, and provide a game where we can experiment with the two concepts.

Definition. We say that two probability experiments are independent if the outcome of one does not affect the outcome of the other.

For example, if we spin two roulettes at the same time, then the two experiments are independent since the outcome of each does not affect the other.

Try spinning the two roulettes below via using the red button. As before, in each the number in red denotes the outcome of the experiment. These two roulettes are not linked and as you can see from a few spins, their outcomes are not related at all: spinning them are independent experiments.

def spin(value):
    if value==True or value==False:

        lucky_number_one = random.choice([1,2,3,4,5,6])

        lucky_number_one_c = random.choice([1,2,3,4,5,6])

        x_t,y_t = [i/10 for i in range(10)],[i/10 for i in range(10)]

        axalpha = 0.05
        figcolor = 'white'
        dpi = 80

        gs = gridspec.GridSpec(2, 3)
        fig = plt.figure(figsize=(15,8), dpi=dpi,facecolor='black')

        fig.patch.set_edgecolor(figcolor)
        fig.patch.set_facecolor(figcolor)

        ax1 = plt.subplot(gs[0, 0])
        ax1.grid(False)
        ax1.axis("Off")
        ax2 = plt.subplot(gs[1, 0])
        ax2.grid(False)
        ax2.axis("Off")
        ax5 = plt.subplot(gs[0, 2])
        ax5.grid(False)
        ax5.axis("Off")
        ax6 = plt.subplot(gs[1, 2])
        ax6.grid(False)
        ax6.axis("Off")
        #ax.axis("Off")

        ax = plt.subplot(gs[0,0],projection='polar',facecolor="red") # row 0, col 0
        plt.plot([0,1])

        ax.plot(x_t,y_t,transform=ax.transData._b,color="#FEE9FF",linewidth=5)

        ax.patch.set_alpha(axalpha)
        ax.set_axisbelow(True)

        number_parititions = 3
        arc = 2. * np.pi
        N = number_parititions
        theta = np.arange(0.0, arc, arc/N)

        radii_ar = [1.0 for i in range(number_parititions)]
        width_ar = [1.3 for i in range(number_parititions)]

        radii = 10 * np.array(radii_ar)
        width = np.pi/4  * np.array(width_ar)

        bars = ax.bar(theta, radii, width=width, bottom=0.0)
        for r, bar in zip(radii, bars):
            bar.set_facecolor("pink")
            bar.set_alpha(0.6)
        ax.text(-4, 7, "1", fontsize=20,transform=ax.transData._b)
        ax.text(3, 6.5, "2", fontsize=20,transform=ax.transData._b)
        ax.text(7, 0, "3", fontsize=20,transform=ax.transData._b)
        ax.text(3, -6.5, "4", fontsize=20,transform=ax.transData._b)
        ax.text(-4, -7, "5", fontsize=20,transform=ax.transData._b)
        ax.text(-8, 0, "6", fontsize=20,transform=ax.transData._b)

        if lucky_number_one==1:
            #bar3.set_facecolor("#000000")
            ax.text(-4, 7, "1", fontsize=20,transform=ax.transData._b,color='red')
        elif lucky_number_one==2:
            #bar2.set_facecolor("#000000")
            ax.text(3, 6.5, "2", fontsize=20,transform=ax.transData._b,color='red')

        elif lucky_number_one==3:
            #bar1.set_facecolor("#000000")
            ax.text(7, 0, "3", fontsize=20,transform=ax.transData._b,color='red')

        elif lucky_number_one==4:
            #bar6.set_facecolor("#000000")
            ax.text(3, -6.5, "4", fontsize=20,transform=ax.transData._b,color='red')

        elif lucky_number_one==5:
            #bar5.set_facecolor("#000000")
            ax.text(-4, -7, "5", fontsize=20,transform=ax.transData._b,color='red')

        elif lucky_number_one==6:
            #bar4.set_facecolor("#000000")
            ax.text(-8, 0, "6", fontsize=20,transform=ax.transData._b,color='red')

        ax.tick_params(labelbottom=False, labeltop=False,
                   labelleft=False, labelright=False)

        ax.grid(False)
        ax.axis("On")
        ax.set_title("Top Roulette",fontsize=20)

        ax4 = plt.subplot(gs[1, 0],projection='polar',facecolor="red") # row 0, col 0
        plt.plot([0,1])

        ax4.plot(x_t,y_t,transform=ax4.transData._b,color="#FEE9FF",linewidth=5)
        ax4.patch.set_alpha(axalpha)
        ax4.set_axisbelow(True)

        bars = ax4.bar(theta, radii, width=width, bottom=0.0)
        for r, bar in zip(radii, bars):
            bar.set_facecolor("pink")
            bar.set_alpha(0.6)
        ax4.text(-4, 7, "1", fontsize=20,transform=ax4.transData._b)
        ax4.text(3, 6.5, "2", fontsize=20,transform=ax4.transData._b)
        ax4.text(7, 0, "3", fontsize=20,transform=ax4.transData._b)
        ax4.text(3, -6.5, "4", fontsize=20,transform=ax4.transData._b)
        ax4.text(-4, -7, "5", fontsize=20,transform=ax4.transData._b)
        ax4.text(-8, 0, "6", fontsize=20,transform=ax4.transData._b)

        if lucky_number_one_c==1:
            #bar3.set_facecolor("#000000")
            ax4.text(-4, 7, "1", fontsize=20,transform=ax4.transData._b,color='red')
        elif lucky_number_one_c==2:
            #bar2.set_facecolor("#000000")
            ax4.text(3, 6.5, "2", fontsize=20,transform=ax4.transData._b,color='red')

        elif lucky_number_one_c==3:
            #bar1.set_facecolor("#000000")
            ax4.text(7, 0, "3", fontsize=20,transform=ax4.transData._b,color='red')

        elif lucky_number_one_c==4:
            #bar6.set_facecolor("#000000")
            ax4.text(3, -6.5, "4", fontsize=20,transform=ax4.transData._b,color='red')

        elif lucky_number_one_c==5:
            #bar5.set_facecolor("#000000")
            ax4.text(-4, -7, "5", fontsize=20,transform=ax4.transData._b,color='red')

        elif lucky_number_one_c==6:
            #bar4.set_facecolor("#000000")
            ax4.text(-8, 0, "6", fontsize=20,transform=ax4.transData._b,color='red')

        ax4.tick_params(labelbottom=False, labeltop=False,
                   labelleft=False, labelright=False)

        ax4.grid(False)
        ax4.axis("On")
        ax4.set_title("Bottom Roulette",fontsize=20)

        x,y = np.array([i/10 for i in range(11)]),np.array([i/10 for i in range(11)])

        ax2 = plt.subplot(gs[0,1:]) # row 0, col 0
        plt.plot([0,1])

        ax2.grid(False)
        ax2.axis("Off")
        #ax2.set_title("Top Roulette Outcome",fontsize=20)
        ax2.plot(x,y,color='white',linewidth=4)
        ax2.text(0.1,0.5,"Top Roulette Outcome: " +str(lucky_number_one),fontsize=20)

        ax5 = plt.subplot(gs[1,1:]) # row 0, col 0
        plt.plot([0,1])

        ax5.grid(False)
        ax5.axis("Off")
        #ax5.set_title("Bottom Roulette Outcome",fontsize=20)
        ax5.plot(x,y,color='white',linewidth=4)
        ax5.text(0.1,0.5,"Bottom Roulette Outcome: " + str(lucky_number_one_c),fontsize=20)

lucky = interact(spin,value = widgets.ToggleButton(
        value=True,
        description="Spin",
        disabled=False,
        button_style='danger', # 'success', 'info', 'warning', 'danger' or ''
        tooltip='Description',
        icon='check'
    ))

Recall that

    Definition. The sample space of an experiment is the set of all possible outcomes of that experiment.

If, for example, we take the two roulettes above, we can define the sample space of spinning both of then at the same time as the set of all ordered pairs $(n_t,n_b)$, where $n_t$ denotes the outcome of spinning the top roulette and $n_b$ denotes the outcome of spinning the bottom roulette. This sample space is given by the table below, which is a 6 by 6 table where each entry is a pair $(n_t,n_b)$.

def spin_sample_space(value):
    if value==True or value==False:

        lucky_number_one = random.choice([1,2,3,4,5,6])

        lucky_number_one_c = random.choice([1,2,3,4,5,6])

        x_t,y_t = [i/10 for i in range(10)],[i/10 for i in range(10)]

        axalpha = 0.05
        figcolor = 'white'
        dpi = 80

        gs = gridspec.GridSpec(2, 2)
        fig = plt.figure(figsize=(15,8), dpi=dpi,facecolor='black')

        fig.patch.set_edgecolor(figcolor)
        fig.patch.set_facecolor(figcolor)
        ax = plt.subplot(gs[0, 0],projection='polar',facecolor="red")
        plt.plot([0,1])

        ax.patch.set_alpha(axalpha)
        ax.set_axisbelow(True)

        ax.plot(x_t,y_t,transform=ax.transData._b,color="#FEE9FF",linewidth=5)

        number_parititions = 3
        arc = 2. * np.pi
        N = number_parititions
        theta = np.arange(0.0, arc, arc/N)

        radii_ar = [1.0 for i in range(number_parititions)]
        width_ar = [1.3 for i in range(number_parititions)]

        radii = 10 * np.array(radii_ar)
        width = np.pi/4  * np.array(width_ar)

        bars = ax.bar(theta, radii, width=width, bottom=0.0)
        for r, bar in zip(radii, bars):
            bar.set_facecolor("pink")
            bar.set_alpha(0.6)
        ax.text(-4, 7, "1", fontsize=20,transform=ax.transData._b)
        ax.text(3, 6.5, "2", fontsize=20,transform=ax.transData._b)
        ax.text(7, 0, "3", fontsize=20,transform=ax.transData._b)
        ax.text(3, -6.5, "4", fontsize=20,transform=ax.transData._b)
        ax.text(-4, -7, "5", fontsize=20,transform=ax.transData._b)
        ax.text(-8, 0, "6", fontsize=20,transform=ax.transData._b)

        if lucky_number_one==1:
            #bar3.set_facecolor("#000000")
            ax.text(-4, 7, "1", fontsize=20,transform=ax.transData._b,color='red')
        elif lucky_number_one==2:
            #bar2.set_facecolor("#000000")
            ax.text(3, 6.5, "2", fontsize=20,transform=ax.transData._b,color='red')

        elif lucky_number_one==3:
            #bar1.set_facecolor("#000000")
            ax.text(7, 0, "3", fontsize=20,transform=ax.transData._b,color='red')

        elif lucky_number_one==4:
            #bar6.set_facecolor("#000000")
            ax.text(3, -6.5, "4", fontsize=20,transform=ax.transData._b,color='red')

        elif lucky_number_one==5:
            #bar5.set_facecolor("#000000")
            ax.text(-4, -7, "5", fontsize=20,transform=ax.transData._b,color='red')

        elif lucky_number_one==6:
            #bar4.set_facecolor("#000000")
            ax.text(-8, 0, "6", fontsize=20,transform=ax.transData._b,color='red')

        ax.tick_params(labelbottom=False, labeltop=False,
                   labelleft=False, labelright=False)

        ax.grid(False)
        ax.axis("On")
        ax.set_title("Top Roulette",fontsize=20)

        ax4 = plt.subplot(gs[1, 0],projection='polar',facecolor="red") # row 0, col 0
        plt.plot([0,1])

        ax4.patch.set_alpha(axalpha)
        ax4.set_axisbelow(True)

        ax4.plot(x_t,y_t,transform=ax4.transData._b,color="#FEE9FF",linewidth=5)
        bars = ax4.bar(theta, radii, width=width, bottom=0.0)
        for r, bar in zip(radii, bars):
            bar.set_facecolor("pink")
            bar.set_alpha(0.6)
        ax4.text(-4, 7, "1", fontsize=20,transform=ax4.transData._b)
        ax4.text(3, 6.5, "2", fontsize=20,transform=ax4.transData._b)
        ax4.text(7, 0, "3", fontsize=20,transform=ax4.transData._b)
        ax4.text(3, -6.5, "4", fontsize=20,transform=ax4.transData._b)
        ax4.text(-4, -7, "5", fontsize=20,transform=ax4.transData._b)
        ax4.text(-8, 0, "6", fontsize=20,transform=ax4.transData._b)

        if lucky_number_one_c==1:
            #bar3.set_facecolor("#000000")
            ax4.text(-4, 7, "1", fontsize=20,transform=ax4.transData._b,color='red')
        elif lucky_number_one_c==2:
            #bar2.set_facecolor("#000000")
            ax4.text(3, 6.5, "2", fontsize=20,transform=ax4.transData._b,color='red')

        elif lucky_number_one_c==3:
            #bar1.set_facecolor("#000000")
            ax4.text(7, 0, "3", fontsize=20,transform=ax4.transData._b,color='red')

        elif lucky_number_one_c==4:
            #bar6.set_facecolor("#000000")
            ax4.text(3, -6.5, "4", fontsize=20,transform=ax4.transData._b,color='red')

        elif lucky_number_one_c==5:
            #bar5.set_facecolor("#000000")
            ax4.text(-4, -7, "5", fontsize=20,transform=ax4.transData._b,color='red')

        elif lucky_number_one_c==6:
            #bar4.set_facecolor("#000000")
            ax4.text(-8, 0, "6", fontsize=20,transform=ax4.transData._b,color='red')

        ax4.tick_params(labelbottom=False, labeltop=False,
                   labelleft=False, labelright=False)

        ax4.grid(False)
        ax4.axis("On")
        ax4.set_title("Bottom Roulette",fontsize=20)

        ax1 = plt.subplot(gs[:, 1:],facecolor='#0475A8') # row 1, span all columns
        plt.plot([0,1])
        ax1.set_axisbelow(True)
        ax1.grid(color='black', linestyle='-', linewidth=2)

        ax1.set_xlim(0.1,0.7)
        ax1.set_ylim(0.1,0.7)

        rec = Rectangle([lucky_number_one/10,lucky_number_one_c/10],0.1,0.1,facecolor="black")
        ax1.add_patch(rec)
        x,y = [lucky_number_one/10,lucky_number_one/10+ 0.1], [lucky_number_one_c/10,lucky_number_one_c/10 + 0.1]
        ax1.plot(x,y,color='black',linewidth=4)
        for i in range(1,7):
            for j in range(1,7):
                ax1.text(i/10 + 0.02,j/10 + 0.055,"(" + str(i)+","+ str(j)+")",fontsize = 15,color='white')
        ax1.set_xticklabels([" ",1,2,3,4,5,6])
        ax1.set_yticklabels([" ",1,2,3,4,5,6])
        ax1.set_xlabel("Top Roulette Outcome",fontsize = 20)
        ax1.set_ylabel("Bottom Roulette Outcome",fontsize = 20)
        ax1.xaxis.tick_top()
        ax1.invert_yaxis()

        plt.show()

lucky = interact(spin_sample_space,value = widgets.ToggleButton(
        value=True,
        description="Spin",
        disabled=False,
        button_style='danger', # 'success', 'info', 'warning', 'danger' or ''
        tooltip='Description',
        icon='check'
    ))

In this case, since the events are independent and each event has six possible outcomes, the sample space contains

$$6 \times 6 = 36$$

possible outcomes, where each outcome is a pair of the form $(n_t,n_b)$.

The probability of obtaining any given pair $(n_t,n_b)$, is given by the probability of obtaining $n_t$ with the top roulette (first experiment) multiplied by the probability of obtaining $n_b$ with the bottom roulette (second experiment). If we assume that each event is equally likely to occur

$$P(n_t,n_b) = \dfrac{1}{6} \times \dfrac{1}{6} = \dfrac{1}{36}$$

Note that we can multiply
* the sizes of the two sample spaces (for each roulette) to obtain the sample space size of the combined experiments
* the probabilities of obtaining $n_t$ and $n_b$ to obtain the probability of the event $(n_t,n_b)$
**because** the two experiments are **independent**. This is an important property of joint probability experiments.

Property. Consider two independent probability experiments $E_1$ and $E_2$ of respective sample space sizes $N_1$ and $N_2$. The size of the sample space of the joint experiment $(E_1,E_2)$ is $N_1\times N_2$. The probability of the event $(n_1,n_2)$ is $P(n_1)\times P(n_2)$.

## Question 4

What is the probability assigned to the event (1,1)?

from ipywidgets import interact_manual,widgets

s = {'description_width': 'initial'}
@interact(answer =widgets.Select(
                    options=["Select option","2/36",\
                            "1/6","36",\
                             "1/36"],
                    value='Select option',
                    description="Probability as fraction",
                    disabled=False,
                    style=s
))

def reflective_angle_question(answer):
    if answer=="Select option":
        print("Click on the correct probability expressed as a fraction.")

    elif answer=="1/36":
        print("Correct!")
    elif answer != "1/36" or answer != "Select Option":
        print("Hint: There are 36 events, each with equal likelihood of occurrence. \nYou also know that each pair is unique.")

A second example of a sample space

Let's take the two roulettes as before but this time let's define the sample space of spinning both of them at the same time as the as the **parity** of the sum $$n_t + n_b$$ where as before $n_t$ denotes the outcome of the Top Roulette and $n_b$ denotes the outcome of the Bottom Roulette.

Then the sample space is given by the set $\lbrace \text{even},\text{odd} \rbrace$

This sample space is given by the table below, which is a 6 by 6 table where each entry is a pair contains the **parity** of the sum $$n_t + n_b$$

def fair(value):
    if value==True or value==False:

        lucky_number_one = random.choice([1,2,3,4,5,6])
        lucky_number_one_c = random.choice([1,2,3,4,5,6])

        x_t,y_t = [i/10 for i in range(10)],[i/10 for i in range(10)]

        axalpha = 0.05
        figcolor = 'white'
        dpi = 80

        gs = gridspec.GridSpec(2, 2)
        fig = plt.figure(figsize=(15,8), dpi=dpi,facecolor='black')

        fig.patch.set_edgecolor(figcolor)
        fig.patch.set_facecolor(figcolor)
        ax = plt.subplot(gs[0, 0],projection='polar',facecolor="red") # row 0, col 0
        plt.plot([0,1])

        ax.patch.set_alpha(axalpha)
        ax.set_axisbelow(True)
        ax.plot(x_t,y_t,transform=ax.transData._b,color="#FEE9FF",linewidth=5)
        number_parititions = 3
        arc = 2. * np.pi
        N = number_parititions
        theta = np.arange(0.0, arc, arc/N)

        radii_ar = [1.0 for i in range(number_parititions)]
        width_ar = [1.3 for i in range(number_parititions)]

        radii = 10 * np.array(radii_ar)
        width = np.pi/4  * np.array(width_ar)

        bars = ax.bar(theta, radii, width=width, bottom=0.0)
        for r, bar in zip(radii, bars):
            bar.set_facecolor("pink")
            bar.set_alpha(0.6)
        ax.text(-4, 7, "1", fontsize=20,transform=ax.transData._b)
        ax.text(3, 6.5, "2", fontsize=20,transform=ax.transData._b)
        ax.text(7, 0, "3", fontsize=20,transform=ax.transData._b)
        ax.text(3, -6.5, "4", fontsize=20,transform=ax.transData._b)
        ax.text(-4, -7, "5", fontsize=20,transform=ax.transData._b)
        ax.text(-8, 0, "6", fontsize=20,transform=ax.transData._b)

        if lucky_number_one==1:
            #bar3.set_facecolor("#000000")
            ax.text(-4, 7, "1", fontsize=20,transform=ax.transData._b,color='red')
        elif lucky_number_one==2:
            #bar2.set_facecolor("#000000")
            ax.text(3, 6.5, "2", fontsize=20,transform=ax.transData._b,color='red')

        elif lucky_number_one==3:
            #bar1.set_facecolor("#000000")
            ax.text(7, 0, "3", fontsize=20,transform=ax.transData._b,color='red')

        elif lucky_number_one==4:
            #bar6.set_facecolor("#000000")
            ax.text(3, -6.5, "4", fontsize=20,transform=ax.transData._b,color='red')

        elif lucky_number_one==5:
            #bar5.set_facecolor("#000000")
            ax.text(-4, -7, "5", fontsize=20,transform=ax.transData._b,color='red')

        elif lucky_number_one==6:
            #bar4.set_facecolor("#000000")
            ax.text(-8, 0, "6", fontsize=20,transform=ax.transData._b,color='red')

        ax.tick_params(labelbottom=False, labeltop=False,
                   labelleft=False, labelright=False)

        ax.grid(False)
        ax.axis("On")
        ax.set_title("Top Roulette",fontsize=20)

        ax1 = plt.subplot(gs[:, 1:],facecolor='#0475A8') # row 1, span all columns
        plt.plot([0,1])
        ax1.set_axisbelow(True)
        ax1.grid(color='black', linestyle='-', linewidth=2)

        ax1.set_xlim(0.1,0.7)
        ax1.set_ylim(0.1,0.7)
        ax1.set_xlabel("Top Roulette Outcome",fontsize=18)
        ax1.set_ylabel("Bottom Roulette Outcome",fontsize=18)
        ax1.xaxis.tick_top()
        ax1.invert_yaxis()
        #
        for i in range(1,7):
            for j in range(1,7):
                if (i+j)%2==0:
                    ax1.text(i/10 + 0.02,j/10 + 0.055,"even" ,fontsize = 15,color='white')
                else:
                    ax1.text(i/10 + 0.02,j/10 + 0.055,"odd" ,fontsize = 15,color='white')

        rec = Rectangle([lucky_number_one/10,lucky_number_one_c/10],0.1,0.1,facecolor="black")
        x,y = [lucky_number_one/10,lucky_number_one/10+ 0.1], [lucky_number_one_c/10,lucky_number_one_c/10 + 0.1]
        ax1.plot(x,y,color='black',linewidth=4)
        #rec_c = Rectangle([lucky_number_one_c/10,lucky_number_two_c/10],0.1,0.1,facecolor="#6b6e72")
        ax1.add_patch(rec)
        #ax1.add_patch(rec_c)
        ax1.set_xticklabels([" ",1,2,3,4,5,6])
        ax1.set_yticklabels([" ",1,2,3,4,5,6])

        ax4 = plt.subplot(gs[1, 0],projection='polar',facecolor="red") # row 0, col 0
        plt.plot([0,1])

        ax4.patch.set_alpha(axalpha)
        ax4.set_axisbelow(True)

        ax4.plot(x_t,y_t,transform=ax4.transData._b,color="#FEE9FF",linewidth=5)

        bars = ax4.bar(theta, radii, width=width, bottom=0.0)
        for r, bar in zip(radii, bars):
            bar.set_facecolor("pink")
            bar.set_alpha(0.6)
        ax4.text(-4, 7, "1", fontsize=20,transform=ax4.transData._b)
        ax4.text(3, 6.5, "2", fontsize=20,transform=ax4.transData._b)
        ax4.text(7, 0, "3", fontsize=20,transform=ax4.transData._b)
        ax4.text(3, -6.5, "4", fontsize=20,transform=ax4.transData._b)
        ax4.text(-4, -7, "5", fontsize=20,transform=ax4.transData._b)
        ax4.text(-8, 0, "6", fontsize=20,transform=ax4.transData._b)

        if lucky_number_one_c==1:
            #bar3.set_facecolor("#000000")
            ax4.text(-4, 7, "1", fontsize=20,transform=ax4.transData._b,color='red')
        elif lucky_number_one_c==2:
            #bar2.set_facecolor("#000000")
            ax4.text(3, 6.5, "2", fontsize=20,transform=ax4.transData._b,color='red')

        elif lucky_number_one_c==3:
            #bar1.set_facecolor("#000000")
            ax4.text(7, 0, "3", fontsize=20,transform=ax4.transData._b,color='red')

        elif lucky_number_one_c==4:
            #bar6.set_facecolor("#000000")
            ax4.text(3, -6.5, "4", fontsize=20,transform=ax4.transData._b,color='red')

        elif lucky_number_one_c==5:
            #bar5.set_facecolor("#000000")
            ax4.text(-4, -7, "5", fontsize=20,transform=ax4.transData._b,color='red')

        elif lucky_number_one_c==6:
            #bar4.set_facecolor("#000000")
            ax4.text(-8, 0, "6", fontsize=20,transform=ax4.transData._b,color='red')

        ax4.tick_params(labelbottom=False, labeltop=False,
                   labelleft=False, labelright=False)

        ax4.grid(False)
        ax4.axis("On")
        ax4.set_title("Bottom Roulette",fontsize=20)

        sum_n = lucky_number_one + lucky_number_one_c

        #print("Top Roulette Outcome + Bottom Roulette Outcome")
        print(str(lucky_number_one) + " + " + str(lucky_number_one_c) + " = " + str(sum_n))
        if sum_n %2==0:
            print("OUTCOME: even" )
        else:
            print("OUTCOME: odd" )

        plt.show()

lucky = interact(fair,value = widgets.ToggleButton(
        value=True,
        description="Let's Play",
        disabled=False,
        button_style='danger', # 'success', 'info', 'warning', 'danger' or ''
        tooltip='Description',
        icon='check'
    ))

### Question 5

We claim that this game is fair, but how can we verify it?

Recall that a probability experiment is fair if every outcome is equally likely to occur. In order for this experiment to be fair, the probability of the event **even**  *must* be equal to the probability of event **odd**.

What is the probability that the sum of the numbers in the top and bottom roulettes is even?

from ipywidgets import interact_manual,widgets

s = {'description_width': 'initial'}
@interact(answer =widgets.Select(
                    options=["Select option","1/2",\
                            "4/36","1/3",\
                             "18/36"],
                    value='Select option',
                    description="Probability sum is even",
                    disabled=False,
                    style=s
))

def fair_game(answer):
    if answer=="Select option":
        print("Click on the correct probability expressed as a fraction.")

    elif answer=="1/2" or answer=="18/36":
        print("Correct!\nThere are a total of 36 possible outcomes. 18 out of 36 are even numbers.\nThus the probability P(even) = 18/36 or 1/2. ")
    elif answer != "1/2" or answer != "Select Option" or answer!="18/36":
        print("Hint: There are 36 entries in our sample space, each with equal likelihood of occurrence.\nHow many of the 36 correspond to even numbers")

### Question 6
What is the probability that the sum of the numbers in the top and bottom roulettes is odd? In other words, what is the probability that Bob will win?

from ipywidgets import interact_manual,widgets

s = {'description_width': 'initial'}
@interact(answer =widgets.Select(
                    options=["Select option","19/36",\
                            "17/36","18/36",\
                             "1/2"],
                    value='Select option',
                    description="Probability sum is odd",
                    disabled=False,
                    style=s
))

def fair_game(answer):
    if answer=="Select option":
        print("Click on the correct probability expressed as a fraction.")

    elif answer=="1/2" or answer=="18/36":
        print("Correct!\nThere are a total of 36 possible outcomes. 18 out of 36 are odd numbers.\nThus the P(odd) = 18/36 or 1/2. ")
    elif answer != "1/2" or answer != "Select Option" or answer!="18/36":
        print("Hint: There are 36 entries in our sample space, each with equal likelihood of occurrence.\nHow many of the 36 correspond to odd numbers?")

In the section above we learned that the there are 18 out of 36 possible outcomes where the sum

$$n_t + n_b$$

is an even number. Thus

$$P(even) = \dfrac{18}{36} = \dfrac{1}{2}$$

Similarly, there are 18 out of 36 possible outcomes where the sum

$$n_t + n_b$$

is an odd number. Thus

$$P(odd) = \dfrac{18}{36} = \dfrac{1}{2}$$

Then $P(odd) = P(even)$. With this we verify that indeed the experiment is fair.

Theoretical vs Experimental Probability

We begin by stating a few definitions.

    Definition. The Theoretical Probability of an event $A$, denoted $P_T(A)$, is the ratio of the number of outcomes corresponding to this event to the number of possible outcomes.

$$P_T(A) = \dfrac{\text{Total Number of Instances of event A in the Sample Space}}{\text{Total Number of Possible Outcomes}}$$

If we take our fair experiment with two roulettes and sample space parity of outcome sum $\lbrace \text{even}, \text{odd} \rbrace$, the theoretical probability of an even event is

$$P_T(\text{even}) = \dfrac{18}{36}$$

Definition. The Experimental Probability of an event $A$, denoted $P_E(A)$, is computed over running the probability experiment a number of times and computing the observed ratio between the number of time the event occured  and the number of trials of the experiment.

$$P_E(A) = \dfrac{\text{Number of Times Event A Actually Occurred}}{\text{Number of trials}}$$

In order to determine $P_E(\text{even})$, we first need to spin the roulettes a few times and compare.

Use the widget below to simulate spinning the two roulettes. Use the slider to set a number of trials. In this interactive exercise, you can set number of trials as an integer between 1 and 100. Press `Run Interact` to run an experiment for the given number of trials.

On the right hand side you will find a printed message outlining the experimental probability of each event: Sum is Even and Sum is Odd from the number of trial specified using the widget.

On the left hand side you can find a graph comparing both.

Press the `Run Interact` button several times.

%matplotlib inline

def die(number):
    count_A,count_C = 0,0

    for i in range(number):
        lucky_number_one = random.choice([1,2,3,4,5,6])
        lucky_number_two = random.choice([1,2,3,4,5,6])
        if lucky_number_one - lucky_number_two >=0:
            count_A +=1
        else:
            count_C +=1

    return [count_A,count_C]

def even_sum(number):
    count_A,count_C = 0,0

    for i in range(number):
        lucky_number_one = random.choice([1,2,3,4,5,6])
        lucky_number_two = random.choice([1,2,3,4,5,6])

        sum_n = lucky_number_one + lucky_number_two
        if  sum_n%2 == 0:
            count_A +=1
        else:
            count_C +=1

    return [count_A,count_C]

def experimental_prob(number):
    [varoi_1,varoi_2] = even_sum(number)
    fig,(ax1,ax2,ax3) = plt.subplots(1,3,sharey=True,figsize=(15,4))

    ax2.axis("Off")
    ax3.axis("Off")
    axalpha = 0.05

    even = varoi_1/number
    odd = varoi_2/number

    labels = ['', '',  'Even Sum', '','Odd Sum']

    ax1 = plt.subplot(131,facecolor="white")
    ax1.set_title("Experimental Probability",fontsize=20)
    ax1.set_ylabel("Probability",fontsize=15)
    ax1.set_xlabel("Outcomes",fontsize=15)
    ax1.set_xlim([0,3])

    ax1.set_xticklabels(labels)

    x = np.arange(1,3)
    f1,f2= ax1.bar(x,[even,odd])
    f1.set_facecolor("#8642f4")
    f2.set_facecolor("#518900")

    ax1.grid(which='both')
    ax1.grid(b=True,which='minor',alpha=0.2,linestyle='--',color='black')
    ax1.grid(which='major', alpha=0.2,linestyle='--',color='black')

    ax3 = plt.subplot(132)
    ax3.axis("Off")

    #ax3.set_title( "Positive vs Negative\nLuck Roulette: Outcome",fontsize=20)
    rec1 = Rectangle((0.1,0.8),0.3,0.1,facecolor="#8642f4")
    ax3.add_patch(rec1)
    ax3.text(0.5,0.83,"Experimental Probability: Sum is Even",fontsize=20)
    rec2 = Rectangle((0.1,0.7),0.3,0.1,facecolor="#518900")
    ax3.add_patch(rec2)
    ax3.text(0.5,0.73,"Experimental Probability: Sum is Odd",fontsize=20)

    ax2 = plt.subplot(133,facecolor="white")
    ax2.axis("Off")

    ax2.text(0.9,0.83,str(varoi_1) + "/" + str(number),fontsize=20)
    ax2.text(0.9,0.73,str(varoi_2) + "/" + str(number),fontsize=20)

    ax3.set_title("                   Even or Odd Sum Experiment:\n",fontsize=20)

    plt.show()

def run_fair_exp(number):
    experimental_prob(number)
    #experimental_prob(number,"Negative")
interact_manual(experimental_prob,number=widgets.IntSlider(
            value=10,
            min=1,
            max=100,
            step=1,
            description='Total Number of Trials',
            disabled=False,
            continuous_update=False,
            orientation='horizontal',
            readout=True,
            readout_format='d',
            style =style
));

### Question 7

Use the box below to enter your observations.

How do the experimental probabilities of each event change as you increase the number of trials?

from ipywidgets import widgets as w
from ipywidgets import Button, Layout
from IPython.display import display, Javascript, Markdown

def rerun_cell( b ):
    display(Javascript('IPython.notebook.execute_cell_range(IPython.notebook.get_selected_index()+1,IPython.notebook.get_selected_index()+2)'))

style = {'description_width': 'initial'}

question7_text = w.Textarea( value='', placeholder='Write your answer here. Press Record Answer when you finish.', description='', disabled=False , layout=Layout(width='100%', height='75px') )
question7_button = w.Button(button_style='info',description="Record Answer", layout=Layout(width='15%', height='30px'))

display(question7_text)
display(question7_button)

question7_button.on_click( rerun_cell )

question7_input = question7_text.value

if(question7_input != ''):
    question7_text.close()
    question7_button.close()
    display(Markdown("### Your answer for Question 7: Conclusions"))
    display(Markdown(question7_input))

**Remarks**

We observe that every time we press the `Run Interact` button on the interactive above, the experimental probability of each event varies. However, it seems like as we increase the number of trials, the experimental probabilities of each event approach $1/2$.

Let us explore what happens if we increase the number of trials to, say 10,000.

In the widget below you can find a slider that allows you to control the number of trials. In this interactive exercise, you can set number of trials as an integer between 1 and 10,000.

On the left hand side you can find a plot like the one we explored above. On the right hand side you can find the theoretical probability of events Sum is Even and Sum is Odd.

Press the `Run Interact` button. Increase the Total Number of Trials and press the `Run Interact` button multiple times.

def toss(number):
    store_head = []
    store_tail = []
    other = []
    for i in range(number):
        toss_coin= random.choice(np.arange(2))

        if toss_coin==0:
            store_head.append(toss_coin)
        elif toss_coin==1:
            store_tail.append(toss_coin)

    return [store_head,store_tail]

def plot_coin_experiment(number):
    varoi = toss(number)
    fig,ax = plt.subplots(figsize=(5,5))

    ax.set_title("Distribution of Experimental Coin Flipping",fontsize=35)
    plt.ylabel("Frequency",fontsize=25)
    plt.xlabel("Heads or Tails",fontsize=25)
    plt.xticks(np.arange(2), ('Total Number of Heads', 'Total Number of Tails'))
    plt.hist(varoi[0])
    plt.hist(varoi[1])
    plt.grid(which='both')
    plt.grid(b=True,which='minor',alpha=0.2,linestyle='--',color='black')
    plt.grid(which='major', alpha=0.2,linestyle='--',color='black')

def plot_die_experiment(number):
    varoi = die(number)
    theor_A = 21/36
    theor_C = 15/36
    #print(theor)
    fig,(ax1,ax2) = plt.subplots(1,2,sharey=True,figsize=(15,8))

    # Experimental Probability
    ax1.set_title("Experimental Distribution",fontsize=25)
    ax1.set_ylabel("Frequency",fontsize=25)
    ax1.set_xlabel("Outcomes",fontsize=25)
    ax1.set_xlim([0,3])

    ax1.set_xticks([])

    x = np.arange(1,3)
    dice = [varoi[0]/number,varoi[1]/number]
    f1,f2 = ax1.bar(x,dice)
    f1.set_facecolor("#8642f4")
    f2.set_facecolor("#518900")
    ax1.grid(which='both')
    ax1.grid(b=True,which='minor',alpha=0.2,linestyle='--',color='black')
    ax1.grid(which='major', alpha=0.2,linestyle='--',color='black')

    # Theoretical Probability
    ax2.set_title("Theoretical Distribution",fontsize=25)
    ax2.set_ylabel("Frequency",fontsize=25)
    ax2.set_xlabel("Outcomes",fontsize=25)
    x = np.arange(1,3)
    dice_exp = [theor_A,theor_C]
    f11,f21 = ax2.bar(x,dice_exp)
    f11.set_facecolor("#8642f4")
    f21.set_facecolor("#518900")

    ax2.set_xlim([0,3])

    ax2.set_xticks(["Even","Odd"])

    ax2.grid(which='both')
    ax2.grid(b=True,which='minor',alpha=0.2,linestyle='--',color='black')
    ax2.grid(b=True,which='major', alpha=0.2,linestyle='--',color='black')

    plt.ylim(0,number)

    plt.show()

def plot_fair_experiment(number):
    [varoi_1,varoi_2] = even_sum(number)
    theor_A = 18/36
    theor_C = 18/36

    even = varoi_1/number
    odd= varoi_2/number
    x = np.arange(1,3)

    labels = ['', '',  'Even Sum', '','Odd Sum']

    fig,(ax1,ax2) = plt.subplots(1,2,sharey=True,figsize=(15,8))

    # Experimental Probability
    ax1.set_title("Even or Odd Sum Probability Experiment:\nExperimental Probability",fontsize=20)
    ax1.set_ylabel("Probability",fontsize=25)
    ax1.set_xlabel("Events",fontsize=25)
    ax1.set_xlim([0,3])
    ax1.set_xticklabels(labels)
    ax1.grid(which='major', alpha=0.2,linestyle='--',color='black')

    f1,f2 = ax1.bar(x,[even,odd])
    f1.set_facecolor("#8642f4")
    f2.set_facecolor("#518900")

    # Theoretical Probability
    ax2.set_title("Even or Odd Sum Probability Experiment:\nTheoretical Probability",fontsize=20)
    ax2.set_ylabel("Probability",fontsize=25)
    ax2.set_xlabel("Events",fontsize=25)
    ax2.set_xlim([0,3])
    ax2.set_xticklabels(labels)
    ax2.grid(b=True,which='major', alpha=0.2,linestyle='--',color='black')

    dice_exp = [theor_A,theor_C]
    f11,f21 = ax2.bar(x,dice_exp)
    f11.set_facecolor("#8642f4")
    f21.set_facecolor("#518900")

    plt.ylim(0,1)
    plt.show()

interact_manual(plot_fair_experiment,number=widgets.IntSlider(
            value=5,
            min=1,
            max=10000,
            step=1,
            description='Total Number of Trials',
            disabled=False,
            continuous_update=False,
            orientation='horizontal',
            readout=True,
            readout_format='d',
            style =style
));

### Question 8

How does the experimental probability of each event change as we increase the number of trials? Use the textbox below to record your answers.

from ipywidgets import widgets as w
from ipywidgets import Button, Layout
from IPython.display import display, Javascript, Markdown

def rerun_cell( b ):

    display(Javascript('IPython.notebook.execute_cell_range(IPython.notebook.get_selected_index()+1,IPython.notebook.get_selected_index()+2)'))
style = {'description_width': 'initial'}

question8_text = w.Textarea( value='', placeholder='Write your answer here. Press Record Answer when you finish.', description='', disabled=False , layout=Layout(width='100%', height='75px') )
question8_button = w.Button(button_style='info',description="Record Answer", layout=Layout(width='15%', height='30px'))

display(question8_text)
display(question8_button)

question8_button.on_click( rerun_cell )

question8_input = question8_text.value

if(question8_input != ''):

    question8_text.close()
    question8_button.close()
    display(Markdown("### Your answer for Question 8: Conclusions"))
    display(Markdown(question8_input))

As the number of trials increases, we observe that the experimental probability of each event approaches the corresponding theoretical probability. This is known as the "Law of Large Numbers".

Conclusion

In this notebook we learned what a probability experiment is, what the sample space, events and outcome associated to a given probability experiment are. We learned how to express probabilities as ratios, fractions and percents.

We learned what independent probability events are and introduced the concept of a fair game.

We learned what theoretical and experimental probability are and with the help of an interactive exercise, we learned that as the number of trials increases, experimental probability approaches theoretical probability.

[![Callysto.ca License](https://github.com/callysto/curriculum-notebooks/blob/master/callysto-notebook-banner-bottom.jpg?raw=true)](https://github.com/callysto/curriculum-notebooks/blob/master/LICENSE.md)


#!/usr/bin/env python
# coding: utf-8

# # TV Script Generation
#
# In this project, you'll generate your own [Seinfeld](https://en.wikipedia.org/wiki/Seinfeld) TV scripts using RNNs.  You'll be using part of the [Seinfeld dataset](https://www.kaggle.com/thec03u5/seinfeld-chronicles#scripts.csv) of scripts from 9 seasons.  The Neural Network you'll build will generate a new ,"fake" TV script, based on patterns it recognizes in this training data.
#
# ## Get the Data
#
# The data is already provided for you in `./data/Seinfeld_Scripts.txt` and you're encouraged to open that file and look at the text.
# >* As a first step, we'll load in this data and look at some samples.
# * Then, you'll be tasked with defining and training an RNN to generate a new script!

# In[1]:

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# load in data
import helper
data_dir = './data/Seinfeld_Scripts.txt'
text = helper.load_data(data_dir)

# ## Explore the Data
# Play around with `view_line_range` to view different parts of the data. This will give you a sense of the data you'll be working with. You can see, for example, that it is all lowercase text, and each new line of dialogue is separated by a newline character `\n`.

# In[2]:

view_line_range = (0, 10)

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
import numpy as np

print('Dataset Stats')
print('Roughly the number of unique words: {}'.format(len({word: None for word in text.split()})))

lines = text.split('\n')
print('Number of lines: {}'.format(len(lines)))
word_count_line = [len(line.split()) for line in lines]
print('Average number of words in each line: {}'.format(np.average(word_count_line)))

print()
print('The lines {} to {}:'.format(*view_line_range))
print('\n'.join(text.split('\n')[view_line_range[0]:view_line_range[1]]))

# ---
# ## Implement Pre-processing Functions
# The first thing to do to any dataset is pre-processing.  Implement the following pre-processing functions below:
# - Lookup Table
# - Tokenize Punctuation
#
# ### Lookup Table
# To create a word embedding, you first need to transform the words to ids.  In this function, create two dictionaries:
# - Dictionary to go from the words to an id, we'll call `vocab_to_int`
# - Dictionary to go from the id to word, we'll call `int_to_vocab`
#
# Return these dictionaries in the following **tuple** `(vocab_to_int, int_to_vocab)`

# In[11]:

import problem_unittests as tests
from collections import Counter
def create_lookup_tables(text):
    """
    Create lookup tables for vocabulary
    :param text: The text of tv scripts split into words
    :return: A tuple of dicts (vocab_to_int, int_to_vocab)
    """
    # TODO: Implement Function
    c=Counter(text)
    vocabs=sorted(c, key=c.get, reverse=True)
    vocab_to_int={vocab: ii for ii, vocab in enumerate(vocabs)}
    int_to_vocab={ii:vocab for vocab, ii in vocab_to_int.items() }
    # return tuple
    return (vocab_to_int, int_to_vocab)

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_create_lookup_tables(create_lookup_tables)

# ### Tokenize Punctuation
# We'll be splitting the script into a word array using spaces as delimiters.  However, punctuations like periods and exclamation marks can create multiple ids for the same word. For example, "bye" and "bye!" would generate two different word ids.
#
# Implement the function `token_lookup` to return a dict that will be used to tokenize symbols like "!" into "||Exclamation_Mark||".  Create a dictionary for the following symbols where the symbol is the key and value is the token:
# - Period ( **.** )
# - Comma ( **,** )
# - Quotation Mark ( **"** )
# - Semicolon ( **;** )
# - Exclamation mark ( **!** )
# - Question mark ( **?** )
# - Left Parentheses ( **(** )
# - Right Parentheses ( **)** )
# - Dash ( **-** )
# - Return ( **\n** )
#
# This dictionary will be used to tokenize the symbols and add the delimiter (space) around it.  This separates each symbols as its own word, making it easier for the neural network to predict the next word. Make sure you don't use a value that could be confused as a word; for example, instead of using the value "dash", try using something like "||dash||".

# In[12]:

def token_lookup():
    """
    Generate a dict to turn punctuation into a token.
    :return: Tokenized dictionary where the key is the punctuation and the value is the token
    """
    # TODO: Implement Function
    token={'.': "||Period||",',':"||Comma||",'"':'||Quotation_Mark||',';':"||Semicolon||",
            '!': "||Exclamation_mark||",'?': "||Question_mark||",'(':"Left_Parentheses",
          ')':"Right_Parentheses","-":"||Dash||",'\n':"||Return||"
          }
    return token

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_tokenize(token_lookup)

# ## Pre-process all the data and save it
#
# Running the code cell below will pre-process all the data and save it to file. You're encouraged to look at the code for `preprocess_and_save_data` in the `helpers.py` file to see what it's doing in detail, but you do not need to change this code.

# In[13]:

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# pre-process training data
helper.preprocess_and_save_data(data_dir, token_lookup, create_lookup_tables)

# # Check Point
# This is your first checkpoint. If you ever decide to come back to this notebook or have to restart the notebook, you can start from here. The preprocessed data has been saved to disk.

# In[9]:

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import helper
import numpy as np
import problem_unittests as tests

int_text, vocab_to_int, int_to_vocab, token_dict = helper.load_preprocess()

# ## Build the Neural Network
# In this section, you'll build the components necessary to build an RNN by implementing the RNN Module and forward and backpropagation functions.
#
# ### Check Access to GPU

# In[10]:

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import torch

# Check for a GPU
train_on_gpu = torch.cuda.is_available()
if not train_on_gpu:
    print('No GPU found. Please use a GPU to train your neural network.')

# ## Input
# Let's start with the preprocessed input data. We'll use [TensorDataset](http://pytorch.org/docs/master/data.html#torch.utils.data.TensorDataset) to provide a known format to our dataset; in combination with [DataLoader](http://pytorch.org/docs/master/data.html#torch.utils.data.DataLoader), it will handle batching, shuffling, and other dataset iteration functions.
#
# You can create data with TensorDataset by passing in feature and target tensors. Then create a DataLoader as usual.
# ```
# data = TensorDataset(feature_tensors, target_tensors)
# data_loader = torch.utils.data.DataLoader(data,
#                                           batch_size=batch_size)
# ```
#
# ### Batching
# Implement the `batch_data` function to batch `words` data into chunks of size `batch_size` using the `TensorDataset` and `DataLoader` classes.
#
# >You can batch words using the DataLoader, but it will be up to you to create `feature_tensors` and `target_tensors` of the correct size and content for a given `sequence_length`.
#
# For example, say we have these as input:
# ```
# words = [1, 2, 3, 4, 5, 6, 7]
# sequence_length = 4
# ```
#
# Your first `feature_tensor` should contain the values:
# ```
# [1, 2, 3, 4]
# ```
# And the corresponding `target_tensor` should just be the next "word"/tokenized word value:
# ```
# 5
# ```
# This should continue with the second `feature_tensor`, `target_tensor` being:
# ```
# [2, 3, 4, 5]  # features
# 6             # target
# ```

# In[53]:

from torch.utils.data import TensorDataset, DataLoader

def batch_data(words, sequence_length, batch_size):
    """
    Batch the neural network data using DataLoader
    :param words: The word ids of the TV scripts
    :param sequence_length: The sequence length of each batch
    :param batch_size: The size of each batch; the number of sequences in a batch
    :return: DataLoader with batched data
    """
    # TODO: Implement function
    seq_batch=sequence_length*batch_size
    nbatch=len(words)//(seq_batch)
    words=words[:(seq_batch*nbatch)]
    inputs,target=[],[]
    for ii in range(len(words)-sequence_length):
#         print(words_looped[ii,ii+sequence_length])
        inputs.append(words[ii:(ii+sequence_length)])
        target.append(words[ii+sequence_length])
#         print(ii,inputs)
    data=TensorDataset(torch.tensor(inputs,dtype=torch.long), torch.tensor(target, dtype=torch.long))
    dataloader=DataLoader(data, batch_size=batch_size)
    # return a dataloader
    return dataloader

# there is no test for this function, but you are encouraged to create
# print statements and tests of your own

# ### Test your dataloader
#
# You'll have to modify this code to test a batching function, but it should look fairly similar.
#
# Below, we're generating some test text data and defining a dataloader using the function you defined, above. Then, we are getting some sample batch of inputs `sample_x` and targets `sample_y` from our dataloader.
#
# Your code should return something like the following (likely in a different order, if you shuffled your data):
#
# ```
# torch.Size([10, 5])
# tensor([[ 28,  29,  30,  31,  32],
#         [ 21,  22,  23,  24,  25],
#         [ 17,  18,  19,  20,  21],
#         [ 34,  35,  36,  37,  38],
#         [ 11,  12,  13,  14,  15],
#         [ 23,  24,  25,  26,  27],
#         [  6,   7,   8,   9,  10],
#         [ 38,  39,  40,  41,  42],
#         [ 25,  26,  27,  28,  29],
#         [  7,   8,   9,  10,  11]])
#
# torch.Size([10])
# tensor([ 33,  26,  22,  39,  16,  28,  11,  43,  30,  12])
# ```
#
# ### Sizes
# Your sample_x should be of size `(batch_size, sequence_length)` or (10, 5) in this case and sample_y should just have one dimension: batch_size (10).
#
# ### Values
#
# You should also notice that the targets, sample_y, are the *next* value in the ordered test_text data. So, for an input sequence `[ 28,  29,  30,  31,  32]` that ends with the value `32`, the corresponding output should be `33`.

# In[57]:

# test dataloader

test_text = list(range(50))
t_loader = batch_data(test_text, sequence_length=5, batch_size=10)

data_iter = iter(t_loader)
sample_x, sample_y = data_iter.next()

print(sample_x.shape)
print(sample_x)
print()
print(sample_y.shape)
print(sample_y)

# ---
# ## Build the Neural Network
# Implement an RNN using PyTorch's [Module class](http://pytorch.org/docs/master/nn.html#torch.nn.Module). You may choose to use a GRU or an LSTM. To complete the RNN, you'll have to implement the following functions for the class:
#  - `__init__` - The initialize function.
#  - `init_hidden` - The initialization function for an LSTM/GRU hidden state
#  - `forward` - Forward propagation function.
#
# The initialize function should create the layers of the neural network and save them to the class. The forward propagation function will use these layers to run forward propagation and generate an output and a hidden state.
#
# **The output of this model should be the *last* batch of word scores** after a complete sequence has been processed. That is, for each input sequence of words, we only want to output the word scores for a single, most likely, next word.
#
# ### Hints
#
# 1. Make sure to stack the outputs of the lstm to pass to your fully-connected layer, you can do this with `lstm_output = lstm_output.contiguous().view(-1, self.hidden_dim)`
# 2. You can get the last batch of word scores by shaping the output of the final, fully-connected layer like so:
#
# ```
# # reshape into (batch_size, seq_length, output_size)
# output = output.view(batch_size, -1, self.output_size)
# # get last batch
# out = output[:, -1]
# ```

# In[123]:

import torch.nn as nn

class RNN(nn.Module):

    def __init__(self, vocab_size, output_size, embedding_dim, hidden_dim, n_layers, dropout=0.5):
        """
        Initialize the PyTorch RNN Module
        :param vocab_size: The number of input dimensions of the neural network (the size of the vocabulary)
        :param output_size: The number of output dimensions of the neural network
        :param embedding_dim: The size of embeddings, should you choose to use them
        :param hidden_dim: The size of the hidden layer outputs
        :param dropout: dropout to add in between LSTM/GRU layers
        """
        super(RNN, self).__init__()
        # TODO: Implement function
        # set class variables
        self.output_size = output_size
        self.n_layers = n_layers
        self.hidden_dim = hidden_dim

        # define model layers
        self.embed=nn.Embedding(vocab_size, embedding_dim)
        self.lstm=nn.LSTM(embedding_dim, hidden_dim, n_layers,dropout=dropout,batch_first=True)

        self.dropout=nn.Dropout(dropout)
        self.fc=nn.Linear(hidden_dim, output_size)

        self.sig = nn.Sigmoid()

    def forward(self, nn_input, hidden):
        """
        Forward propagation of the neural network
        :param nn_input: The input to the neural network
        :param hidden: The hidden state
        :return: Two Tensors, the output of the neural network and the latest hidden state
        """
        # TODO: Implement function
        batch_size= nn_input.size(0)
        embed=self.embed(nn_input)
        lstm_output, hidden=self.lstm(embed,hidden)
#         print(lstm_output.shape)
#         lstm_output=lstm_output[:,-1,:]
        lstm_output = lstm_output.contiguous().view(-1, self.hidden_dim)
#         lstm_output=self.dropout(lstm_output)
        fc_out=self.fc(lstm_output)
        fc_out = fc_out.view(batch_size, -1, self.output_size)
#         sig_out=self.sig(fc_out)
        out = fc_out[:, -1]
        # return one batch of output word scores and the hidden state
        return out, hidden

    def init_hidden(self, batch_size):
        '''
        Initialize the hidden state of an LSTM/GRU
        :param batch_size: The batch_size of the hidden state
        :return: hidden state of dims (n_layers, batch_size, hidden_dim)
        '''
        # Implement function
        weight =next(self.parameters())
        if (train_on_gpu):
            hidden = (weight.new(self.n_layers, batch_size, self.hidden_dim).zero_().cuda(),
                  weight.new(self.n_layers, batch_size, self.hidden_dim).zero_().cuda())
        else:
            hidden = (weight.new(self.n_layers, batch_size, self.hidden_dim).zero_(),
                      weight.new(self.n_layers, batch_size, self.hidden_dim).zero_())

        return hidden

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_rnn(RNN, train_on_gpu)

# ### Define forward and backpropagation
#
# Use the RNN class you implemented to apply forward and back propagation. This function will be called, iteratively, in the training loop as follows:
# ```
# loss = forward_back_prop(decoder, decoder_optimizer, criterion, inp, target)
# ```
#
# And it should return the average loss over a batch and the hidden state returned by a call to `RNN(inp, hidden)`. Recall that you can get this loss by computing it, as usual, and calling `loss.item()`.
#
# **If a GPU is available, you should move your data to that GPU device, here.**

# In[124]:

def forward_back_prop(rnn, optimizer, criterion, inp, target, hidden):
    """
    Forward and backward propagation on the neural network
    :param rnn: The PyTorch Module that holds the neural network
    :param optimizer: The PyTorch optimizer for the neural network
    :param criterion: The PyTorch loss function
    :param inp: A batch of input to the neural network
    :param target: The target output for the batch of input
    :return: The loss and the latest hidden state Tensor
    """

    # TODO: Implement Function
    # move data to GPU, if available
    if train_on_gpu:
        inp, target=inp.cuda(),target.cuda()
        rnn.cuda()

    h = tuple([each.data for each in hidden])
    rnn.zero_grad()

#     hidden = tuple([each.data for each in hidden])
    out,hidden = rnn(inp,h)
    loss=criterion(out,target)

    # perform backpropagation and optimization
    loss.backward()
    optimizer.step()
    nn.utils.clip_grad_norm_(rnn.parameters(), 5)
    # return the loss over a batch and the hidden state produced by our model
    return loss.item(), hidden

# Note that these tests aren't completely extensive.
# they are here to act as general checks on the expected outputs of your functions
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_forward_back_prop(RNN, forward_back_prop, train_on_gpu)

# ## Neural Network Training
#
# With the structure of the network complete and data ready to be fed in the neural network, it's time to train it.
#
# ### Train Loop
#
# The training loop is implemented for you in the `train_decoder` function. This function will train the network over all the batches for the number of epochs given. The model progress will be shown every number of batches. This number is set with the `show_every_n_batches` parameter. You'll set this parameter along with other parameters in the next section.

# In[125]:

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""

def train_rnn(rnn, batch_size, optimizer, criterion, n_epochs, show_every_n_batches=100):
    batch_losses = []

    rnn.train()

    print("Training for %d epoch(s)..." % n_epochs)
    for epoch_i in range(1, n_epochs + 1):

        # initialize hidden state
        hidden = rnn.init_hidden(batch_size)

        for batch_i, (inputs, labels) in enumerate(train_loader, 1):

            # make sure you iterate over completely full batches, only
            n_batches = len(train_loader.dataset)//batch_size
            if(batch_i > n_batches):
                break
            # forward, back prop
            loss, hidden = forward_back_prop(rnn, optimizer, criterion, inputs, labels, hidden)
            # record loss
            batch_losses.append(loss)
#             print(loss)
            # printing loss stats
            if batch_i % show_every_n_batches == 0:
                print('E{}L{:0.2f}'.format(
                    epoch_i, np.average(batch_losses)), end=' ')
                batch_losses = []

    # returns a trained rnn
    return rnn

# ### Hyperparameters
#
# Set and train the neural network with the following parameters:
# - Set `sequence_length` to the length of a sequence.
# - Set `batch_size` to the batch size.
# - Set `num_epochs` to the number of epochs to train for.
# - Set `learning_rate` to the learning rate for an Adam optimizer.
# - Set `vocab_size` to the number of unique tokens in our vocabulary.
# - Set `output_size` to the desired size of the output.
# - Set `embedding_dim` to the embedding dimension; smaller than the vocab_size.
# - Set `hidden_dim` to the hidden dimension of your RNN.
# - Set `n_layers` to the number of layers/cells in your RNN.
# - Set `show_every_n_batches` to the number of batches at which the neural network should print progress.
#
# If the network isn't getting the desired results, tweak these parameters and/or the layers in the `RNN` class.

# In[129]:

# Data params
# Sequence Length
sequence_length = 10  # of words in a sequence
# Batch Size
batch_size = 128

# data loader - do not change"
train_loader = batch_data(int_text, sequence_length, batch_size)

# In[130]:

# Training parameters
# Number of Epochs
num_epochs = 10
# Learning Rate
learning_rate = 0.001

# Model parameters
# Vocab size
vocab_size = len(vocab_to_int)
# Output size
output_size = vocab_size
# Embedding Dimension
embedding_dim = 300
# Hidden Dimension
hidden_dim = 256
# Number of RNN Layers
n_layers = 2

# Show stats for every n number of batches
show_every_n_batches = 2000
print(vocab_size)

# ### Train
# In the next cell, you'll train the neural network on the pre-processed data.  If you have a hard time getting a good loss, you may consider changing your hyperparameters. In general, you may get better results with larger hidden and n_layer dimensions, but larger models take a longer time to train.
# > **You should aim for a loss less than 3.5.**
#
# You should also experiment with different sequence lengths, which determine the size of the long range dependencies that a model can learn.

# In[131]:

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
train_on_gpu=True
# create model and move to gpu if available
rnn = RNN(vocab_size, output_size, embedding_dim, hidden_dim, n_layers, dropout=0.5)
if train_on_gpu:
    rnn.cuda()

# defining loss and optimization functions for training
optimizer = torch.optim.Adam(rnn.parameters(), lr=learning_rate)
criterion = nn.CrossEntropyLoss()

# training the model
trained_rnn = train_rnn(rnn, batch_size, optimizer, criterion, num_epochs, show_every_n_batches)

# saving the trained model
helper.save_model('./save/trained_rnn', trained_rnn)
print('Model Trained and Saved')

# In[ ]:

# E1L4.86 E2L4.68 E3L4.59 E4L4.58 E5L4.58 E6L4.58
# E1L5.00 E1L4.73 E1L4.64 E2L4.46 E2L4.39 E2L4.38 E3L4.32 E3L4.30 E3L4.31 E4L4.27 E4L4.26 E4L4.25 E5L4.23 E5L4.23 E5L4.22
# E6L4.21 E6L4.21 E6L4.20 E7L4.19 E7L4.20 E7L4.18 E8L4.18 E8L4.19 E8L4.17
# E9L4.17 E9L4.17 E9L4.16 E10L4.16 E10L4.16 E10L4.15 E11L4.15 E11L4.16 E11L4.14 E12L4.15 E12L4.15 E12L4.14

# In[ ]:

# E1L4.77 E2L4.44 E3L4.31 E4L4.26 E5L4.23 E6L4.21 E7L4.19 E8L4.18 E9L4.17 E10L4.16
# E1L5.17 E1L4.71 E1L4.74 E1L4.64 E1L4.54 E1L4.66 E2L4.51 E2L4.28 E2L4.41 E2L4.35 E2L4.30 E2L4.44
# E1L5.13 E1L4.71 E1L4.73 E1L4.63 E1L4.54 E1L4.66 E2L4.51 E2L4.28 E2L4.41

# ### Question: How did you decide on your model hyperparameters?
# For example, did you try different sequence_lengths and find that one size made the model converge faster? What about your hidden_dim and n_layers; how did you decide on those?

# **Answer:**
# - n_layers: from FeiFei Li's research, we choose 2 to be the optimized LSTM layer.
# - sequence_lengths: since a typical sentence contains words ranges from 7-16, we tested 10 ,16, 20. 10 converges faster.
# - hidden_dim: values of 128, 256, 512 are tried, 256 converges the best.
# - embed dim: we fisrt tried values of 768, 1024 are tried since these are the embed dimension for BERT and GLOVE. None of them converges fast. Thus, we pick 300 as in previous rnn assignments.
# - learning rate: values of 1, 0.1, 0.01,0.003,0.001 are tried, 0.001 converges faster.
# - since batchsize and nepoch have the same effect on training, we fixed batch size to be 128 and set the number of epochs to be 10 to avoid overfitting.

# ---
# # Checkpoint
#
# After running the above training cell, your model will be saved by name, `trained_rnn`, and if you save your notebook progress, **you can pause here and come back to this code at another time**. You can resume your progress by running the next cell, which will load in our word:id dictionaries _and_ load in your saved model by name!

# In[132]:

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import torch
import helper
import problem_unittests as tests

_, vocab_to_int, int_to_vocab, token_dict = helper.load_preprocess()
trained_rnn = helper.load_model('./save/trained_rnn')

# ## Generate TV Script
# With the network trained and saved, you'll use it to generate a new, "fake" Seinfeld TV script in this section.
#
# ### Generate Text
# To generate the text, the network needs to start with a single word and repeat its predictions until it reaches a set length. You'll be using the `generate` function to do this. It takes a word id to start with, `prime_id`, and generates a set length of text, `predict_len`. Also note that it uses topk sampling to introduce some randomness in choosing the most likely next word, given an output set of word scores!

# In[133]:

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
import torch.nn.functional as F

def generate(rnn, prime_id, int_to_vocab, token_dict, pad_value, predict_len=100):
    """
    Generate text using the neural network
    :param decoder: The PyTorch Module that holds the trained neural network
    :param prime_id: The word id to start the first prediction
    :param int_to_vocab: Dict of word id keys to word values
    :param token_dict: Dict of puncuation tokens keys to puncuation values
    :param pad_value: The value used to pad a sequence
    :param predict_len: The length of text to generate
    :return: The generated text
    """
    rnn.eval()

    # create a sequence (batch_size=1) with the prime_id
    current_seq = np.full((1, sequence_length), pad_value)
    current_seq[-1][-1] = prime_id
    predicted = [int_to_vocab[prime_id]]

    for _ in range(predict_len):
        if train_on_gpu:
            current_seq = torch.LongTensor(current_seq).cuda()
        else:
            current_seq = torch.LongTensor(current_seq)

        # initialize the hidden state
        hidden = rnn.init_hidden(current_seq.size(0))

        # get the output of the rnn
        output, _ = rnn(current_seq, hidden)

        # get the next word probabilities
        p = F.softmax(output, dim=1).data
        if(train_on_gpu):
            p = p.cpu() # move to cpu

        # use top_k sampling to get the index of the next word
        top_k = 5
        p, top_i = p.topk(top_k)
        top_i = top_i.numpy().squeeze()

        # select the likely next word index with some element of randomness
        p = p.numpy().squeeze()
        word_i = np.random.choice(top_i, p=p/p.sum())

        # retrieve that word from the dictionary
        word = int_to_vocab[word_i]
        predicted.append(word)

        if(train_on_gpu):
            current_seq = current_seq.cpu() # move to cpu
        # the generated word becomes the next "current sequence" and the cycle can continue
        if train_on_gpu:
            current_seq = current_seq.cpu()
        current_seq = np.roll(current_seq, -1, 1)
        current_seq[-1][-1] = word_i

    gen_sentences = ' '.join(predicted)

    # Replace punctuation tokens
    for key, token in token_dict.items():
        ending = ' ' if key in ['\n', '(', '"'] else ''
        gen_sentences = gen_sentences.replace(' ' + token.lower(), key)
    gen_sentences = gen_sentences.replace('\n ', '\n')
    gen_sentences = gen_sentences.replace('( ', '(')

    # return all the sentences
    return gen_sentences

# ### Generate a New Script
# It's time to generate the text. Set `gen_length` to the length of TV script you want to generate and set `prime_word` to one of the following to start the prediction:
# - "jerry"
# - "elaine"
# - "george"
# - "kramer"
#
# You can set the prime word to _any word_ in our dictionary, but it's best to start with a name for generating a TV script. (You can also start with any other names you find in the original text file!)

# In[134]:

# run the cell multiple times to get different results!
gen_length = 400 # modify the length to your preference
prime_word = 'jerry' # name for starting the script

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
pad_word = helper.SPECIAL_WORDS['PADDING']
generated_script = generate(trained_rnn, vocab_to_int[prime_word + ':'], int_to_vocab, token_dict, vocab_to_int[pad_word], gen_length)
print(generated_script)

# #### Save your favorite scripts
#
# Once you have a script that you like (or find interesting), save it to a text file!

# In[135]:

# save script to a text file
f =  open("generated_script_1.txt","w")
f.write(generated_script)
f.close()

# # The TV Script is Not Perfect
# It's ok if the TV script doesn't make perfect sense. It should look like alternating lines of dialogue, here is one such example of a few generated lines.
#
# ### Example generated script
#
# >jerry: what about me?
# >
# >jerry: i don't have to wait.
# >
# >kramer:(to the sales table)
# >
# >elaine:(to jerry) hey, look at this, i'm a good doctor.
# >
# >newman:(to elaine) you think i have no idea of this...
# >
# >elaine: oh, you better take the phone, and he was a little nervous.
# >
# >kramer:(to the phone) hey, hey, jerry, i don't want to be a little bit.(to kramer and jerry) you can't.
# >
# >jerry: oh, yeah. i don't even know, i know.
# >
# >jerry:(to the phone) oh, i know.
# >
# >kramer:(laughing) you know...(to jerry) you don't know.
#
# You can see that there are multiple characters that say (somewhat) complete sentences, but it doesn't have to be perfect! It takes quite a while to get good results, and often, you'll have to use a smaller vocabulary (and discard uncommon words), or get more data.  The Seinfeld dataset is about 3.4 MB, which is big enough for our purposes; for script generation you'll want more than 1 MB of text, generally.
#
# # Submitting This Project
# When submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as "dlnd_tv_script_generation.ipynb" and save another copy as an HTML file by clicking "File" -> "Download as.."->"html". Include the "helper.py" and "problem_unittests.py" files in your submission. Once you download these files, compress them into one zip file for submission.

# In[ ]:


"""
1. How to import pandas and check the version?
"""
"""

"""

import numpy as np  # optional
import pandas as pd
print(pd.__version__)
print(pd.show_versions(as_json=True))


import unittest

import numpy as np
import scipy.io as sp_io

import two_d.geometry as geo

class TestGeometry(unittest.TestCase):
    """Checks the correctness of functions in geometry.py."""

    def setUp(self):
        """Initializes common variables in the test."""
        super().setUp()

        self.mesh = sp_io.loadmat('two_d/test_data/maxwell.mat')

    def test_connect_provides_correct_connectivity_matrices(self):
        """Checks if the connect function generates correct 2D connectivity."""
        e_to_e, e_to_f = geo.connect(self.mesh['EToV'])

        with self.subTest(name='EToE'):
            np.testing.assert_array_equal(self.mesh['EToE'], e_to_e)

        with self.subTest(name='EToF'):
            np.testing.assert_array_equal(self.mesh['EToF'], e_to_f)

if __name__ == '__main__':
    unittest.main()