<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Hypothesis Testing
Copyright 2016 Allen Downey
License
Step1:  Part One
Suppose you observe an apparent difference between two groups and you want to check whether it might be due to chance.
As an example, we'll look at differences between first babies and others.  The first module provides code to read data from the National Survey of Family Growth (NSFG).
Step2:  We'll look at a couple of variables, including pregnancy length and birth weight.  The effect size we'll consider is the difference in the means.
Other examples might include a correlation between variables or a coefficient in a linear regression.  The number that quantifies the size of the effect is called the "test statistic".
Step3:  For the first example, I extract the pregnancy length for first babies and others.  The results are pandas Series objects.
Step4:  The actual difference in the means is 0.078 weeks, which is only 13 hours.
Step5:  The null hypothesis is that there is no difference between the groups.  We can model that by forming a pooled sample that includes first babies and others.
Step6:  Then we can simulate the null hypothesis by shuffling the pool and dividing it into two groups, using the same sizes as the actual sample.
Step7:  The result of running the model is two NumPy arrays with the shuffled pregnancy lengths
Step8:  Then we compute the same test statistic using the simulated data
Step9:  If we run the model 1000 times and compute the test statistic, we can see how much the test statistic varies under the null hypothesis.
Step10:  Here's the sampling distribution of the test statistic under the null hypothesis, with the actual difference in means indicated by a gray line.
Step11:  The p-value is the probability that the test statistic under the null hypothesis exceeds the actual value.
Step20:  In this case the result is about 15%, which means that even if there is no difference between the groups, it is plausible that we could see a sample difference as big as 0.078 weeks.
We conclude that the apparent effect might be due to chance, so we are not confident that it would appear in the general population, or in another sample from the same population.
STOP HERE
Part Two
We can take the pieces from the previous section and organize them in a class that represents the structure of a hypothesis test.
Step25:  HypothesisTest is an abstract parent class that encodes the template.  Child classes fill in the missing methods.  For example, here's the test from the previous section.
Step26:  Now we can run the test by instantiating a DiffMeansPermute object
Step27:  And we can plot the sampling distribution of the test statistic under the null hypothesis.
Step28:  Difference in standard deviation
Exercize 1
Step29:  Here's the code to test your solution to the previous exercise.
Step30:  Difference in birth weights
Now let's run DiffMeansPermute again to see if there is a difference in birth weight between first babies and others.
<ASSISTANT_TASK:>
Python Code:
%matplotlib inline
from __future__ import print_function, division
import numpy
import scipy.stats
import matplotlib.pyplot as pyplot
from ipywidgets import interact, interactive, fixed
import ipywidgets as widgets
import first
# seed the random number generator so we all get the same results
numpy.random.seed(19)
# some nicer colors from http://colorbrewer2.org/
COLOR1 = '#7fc97f'
COLOR2 = '#beaed4'
COLOR3 = '#fdc086'
COLOR4 = '#ffff99'
COLOR5 = '#386cb0'
Explanation: Hypothesis Testing
Copyright 2016 Allen Downey
License: Creative Commons Attribution 4.0 International
End of explanation
live, firsts, others = first.MakeFrames()
Explanation: Part One
Suppose you observe an apparent difference between two groups and you want to check whether it might be due to chance.
As an example, we'll look at differences between first babies and others.  The first module provides code to read data from the National Survey of Family Growth (NSFG).
End of explanation
def TestStatistic(data):
    group1, group2 = data
    test_stat = abs(group1.mean() - group2.mean())
    return test_stat
Explanation: We'll look at a couple of variables, including pregnancy length and birth weight.  The effect size we'll consider is the difference in the means.
Other examples might include a correlation between variables or a coefficient in a linear regression.  The number that quantifies the size of the effect is called the "test statistic".
End of explanation
group1 = firsts.prglngth
group2 = others.prglngth
Explanation: For the first example, I extract the pregnancy length for first babies and others.  The results are pandas Series objects.
End of explanation
actual = TestStatistic((group1, group2))
actual
Explanation: The actual difference in the means is 0.078 weeks, which is only 13 hours.
End of explanation
n, m = len(group1), len(group2)
pool = numpy.hstack((group1, group2))
Explanation: The null hypothesis is that there is no difference between the groups.  We can model that by forming a pooled sample that includes first babies and others.
End of explanation
def RunModel():
    numpy.random.shuffle(pool)
    data = pool[:n], pool[n:]
    return data
Explanation: Then we can simulate the null hypothesis by shuffling the pool and dividing it into two groups, using the same sizes as the actual sample.
End of explanation
RunModel()
Explanation: The result of running the model is two NumPy arrays with the shuffled pregnancy lengths:
End of explanation
TestStatistic(RunModel())
Explanation: Then we compute the same test statistic using the simulated data:
End of explanation
test_stats = numpy.array([TestStatistic(RunModel()) for i in range(1000)])
test_stats.shape
Explanation: If we run the model 1000 times and compute the test statistic, we can see how much the test statistic varies under the null hypothesis.
End of explanation
pyplot.vlines(actual, 0, 300, linewidth=3, color='0.8')
pyplot.hist(test_stats, color=COLOR5)
pyplot.xlabel('difference in means')
pyplot.ylabel('count')
None # It's interesting that these None's are significant to the notebook.
Explanation: Here's the sampling distribution of the test statistic under the null hypothesis, with the actual difference in means indicated by a gray line.
End of explanation
pvalue = sum(test_stats >= actual) / len(test_stats)
pvalue
Explanation: The p-value is the probability that the test statistic under the null hypothesis exceeds the actual value.
End of explanation
class HypothesisTest(object):
    Represents a hypothesis test.
    def __init__(self, data):
        Initializes.
        data: data in whatever form is relevant
        
        self.data = data
        self.MakeModel()
        self.actual = self.TestStatistic(data)
        self.test_stats = None
    def PValue(self, iters=1000):
        Computes the distribution of the test statistic and p-value.
        iters: number of iterations
        returns: float p-value
        
        self.test_stats = numpy.array([self.TestStatistic(self.RunModel()) 
                                       for _ in range(iters)])
        count = sum(self.test_stats >= self.actual)
        return count / iters
    def MaxTestStat(self):
        Returns the largest test statistic seen during simulations.
        
        return max(self.test_stats)
    def PlotHist(self, label=None):
        Draws a Cdf with vertical lines at the observed test stat.
        
        ys, xs, patches = pyplot.hist(ht.test_stats, color=COLOR4)
        pyplot.vlines(self.actual, 0, max(ys), linewidth=3, color='0.8')
        pyplot.xlabel('test statistic')
        pyplot.ylabel('count')
    def TestStatistic(self, data):
        Computes the test statistic.
        data: data in whatever form is relevant        
        
        raise UnimplementedMethodException()
    def MakeModel(self):
        Build a model of the null hypothesis.
        
        pass
    def RunModel(self):
        Run the model of the null hypothesis.
        returns: simulated data
        
        raise UnimplementedMethodException()
Explanation: In this case the result is about 15%, which means that even if there is no difference between the groups, it is plausible that we could see a sample difference as big as 0.078 weeks.
We conclude that the apparent effect might be due to chance, so we are not confident that it would appear in the general population, or in another sample from the same population.
STOP HERE
Part Two
We can take the pieces from the previous section and organize them in a class that represents the structure of a hypothesis test.
End of explanation
class DiffMeansPermute(HypothesisTest):
    Tests a difference in means by permutation.
    def TestStatistic(self, data):
        Computes the test statistic.
        data: data in whatever form is relevant        
        
        group1, group2 = data
        test_stat = abs(group1.mean() - group2.mean())
        return test_stat
    def MakeModel(self):
        Build a model of the null hypothesis.
        
        group1, group2 = self.data
        self.n, self.m = len(group1), len(group2)
        self.pool = numpy.hstack((group1, group2))
    def RunModel(self):
        Run the model of the null hypothesis.
        returns: simulated data
        
        numpy.random.shuffle(self.pool)
        data = self.pool[:self.n], self.pool[self.n:]
        return data
Explanation: HypothesisTest is an abstract parent class that encodes the template.  Child classes fill in the missing methods.  For example, here's the test from the previous section.
End of explanation
data = (firsts.prglngth, others.prglngth)
ht = DiffMeansPermute(data)
p_value = ht.PValue(iters=1000)
print('\nmeans permute pregnancy length')
print('p-value =', p_value)
print('actual =', ht.actual)
print('ts max =', ht.MaxTestStat())
Explanation: Now we can run the test by instantiating a DiffMeansPermute object:
End of explanation
ht.PlotHist()
Explanation: And we can plot the sampling distribution of the test statistic under the null hypothesis.
End of explanation
# Solution goes here
Explanation: Difference in standard deviation
Exercize 1: Write a class named DiffStdPermute that extends DiffMeansPermute and overrides TestStatistic to compute the difference in standard deviations.  Is the difference in standard deviations statistically significant?
End of explanation
data = (firsts.prglngth, others.prglngth)
ht = DiffStdPermute(data)
p_value = ht.PValue(iters=1000)
print('\nstd permute pregnancy length')
print('p-value =', p_value)
print('actual =', ht.actual)
print('ts max =', ht.MaxTestStat())
Explanation: Here's the code to test your solution to the previous exercise.
End of explanation
data = (firsts.totalwgt_lb.dropna(), others.totalwgt_lb.dropna())
ht = DiffMeansPermute(data)
p_value = ht.PValue(iters=1000)
print('\nmeans permute birthweight')
print('p-value =', p_value)
print('actual =', ht.actual)
print('ts max =', ht.MaxTestStat())
Explanation: Difference in birth weights
Now let's run DiffMeansPermute again to see if there is a difference in birth weight between first babies and others.
End of explanation
<END_TASK>