<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Examples and Exercises from Think Stats, 2nd Edition
http
Step1:  Given a list of values, there are several ways to count the frequency of each value.
Step2:  You can use a Python dictionary
Step3:  You can use a Counter (which is a dictionary with additional methods)
Step4:  Or you can use the Hist object provided by thinkstats2
Step5:  Hist provides Freq, which looks up the frequency of a value.
Step6:  You can also use the bracket operator, which does the same thing.
Step7:  If the value does not appear, it has frequency 0.
Step8:  The Values method returns the values
Step9:  So you can iterate the values and their frequencies like this
Step10:  Or you can use the Items method
Step11:  thinkplot is a wrapper for matplotlib that provides functions that work with the objects in thinkstats2.
For example Hist plots the values and their frequencies as a bar graph.
Config takes parameters that label the x and y axes, among other things.
Step12:  As an example, I'll replicate some of the figures from the book.
First, I'll load the data from the pregnancy file and select the records for live births.
Step13:  Here's the histogram of birth weights in pounds.  Notice that Hist works with anything iterable, including a Pandas Series.  The label attribute appears in the legend when you plot the Hist.
Step14:  Before plotting the ages, I'll apply floor to round down
Step15:  As an exercise, plot the histogram of pregnancy lengths (column prglngth).
Step16:  Hist provides smallest, which select the lowest values and their frequencies.
Step17:  Use Largest to display the longest pregnancy lengths.
Step18:  From live births, we can selection first babies and others using birthord, then compute histograms of pregnancy length for the two groups.
Step19:  We can use width and align to plot two histograms side-by-side.
Step20:  Series provides methods to compute summary statistics
Step21:  Here are the mean and standard deviation
Step22:  As an exercise, confirm that std is the square root of var
Step23:  Here's are the mean pregnancy lengths for first babies and others
Step24:  And here's the difference (in weeks)
Step26:  This functon computes the Cohen effect size, which is the difference in means expressed in number of standard deviations
Step27:  Compute the Cohen effect size for the difference in pregnancy length for first babies and others.
Step28:  Exercises
Using the variable totalwgt_lb, investigate whether first babies are lighter or heavier than others. 
Compute Cohen’s effect size to quantify the difference between the groups.  How does it compare to the difference in pregnancy length?
Step29:  For the next few exercises, we'll load the respondent file
Step30:  Make a histogram of <tt>totincr</tt> the total income for the respondent's family.  To interpret the codes see the codebook.
Step31:  Make a histogram of <tt>age_r</tt>, the respondent's age at the time of interview.
Step32:  Make a histogram of <tt>numfmhh</tt>, the number of people in the respondent's household.
Step33:  Make a histogram of <tt>parity</tt>, the number of children borne by the respondent.  How would you describe this distribution?
Step34:  Use Hist.Largest to find the largest values of <tt>parity</tt>.
Step35:  Let's investigate whether people with higher income have higher parity.  Keep in mind that in this study, we are observing different people at different times during their lives, so this data is not the best choice for answering this question.  But for now let's take it at face value.
Use <tt>totincr</tt> to select the respondents with the highest income (level 14).  Plot the histogram of <tt>parity</tt> for just the high income respondents.
Step36:  Find the largest parities for high income respondents.
Step37:  Compare the mean <tt>parity</tt> for high income respondents and others.
Step38:  Compute the Cohen effect size for this difference.  How does it compare with the difference in pregnancy length for first babies and others?
<ASSISTANT_TASK:>
Python Code:
from __future__ import print_function, division
%matplotlib inline
import numpy as np
import nsfg
import first
Explanation: Examples and Exercises from Think Stats, 2nd Edition
http://thinkstats2.com
Copyright 2016 Allen B. Downey
MIT License: https://opensource.org/licenses/MIT
End of explanation
t = [1, 2, 2, 3, 5]
Explanation: Given a list of values, there are several ways to count the frequency of each value.
End of explanation
hist = {}
for x in t:
    hist[x] = hist.get(x, 0) + 1
    
hist
Explanation: You can use a Python dictionary:
End of explanation
from collections import Counter
counter = Counter(t)
counter
Explanation: You can use a Counter (which is a dictionary with additional methods):
End of explanation
import thinkstats2
hist = thinkstats2.Hist([1, 2, 2, 3, 5])
hist
Explanation: Or you can use the Hist object provided by thinkstats2:
End of explanation
hist.Freq(2)
Explanation: Hist provides Freq, which looks up the frequency of a value.
End of explanation
hist[2]
Explanation: You can also use the bracket operator, which does the same thing.
End of explanation
hist[4]
Explanation: If the value does not appear, it has frequency 0.
End of explanation
hist.Values()
Explanation: The Values method returns the values:
End of explanation
for val in sorted(hist.Values()):
    print(val, hist[val])
Explanation: So you can iterate the values and their frequencies like this:
End of explanation
for val, freq in hist.Items():
     print(val, freq)
Explanation: Or you can use the Items method:
End of explanation
import thinkplot
thinkplot.Hist(hist)
thinkplot.Config(xlabel='value', ylabel='frequency')
Explanation: thinkplot is a wrapper for matplotlib that provides functions that work with the objects in thinkstats2.
For example Hist plots the values and their frequencies as a bar graph.
Config takes parameters that label the x and y axes, among other things.
End of explanation
preg = nsfg.ReadFemPreg()
live = preg[preg.outcome == 1]
Explanation: As an example, I'll replicate some of the figures from the book.
First, I'll load the data from the pregnancy file and select the records for live births.
End of explanation
hist = thinkstats2.Hist(live.birthwgt_lb, label='birthwgt_lb')
thinkplot.Hist(hist)
thinkplot.Config(xlabel='Birth weight (pounds)', ylabel='Count')
Explanation: Here's the histogram of birth weights in pounds.  Notice that Hist works with anything iterable, including a Pandas Series.  The label attribute appears in the legend when you plot the Hist.
End of explanation
ages = np.floor(live.agepreg)
hist = thinkstats2.Hist(ages, label='agepreg')
thinkplot.Hist(hist)
thinkplot.Config(xlabel='years', ylabel='Count')
Explanation: Before plotting the ages, I'll apply floor to round down:
End of explanation
# Solution
hist = thinkstats2.Hist(live.prglngth, label='prglngth')
thinkplot.Hist(hist)
thinkplot.Config(xlabel='weeks', ylabel='Count')
Explanation: As an exercise, plot the histogram of pregnancy lengths (column prglngth).
End of explanation
for weeks, freq in hist.Smallest(10):
    print(weeks, freq)
Explanation: Hist provides smallest, which select the lowest values and their frequencies.
End of explanation
# Solution
for weeks, freq in hist.Largest(10):
    print(weeks, freq)
Explanation: Use Largest to display the longest pregnancy lengths.
End of explanation
firsts = live[live.birthord == 1]
others = live[live.birthord != 1]
first_hist = thinkstats2.Hist(firsts.prglngth, label='first')
other_hist = thinkstats2.Hist(others.prglngth, label='other')
Explanation: From live births, we can selection first babies and others using birthord, then compute histograms of pregnancy length for the two groups.
End of explanation
width = 0.45
thinkplot.PrePlot(2)
thinkplot.Hist(first_hist, align='right', width=width)
thinkplot.Hist(other_hist, align='left', width=width)
thinkplot.Config(xlabel='weeks', ylabel='Count', xlim=[27, 46])
Explanation: We can use width and align to plot two histograms side-by-side.
End of explanation
mean = live.prglngth.mean()
var = live.prglngth.var()
std = live.prglngth.std()
Explanation: Series provides methods to compute summary statistics:
End of explanation
mean, std
Explanation: Here are the mean and standard deviation:
End of explanation
# Solution
np.sqrt(var) == std
Explanation: As an exercise, confirm that std is the square root of var:
End of explanation
firsts.prglngth.mean(), others.prglngth.mean()
Explanation: Here's are the mean pregnancy lengths for first babies and others:
End of explanation
firsts.prglngth.mean() - others.prglngth.mean()
Explanation: And here's the difference (in weeks):
End of explanation
def CohenEffectSize(group1, group2):
    Computes Cohen's effect size for two groups.
    
    group1: Series or DataFrame
    group2: Series or DataFrame
    
    returns: float if the arguments are Series;
             Series if the arguments are DataFrames
    
    diff = group1.mean() - group2.mean()
    var1 = group1.var()
    var2 = group2.var()
    n1, n2 = len(group1), len(group2)
    pooled_var = (n1 * var1 + n2 * var2) / (n1 + n2)
    d = diff / np.sqrt(pooled_var)
    return d
Explanation: This functon computes the Cohen effect size, which is the difference in means expressed in number of standard deviations:
End of explanation
# Solution
CohenEffectSize(firsts.prglngth, others.prglngth)
Explanation: Compute the Cohen effect size for the difference in pregnancy length for first babies and others.
End of explanation
# Solution
firsts.totalwgt_lb.mean(), others.totalwgt_lb.mean()
# Solution
CohenEffectSize(firsts.totalwgt_lb, others.totalwgt_lb)
Explanation: Exercises
Using the variable totalwgt_lb, investigate whether first babies are lighter or heavier than others. 
Compute Cohen’s effect size to quantify the difference between the groups.  How does it compare to the difference in pregnancy length?
End of explanation
resp = nsfg.ReadFemResp()
Explanation: For the next few exercises, we'll load the respondent file:
End of explanation
# Solution
hist = thinkstats2.Hist(resp.totincr)
thinkplot.Hist(hist, label='totincr')
thinkplot.Config(xlabel='income (category)', ylabel='Count')
Explanation: Make a histogram of <tt>totincr</tt> the total income for the respondent's family.  To interpret the codes see the codebook.
End of explanation
# Solution
hist = thinkstats2.Hist(resp.ager)
thinkplot.Hist(hist, label='ager')
thinkplot.Config(xlabel='age (years)', ylabel='Count')
Explanation: Make a histogram of <tt>age_r</tt>, the respondent's age at the time of interview.
End of explanation
# Solution
hist = thinkstats2.Hist(resp.numfmhh)
thinkplot.Hist(hist, label='numfmhh')
thinkplot.Config(xlabel='number of people', ylabel='Count')
Explanation: Make a histogram of <tt>numfmhh</tt>, the number of people in the respondent's household.
End of explanation
# Solution
# This distribution is positive-valued and skewed to the right.
hist = thinkstats2.Hist(resp.parity)
thinkplot.Hist(hist, label='parity')
thinkplot.Config(xlabel='parity', ylabel='Count')
Explanation: Make a histogram of <tt>parity</tt>, the number of children borne by the respondent.  How would you describe this distribution?
End of explanation
# Solution
hist.Largest(10)
Explanation: Use Hist.Largest to find the largest values of <tt>parity</tt>.
End of explanation
# Solution
rich = resp[resp.totincr == 14]
hist = thinkstats2.Hist(rich.parity)
thinkplot.Hist(hist, label='parity')
thinkplot.Config(xlabel='parity', ylabel='Count')
Explanation: Let's investigate whether people with higher income have higher parity.  Keep in mind that in this study, we are observing different people at different times during their lives, so this data is not the best choice for answering this question.  But for now let's take it at face value.
Use <tt>totincr</tt> to select the respondents with the highest income (level 14).  Plot the histogram of <tt>parity</tt> for just the high income respondents.
End of explanation
# Solution
hist.Largest(10)
Explanation: Find the largest parities for high income respondents.
End of explanation
# Solution
notrich = resp[resp.totincr < 14]
rich.parity.mean(), notrich.parity.mean()
Explanation: Compare the mean <tt>parity</tt> for high income respondents and others.
End of explanation
# Solution
# This effect is about 10 times stronger than the difference in pregnancy length.
# But remembering the design of the study, we should not make too much of this
# apparent effect.
CohenEffectSize(rich.parity, notrich.parity)
Explanation: Compute the Cohen effect size for this difference.  How does it compare with the difference in pregnancy length for first babies and others?
End of explanation
<END_TASK>