<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Chapter 2  Distributions
Histograms
The most common representation of a distribution is a histogram. which is a graph that shows the frequency of each value
Step1:  NSFG variables
Step2:  Histogram of pregnancy length in weeks
Step3:  Histogram of pregnancy lengths
Step4:  Summarizing distributions
Some of the characteristics we might want to report are
Step5:  Make a histogram of totincr the total income for the respondent's family.
Step6:  Make a histogram of age_r, the respondent's age at the time of interview.
Step7:  Use totincr to select the respondents with the highest income. Compute the distribution of parity for just the high income respondents.
Step8:  Compare the mean parity for high income respondents and others.
Step9:  Exercise 4
Using the variable totalwgt_lb, investigate whether first babies are lighter or heavier than others. Compute Cohen’s d to quantify the difference between the groups. How does it compare to the difference in pregnancy length?
<ASSISTANT_TASK:>
Python Code:
from matplotlib import pyplot as plt
%matplotlib inline  
import seaborn as sns
import numpy as np
import pandas as pd
import thinkstats2
import thinkplot
hist = thinkstats2.Hist([1, 2, 2, 3, 5])
hist
hist.Freq(2) # hist[2]
hist.Values()
thinkplot.Hist(hist)
thinkplot.Show(xlabel='value', ylabel='frequency')
Explanation: Chapter 2  Distributions
Histograms
The most common representation of a distribution is a histogram. which is a graph that shows the frequency of each value
End of explanation
import nsfg
Explanation: NSFG variables
End of explanation
preg = nsfg.ReadFemPreg()
live = preg[preg.outcome == 1]
hist = thinkstats2.Hist(live.birthwgt_lb, label='birthwgt_lb')
thinkplot.Hist(hist)
thinkplot.Show(xlabel='pounds', ylabel='frequency')
Explanation: Histogram of pregnancy length in weeks
End of explanation
firsts = live[live.birthord == 1]
others = live[live.birthord != 1]
firsts.prglngth.plot(kind='hist', width=2)
others.prglngth.plot(kind='hist', width=2)
Explanation: Histogram of pregnancy lengths
End of explanation
import thinkstats2
resp = thinkstats2.ReadStataDct('2002FemResp.dct').ReadFixedWidth('2002FemResp.dat.gz', compression='gzip')
Explanation: Summarizing distributions
Some of the characteristics we might want to report are:
- central tendency
- modes
- spread
- tails
- outliers
mean
$$\overline{x}= \frac{1}{n}\sum_{i}a_i$$
Variance
$$S^{2} = \frac{1}{n}\sum_i(x_{i}-\overline{x})^{2}$$
$x_{i}-\overline{x}$ is called the “deviation from the mean”
$\sqrt{S}$ is the standard deviation.
Pandas data structures provides methods to compute mean, variance and standard deviation:
```python
mean = live.prglngth.mean()
var = live.prglngth.var() # variance
std = live.prglngth.std() # standard deviation
```
Effect size
An effect size is a quantitative measure of the strength of an event.
One obvious choice is the difference in the means.
Another way to convey the size of the effect is to compare the difference between groups to the variability within groups.
Cohen's d
$$d = \frac{\overline{x_1} -\overline{x_2}}{s}$$
s is the “pooled standard deviation”
$$s=\sqrt{\frac{(n_1-1)S_1^2 + (n_2-1)S_2^2}{n_1 +n_2 -2}}$$
$n_i$ is the sample size of $x_i$, $S_i$ is the variance.
Reporting results
Who
A scientist might be interested in any (real) effect, no matter how small.
A doctor might only care about effects that are clinically significant.
How
Goals
Exercise2
End of explanation
resp.totincr.plot.hist(bins=range(17))
Explanation: Make a histogram of totincr the total income for the respondent's family.
End of explanation
resp.ager.plot.hist(bins=range(15,46))
Explanation: Make a histogram of age_r, the respondent's age at the time of interview.
End of explanation
rich = resp[resp.totincr == resp.totincr.max() ]
rich.parity.plot.hist(bins=range(10))
Explanation: Use totincr to select the respondents with the highest income. Compute the distribution of parity for just the high income respondents.
End of explanation
rich = resp[resp.totincr == resp.totincr.max() ]
notrich = resp[resp.totincr < resp.totincr.max()]
rich.parity.mean(), notrich.parity.mean()
Explanation: Compare the mean parity for high income respondents and others.
End of explanation
preg = nsfg.ReadFemPreg()
first = preg[preg.birthord ==1 ]
others = preg[preg.birthord >1 ]
first.totalwgt_lb.mean(), others.totalwgt_lb.mean()
def CohenEffectSize(group1, group2):
    mean_diff = group1.mean() - group2.mean()
    n1= len(group1)
    n2 = len(group2)
    
    pooled_var = (n1*group1.var() + n2* group2.var())/(n1+n2)
    d = mean_diff / np.math.sqrt(pooled_var)
    return d
CohenEffectSize(first.totalwgt_lb, others.totalwgt_lb)
Explanation: Exercise 4
Using the variable totalwgt_lb, investigate whether first babies are lighter or heavier than others. Compute Cohen’s d to quantify the difference between the groups. How does it compare to the difference in pregnancy length?
End of explanation
<END_TASK>