<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 <img src="images/hanford_variables.png">
1. Import the necessary packages to read in the data, plot, and create a linear regression model
Step1:  2. Read in the hanford.csv file
Step2:  3. Calculate the basic descriptive statistics on the data
Step3:  4. Calculate the coefficient of correlation (r) and generate the scatter plot. Does there seem to be a correlation worthy of investigation?
Step4:  5. Create a linear regression model based on the available data to predict the mortality rate given a level of exposure
Step5:  6. Plot the linear regression line on the scatter plot of values. Calculate the r^2 (coefficient of determination)
Step6:  7. Predict the mortality rate (Cancer per 100,000 man years) given an index of exposure = 10
Step7:  Now using statsmodels
<ASSISTANT_TASK:>
Python Code:
import pandas as pd
%matplotlib inline
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
import numpy as np
Explanation: <img src="images/hanford_variables.png">
1. Import the necessary packages to read in the data, plot, and create a linear regression model
End of explanation
df = pd.read_csv("data/hanford.csv")
Explanation: 2. Read in the hanford.csv file
End of explanation
df
df.describe()
df.hist()
Explanation: 3. Calculate the basic descriptive statistics on the data
End of explanation
df.corr()
df.plot(kind='scatter',x='Exposure',y='Mortality')
Explanation: 4. Calculate the coefficient of correlation (r) and generate the scatter plot. Does there seem to be a correlation worthy of investigation?
End of explanation
lm = LinearRegression()
data = np.asarray(df[['Mortality','Exposure']])
x = data[:,1:]
y = data[:,0]
lm.fit(x,y)
lm.score(x,y)
m = lm.coef_[0]
m
b = lm.intercept_
b
Explanation: 5. Create a linear regression model based on the available data to predict the mortality rate given a level of exposure
End of explanation
df.plot(kind='scatter',x='Exposure',y='Mortality')
plt.plot(df['Exposure'],m*df['Exposure']+b,'-')
Explanation: 6. Plot the linear regression line on the scatter plot of values. Calculate the r^2 (coefficient of determination)
End of explanation
lm.predict(10)
Explanation: 7. Predict the mortality rate (Cancer per 100,000 man years) given an index of exposure = 10
End of explanation
import statsmodels.formula.api as smf
lm = smf.ols(formula='Mortality~Exposure',data=df).fit()
lm.params
intercept, slope = lm.params
df.plot(kind='scatter',x='Exposure',y='Mortality')
plt.plot(df['Exposure'],slope*df['Exposure']+intercept,'-')
plt.xkcd()
df.plot(kind='scatter',x='Exposure',y='Mortality')
plt.plot(df['Exposure'],slope*df['Exposure']+intercept,'-')
lm.summary()
lm.mse_model
lm.pvalues
Explanation: Now using statsmodels
End of explanation
<END_TASK>