<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 4-Model the Data
"All models are wrong, Some of them are useful"
The power and limits of models
Tradeoff between Prediction Accuracy and Model Interpretability
Assessing Model Accuracy
Regression models (Simple, Multiple)
Classification model
Step1:  Correlation
Step2:  Exercise Find correlation between percent_white and highQ
Impact of de-regulation
Step3:  Exercise Find mean prices of HighQ weed for states that are legal and for states that are illegal
Finding good time of the week to buy weed in California
Step4:  Exercise If I need to buy weed on a wednesday, which state should I be in?
Step5:  Regression
Predicting price of HighQ weed in CA
<ASSISTANT_TASK:>
Python Code:
# Load the libraries
import numpy as np
import pandas as pd
from scipy import stats
from sklearn import linear_model
# Load the data again!
df = pd.read_csv("data/Weed_Price.csv", parse_dates=[-1])
df.sort(columns=['State','date'], inplace=True)
df1 = df[df.State=="California"].copy()
df1.set_index("date", inplace=True)
print df1.shape
idx = pd.date_range(df1.index.min(), df1.index.max())
df1 = df1.reindex(idx)
df1.fillna(method = "ffill", inplace=True)
print df1.shape
df1.head()
#Reading demographics data
demographics = pd.DataFrame.from_csv("data/Demographics_State.csv",header=0,index_col=False,sep=',')
demographics.rename(columns={'region':'State'}, inplace=True)
demographics.head()
df['State'] = df['State'].str.lower()
df.head()
df_demo = pd.merge(df, demographics, how="inner", on="State")
df_demo.head()
Explanation: 4-Model the Data
"All models are wrong, Some of them are useful"
The power and limits of models
Tradeoff between Prediction Accuracy and Model Interpretability
Assessing Model Accuracy
Regression models (Simple, Multiple)
Classification model
End of explanation
corr_bw_percapita_highq = stats.pearsonr(df_demo.per_capita_income, df_demo.HighQ)[0]
print corr_bw_percapita_highq
Explanation: Correlation
End of explanation
state_location = pd.read_csv("data/State_Location.csv")
state_location.head()
pd.unique(state_location.status)
Explanation: Exercise Find correlation between percent_white and highQ
Impact of de-regulation
End of explanation
df['year'] = pd.DatetimeIndex(df['date']).year
df['month'] = pd.DatetimeIndex(df['date']).month
df['week'] = pd.DatetimeIndex(df['date']).week
df['weekday'] = pd.DatetimeIndex(df['date']).weekday
df_demo_ca = df_demo[df_demo.State=="california"].copy()
df_demo_ca['year'] = pd.DatetimeIndex(df_demo_ca['date']).year
df_demo_ca['month'] = pd.DatetimeIndex(df_demo_ca['date']).month
df_demo_ca['week'] = pd.DatetimeIndex(df_demo_ca['date']).week
df_demo_ca['weekday'] = pd.DatetimeIndex(df_demo_ca['date']).weekday
df_demo_ca.head()
df_demo_ca.groupby("weekday").HighQ.mean()
Explanation: Exercise Find mean prices of HighQ weed for states that are legal and for states that are illegal
Finding good time of the week to buy weed in California
End of explanation
df.groupby(["State", "weekday"]).HighQ.mean()
df_st_wk = df.groupby(["State", "weekday"]).HighQ.mean()
df_st_wk.reset_index()
#Answer:
Explanation: Exercise If I need to buy weed on a wednesday, which state should I be in?
End of explanation
model_data = df1.loc[:,['HighQ']].copy()
idx = pd.date_range(model_data.index.min(), model_data.index.max()+ 30)
model_data.reset_index(inplace=True)
model_data.set_index("index", inplace=True)
model_data = model_data.reindex(idx)
model_data.tail(35)
model_data['IND'] = np.arange(model_data.shape[0])
model_data.tail(35)
model_data['IND_SQ'] = model_data['IND']**2
x = model_data.ix[0:532, ["IND","IND_SQ"]]
y = model_data.ix[0:532, "HighQ"]
x_test = model_data.ix[532:, ["IND","IND_SQ"]]
print x.shape, y.shape
ols = linear_model.LinearRegression(fit_intercept=True)
ols.fit(x, y)
ols_predict = ols.predict(x_test)
ols_predict
ols.coef_
Explanation: Regression
Predicting price of HighQ weed in CA
End of explanation
<END_TASK>