<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 On souhaite prédire la colonne "SalePrice". Donc toutes les autres colonnes sont des variables à faire apprendre
Step1:  Le model peux prendre en entré que des chiffre, il faut donc transformer les données en string en chiffre
Step2:  Il y a des données manquante qui ne peuvent pas etre prisent en compte par la modèle, il faut donc les remplacer
Step3:  Compréhension du Score de performance de notre modèle 
Step4:  Contruction de notre stratégie de Cross - Validation
Step5:  Voila notre 1er score !!
Step6:  Une valeur semble complétement perdu; on prédit 900.000 alors qu'elle devrait etre à moins de 200.000 ...
Step7:  transformation de notre Prix pour améliorer le score 
Step8:  Paramettre d'un modèle
Step9:  Aller plus loin 
Step10:  Encore plus loin 
Step11:  Toujours tester un ajout de features pour savoir si celle-ci va avoir un impact positif ou négatif
Regardons la colonnes "MasVnrArea"  & "MasVnrType" 
Step12:  BsmtFinType1 & BsmtFinSF1 / BsmtFinType2 & BsmtFinSF2
Step13:  Let's talk about Tree now 
Step14:  Problème avec nos données en string...
Step15:  On change chaque valeur en string en valeur numérique
Step16:  yeah mixer 2 algos !!!
Step17:  Importance des features dans un tree 
<ASSISTANT_TASK:>
Python Code:
features = [col for col in  data.columns if col not in "SalePrice"]
features
train = data[features]
y = data.SalePrice
#y = data['SalePrice']
train.head()
y.head()
sns.distplot(y)
# Modele pour la regression
from sklearn.linear_model import Ridge
import sklearn
sklearn.__version__
# Initialisation du model
model_ridge = Ridge()
# 1) On fait apprendre le model
model_ridge.fit(train, y)
# Error ...
Explanation: On souhaite prédire la colonne "SalePrice". Donc toutes les autres colonnes sont des variables à faire apprendre
End of explanation
data['SaleCondition'].head()
pd.get_dummies(data['SaleCondition'], prefix="SaleCondition").head()
def prepare_data(data):
    
    features = [col for col in  data.columns if col not in "SalePrice"] # 80 col
    
    train = data[features]
    y = data.SalePrice
    
    # Transform Object features to columns
    train = pd.get_dummies(train)
    
    return train, y
train, y = prepare_data(data.copy())
train.head()
# 2) On fait apprendre le model
model_ridge.fit(train, y)
# Error ...
data.BsmtFinType2.value_counts(dropna=False)
Explanation: Le model peux prendre en entré que des chiffre, il faut donc transformer les données en string en chiffre
End of explanation
pd.isnull(data).sum()
def prepare_data(data):
    
    features = [col for col in  data.columns if col not in "SalePrice"]
    
    train = data[features]
    y = data.SalePrice
    
    # Transform Object features to columns
    train = pd.get_dummies(train) 
    
    # Replace Nan value by mean of the column
    train = train.fillna(train.mean())
    return train, y
train, y = prepare_data(data.copy())
# 3) On fait apprendre le model
model_ridge.fit(train, y)
# yeah !!!
Explanation: Il y a des données manquante qui ne peuvent pas etre prisent en compte par la modèle, il faut donc les remplacer
End of explanation
from sklearn.metrics import mean_absolute_error
vrai = np.array([1000, 2000, 1500])
prediction = np.array([900, 2200, 1300])   # classic
#prediction = np.array([990, 2005, 1500])   # Best
#prediction = np.array([9000, 22000, 13000])    # Bad
mean_absolute_error(vrai, prediction)
1000 - 900
2000 - 2200
1500 - 1300
(100 + 200 + 200) / 3.0
Explanation: Compréhension du Score de performance de notre modèle :
End of explanation
from sklearn.model_selection import cross_val_score
def cross_validation(model, train, y, cv=5):
    mae = -cross_val_score(model, train, y, scoring="neg_mean_absolute_error", cv = cv)
    return mae
score = cross_validation(model_ridge, train, y)
print score
score.mean(), score.std()
data.SalePrice.describe()
Explanation: Contruction de notre stratégie de Cross - Validation
End of explanation
preds = pd.DataFrame({"preds":model_ridge.predict(train), "true":y})
preds["residuals"] = np.abs(preds["true"] - preds["preds"])
preds.plot(x = "preds", y = "residuals",kind = "scatter")
preds[preds.residuals >150000]
data.shape
def prepare_data_outlier(data):
    
    features = [col for col in  data.columns if col not in "SalePrice"]
    
    #on enleve les id qui sont trop extreme
    data = data.drop(data.index[[523,898, 1298]])
    
    train = data[features]
    y = data.SalePrice
    
    # Transform Object features to columns
    train = pd.get_dummies(train)
    
    # Replace Nan value by mean of the column
    train = train.fillna(train.mean())
    
    return train, y
train, y = prepare_data_outlier(data.copy())
print train.shape, y.shape
score = cross_validation(model_ridge, train, y)
print score.mean()
train, y = prepare_data(data.copy())
print train.shape, y.shape
score = cross_validation(model_ridge, train, y)
print score.mean()
from sklearn.model_selection import train_test_split
X_train, X_validation, y_train, y_validation = train_test_split(train, y, random_state = 3)
print"X_train : " + str(X_train.shape)
print"X_validation : " + str(X_validation.shape)
print"y_train : " + str(y_train.shape)
print"y_validation : " + str(y_validation.shape)
model_ridge.fit(X_train, y_train)
mes_predictions = model_ridge.predict(X_validation)
# Mes prédiction
mes_predictions[0:5]
# Les vrai valeurs 
y_validation[0:5]
mean_absolute_error(y_validation, mes_predictions)
plt.scatter(mes_predictions, y_validation)
plt.plot([min(mes_predictions),max(mes_predictions)], [min(mes_predictions),max(mes_predictions)], c="red")
plt.xlabel('Mes predicitons')
plt.ylabel('Vrai valeurs')
Explanation: Voila notre 1er score !!
End of explanation
analyse = X_validation.copy()
analyse.head()
analyse['prix'] = y_validation
analyse.head()
analyse['prediction'] = mes_predictions
analyse.head()
analyse[analyse.prediction >= 800000]
sns.countplot(data.SaleCondition)
Explanation: Une valeur semble complétement perdu; on prédit 900.000 alors qu'elle devrait etre à moins de 200.000 ...
End of explanation
sns.distplot(data.SalePrice)
data.SalePrice.describe()
sns.distplot(np.log1p(data.SalePrice))
np.log1p(data.SalePrice).describe()
def prepare_data_log(data):
    
    features = [col for col in  data.columns if col not in "SalePrice"]
    
    train = data[features]
    y = data.SalePrice
    # Transforme log
    y = np.log1p(y)
    
    # Transform Object features to columns
    train = pd.get_dummies(train) 
    
    # Replace Nan value by mean of the column
    train = train.fillna(train.mean())
    return train, y
train, y = prepare_data_log(data)
score = cross_validation(model_ridge, train, y)
print score.mean()
X_train, X_validation, y_train, y_validation = train_test_split(train, y, random_state = 3)
model_ridge.fit(X_train, y_train)
mes_predictions = model_ridge.predict(X_validation)
mes_predictions[0:5]
# Les vrai valeurs 
y_validation[0:5]
mean_absolute_error(y_validation, mes_predictions)
mes_predictions_exp = np.expm1(mes_predictions)
y_validation_exp = np.expm1(y_validation)
# Redonner les valeurs un transformation normal (exp)
mean_absolute_error(y_validation_exp, mes_predictions_exp)
plt.scatter(mes_predictions_exp, y_validation_exp)
plt.plot([min(mes_predictions_exp),max(mes_predictions_exp)], [min(mes_predictions_exp),max(mes_predictions_exp)]
         , c="red")
plt.xlabel('Mes predicitons')
plt.ylabel('Vrai valeurs')
def prepare_data_outlier_log(data):
    
    features = [col for col in  data.columns if col not in "SalePrice"]
    
    #on enleve les id qui sont trop extreme
    data = data.drop(data.index[[523,898, 1298]])
    
    train = data[features]
    y = data.SalePrice
    # Transforme log
    y = np.log1p(y)
    
    # Transform Object features to columns
    train = pd.get_dummies(train) 
    
    # Replace Nan value by mean of the column
    train = train.fillna(train.mean())
    return train, y
train, y = prepare_data_outlier_log(data.copy())
score = cross_validation(model_ridge, train, y)
print score.mean()
X_train, X_validation, y_train, y_validation = train_test_split(train, y, random_state = 3)
model_ridge.fit(X_train, y_train)
mes_predictions = model_ridge.predict(X_validation)
mes_predictions[0:5]
# Les vrai valeurs 
y_validation[0:5]
mes_predictions_exp = np.expm1(mes_predictions)
y_validation_exp = np.expm1(y_validation)
# Redonner les valeurs un transformation normal (exp)
mean_absolute_error(y_validation_exp, mes_predictions_exp)
plt.scatter(mes_predictions_exp, y_validation_exp)
plt.plot([min(mes_predictions_exp),max(mes_predictions_exp)], [min(mes_predictions_exp),max(mes_predictions_exp)]
         , c="red")
plt.xlabel('Mes predicitons')
plt.ylabel('Vrai valeurs')
Explanation: transformation de notre Prix pour améliorer le score :
End of explanation
model_ridge = Ridge()
model_ridge
#alphas = [0.05, 0.1, 0.3, 1, 3, 5, 10, 15, 30, 50, 75]
alphas = [10, 10.5, 11, 11.5, 12, 12.5, 13, 13.5, 14, 14.5, 15, 15.5, 16]
cv_ridge = [cross_validation(Ridge(alpha = alpha ,random_state=42), train, y).mean() 
            for alpha in alphas]
cv_ridge = pd.Series(cv_ridge, index = alphas)
cv_ridge.plot()
plt.xlabel("alpha")
plt.ylabel("mean absolute error")
cv_ridge.argmin()
cv_ridge
score = cross_validation(Ridge(alpha=13.5, random_state=42), train, y)
print score.mean()
X_train, X_validation, y_train, y_validation = train_test_split(train, y, random_state = 3)
model_ridge = Ridge(alpha=13.5, random_state=42)
model_ridge.fit(X_train, y_train)
mes_predictions_exp = np.expm1(model_ridge.predict(X_validation))
y_validation_exp = np.expm1(y_validation)
mean_absolute_error(y_validation_exp, mes_predictions_exp)
plt.scatter(mes_predictions_exp, y_validation_exp)
plt.plot([min(mes_predictions_exp),max(mes_predictions_exp)], [min(mes_predictions_exp),max(mes_predictions_exp)]
         , c="red")
plt.xlabel('Mes predicitons')
plt.ylabel('Vrai valeurs')
Explanation: Paramettre d'un modèle
End of explanation
data.plot(kind='scatter', x="1stFlrSF", y='SalePrice')
data.plot(kind='scatter', x="2ndFlrSF", y='SalePrice')
data['1stFlr_2ndFlr_Sf'] = data['1stFlrSF'] + data['2ndFlrSF']
data.plot(kind='scatter', x="1stFlr_2ndFlr_Sf", y='SalePrice')
sns.distplot(np.log1p(data['1stFlr_2ndFlr_Sf']))
data[(data['1stFlr_2ndFlr_Sf'] > 4000) & (data.SalePrice <= 700000)]
def prepare_data_outlier_log_plus(data):
    
    #on enleve les id qui sont trop extreme
    data = data.drop(data.index[[523,898, 1298]])
    
    # Ajout de nouvelle variables
    data['1stFlr_2ndFlr_Sf'] = np.log1p(data['1stFlrSF'] + data['2ndFlrSF'])
    
    
    features = [col for col in  data.columns if col not in "SalePrice"]
    
    train = data[features]
    
    y = data.SalePrice
    # Transforme log
    y = np.log1p(y)
    
    # Transform Object features to columns
    train = pd.get_dummies(train) 
    
    # Replace Nan value by mean of the column
    train = train.fillna(train.mean())
    
    print train.shape
    return train, y
train, y = prepare_data_outlier_log_plus(data.copy())
score = cross_validation(Ridge(alpha=13.5, random_state=42), train, y)
print score.mean()
X_train, X_validation, y_train, y_validation = train_test_split(train, y, random_state = 3)
model_ridge = Ridge(alpha=13.5, random_state=42)
model_ridge.fit(X_train, y_train)
mes_predictions_exp = np.expm1(model_ridge.predict(X_validation))
y_validation_exp = np.expm1(y_validation)
mean_absolute_error(y_validation_exp, mes_predictions_exp)
plt.scatter(mes_predictions_exp, y_validation_exp)
plt.plot([min(mes_predictions_exp),max(mes_predictions_exp)], [min(mes_predictions_exp),max(mes_predictions_exp)]
         , c="red")
plt.xlabel('Mes predicitons')
plt.ylabel('Vrai valeurs')
model_ridge.coef_[0:10]
coef = pd.Series(model_ridge.coef_, index = X_train.columns)
# On prend les 10 plus important features postive et négative
nb_important = 25
imp_coef = pd.concat([coef.sort_values().head(nb_important),
                     coef.sort_values().tail(nb_important)])
imp_coef.plot(kind = "barh", figsize=(10, 8))
plt.title("Coefficients in Model")
Explanation: Aller plus loin :
End of explanation
# Pour afficher des images (pas besoin de taper cet import)
from IPython.display import Image
data[['YearBuilt', 'GarageYrBlt']].head()
df = data.copy() # To work on df with no change in Daframe data
df['build_home_garage_same_year'] = 0
df.loc[data['YearBuilt'] == data['GarageYrBlt'], 'build_home_garage_same_year'] = 1
df.build_home_garage_same_year.value_counts()
def prepare_data_outlier_log_plus_2(data):
    
    #on enleve les id qui sont trop extreme
    data = data.drop(data.index[[523,898, 1298]])
    data['1stFlr_2ndFlr_Sf'] = np.log1p(data['1stFlrSF'] + data['2ndFlrSF'])
    
    data['build_home_garage_same_year'] = "N"
    data.loc[data['YearBuilt'] == data['GarageYrBlt'], 'build_home_garage_same_year'] = "Y"
    
    features = [col for col in  data.columns if col not in "SalePrice"]
    
    train = data[features]
    
    y = data.SalePrice
    # Transforme log
    y = np.log1p(y)
    
    # Transform Object features to columns
    train = pd.get_dummies(train) 
    
    # Replace Nan value by mean of the column
    train = train.fillna(train.mean())
    
    print train.shape
    return train, y
train.head()
data.shape
train, y = prepare_data_outlier_log_plus_2(data.copy())
score = cross_validation(Ridge(alpha=13.5, random_state=42), train, y)
print score.mean()
# Best was 0.0794542370234 donc ce n'est pas positif comme features
Explanation: Encore plus loin :
End of explanation
df.MasVnrType.value_counts()
df.MasVnrType.head()
#df.MasVnrArea.value_counts()
df.shape
df[df.MasVnrType == "None"].MasVnrArea.value_counts()
def prepare_data_outlier_log_plus_3(data):
    
    #on enleve les id qui sont trop extreme
    data = data.drop(data.index[[523,898, 1298]])
    data['1stFlr_2ndFlr_Sf'] = np.log1p(data['1stFlrSF'] + data['2ndFlrSF'])
    
    data.loc[data.MasVnrType == 'None', 'MasVnrArea'] = 0
    
    features = [col for col in  data.columns if col not in "SalePrice"]
    
    train = data[features]
    
    y = data.SalePrice
    # Transforme log
    y = np.log1p(y)
    
    # Transform Object features to columns
    train = pd.get_dummies(train) 
    
    # Replace Nan value by mean of the column
    train = train.fillna(train.mean())
    
    print train.shape
    return train, y
train, y = prepare_data_outlier_log_plus_3(data.copy())
score = cross_validation(Ridge(alpha=13.5, random_state=42), train, y)
print score.mean()
Image(url="http://i.giphy.com/GPq3wxmLbwUGA.gif")
Explanation: Toujours tester un ajout de features pour savoir si celle-ci va avoir un impact positif ou négatif
Regardons la colonnes "MasVnrArea"  & "MasVnrType" :
End of explanation
df.BsmtFinType2.value_counts(dropna=False)
df.BsmtFinSF2.describe()
df[pd.isnull(df.BsmtFinType2)].BsmtFinSF2.value_counts()
def prepare_data_outlier_log_plus_4(data):
    
    #on enleve les id qui sont trop extreme
    data = data.drop(data.index[[523,898, 1298]])
    data['1stFlr_2ndFlr_Sf'] = np.log1p(data['1stFlrSF'] + data['2ndFlrSF'])
    
    data.loc[data.MasVnrType == 'None', 'MasVnrArea'] = 0
    
    data.loc[pd.isnull(data.BsmtFinType2), 'BsmtFinSF2'] = 0
    
    features = [col for col in  data.columns if col not in "SalePrice"]
    
    train = data[features]
    
    
    y = data.SalePrice
    # Transforme log
    y = np.log1p(y)
    
    # Transform Object features to columns
    train = pd.get_dummies(train) 
    
    # Replace Nan value by mean of the column
    train = train.fillna(train.mean())
    
    print train.shape
    return train, y
train, y = prepare_data_outlier_log_plus_4(data.copy())
score = cross_validation(Ridge(alpha=13.5, random_state=42), train, y)
print score.mean()
X_train_ridge, X_validation_ridge, y_train_ridge, y_validation_ridge = train_test_split(train, y, random_state = 3)
model_ridge = Ridge(alpha=13.5, random_state=42)
model_ridge.fit(X_train_ridge, y_train_ridge)
coef = pd.Series(np.abs(model_ridge.coef_), index = X_train.columns)
# On prend les 10 plus important features postive et négative
nb_important = 15
#imp_coef = pd.concat([coef.sort_values().head(nb_important),
           #          coef.sort_values().tail(nb_important)])
    
imp_coef = coef.sort_values().head(nb_important)
imp_coef.plot(kind = "barh", figsize=(10, 8))
plt.title("Coefficients in Model")
coef.sort_values().head(10)
features_to_delete = ["GarageCond_Ex",
"Condition2_RRAe",
"Exterior1st_Stone",
"MiscFeature_TenC",
"MiscVal",
"BsmtUnfSF",
"LotArea",
"MasVnrArea",
"GarageYrBlt",
"Id"]
def prepare_data_outlier_log_plus_4_bis(data):
    
    #on enleve les id qui sont trop extreme
    data = data.drop(data.index[[523,898, 1298]])
    data['1stFlr_2ndFlr_Sf'] = np.log1p(data['1stFlrSF'] + data['2ndFlrSF'])
    
    data.loc[data.MasVnrType == 'None', 'MasVnrArea'] = 0
    
    data.loc[pd.isnull(data.BsmtFinType2), 'BsmtFinSF2'] = 0
    
    features = [col for col in  data.columns if col not in "SalePrice"]
    
    train = data[features]
    
    
    y = data.SalePrice
    # Transforme log
    y = np.log1p(y)
    
    # Transform Object features to columns
    train = pd.get_dummies(train) 
    
    # Replace Nan value by mean of the column
    train = train.fillna(train.mean())
    
    train = train.drop(features_to_delete, axis=1)
    
    print train.shape
    return train, y
train, y = prepare_data_outlier_log_plus_4_bis(data.copy())
score = cross_validation(Ridge(alpha=13.5, random_state=42), train, y)
print score.mean()
#pd.isnull(df).sum()
df.shape
column_detail = pd.DataFrame(pd.isnull(df).sum(), columns=['nbr_null'])
column_detail.sort_values('nbr_null', ascending=0, inplace=True)
column_detail.head(10)
def prepare_data_outlier_log_plus_5(data):
    
    #on enleve les id qui sont trop extreme
    data = data.drop(data.index[[523,898, 1298]])
    data['1stFlr_2ndFlr_Sf'] = np.log1p(data['1stFlrSF'] + data['2ndFlrSF'])
    
    data.loc[data.MasVnrType == 'None', 'MasVnrArea'] = 0
    
    data.loc[pd.isnull(data.BsmtFinType2), 'BsmtFinSF2'] = 0
    
    #Drop features with too much Null value
    data = data.drop('PoolQC', axis=1)
    data = data.drop('MiscFeature', axis=1)
    data = data.drop('Alley', axis=1)
    data = data.drop('Fence', axis=1)
    
    features = [col for col in  data.columns if col not in "SalePrice"]
    
    train = data[features]
    
    y = data.SalePrice
    # Transforme log
    y = np.log1p(y)
    
    # Transform Object features to columns
    train = pd.get_dummies(train) 
    
    # Replace Nan value by mean of the column
    train = train.fillna(train.mean())
    
    print train.shape
    return train, y
train, y = prepare_data_outlier_log_plus_5(data.copy())
score = cross_validation(Ridge(alpha=13.5, random_state=42), train, y)
print score.mean()
Image(url="http://i.giphy.com/LZfZXcFNOOzw4.gif")
Explanation: BsmtFinType1 & BsmtFinSF1 / BsmtFinType2 & BsmtFinSF2
End of explanation
from sklearn.tree import DecisionTreeRegressor
np.random.seed(42)
dt = DecisionTreeRegressor(random_state=0)
def dt_prepare_data(data):
    
    features = [col for col in  data.columns if col not in "SalePrice"]
    
    train = data[features]
    y = data.SalePrice
    
    # Replace Nan value by mean of the column
    train = train.fillna(train.mean())
    
    print train.shape
    return train, y
train, y = dt_prepare_data(data.copy())
train.head()
# 1) On fait apprendre le model
dt.fit(train, y)
#Error...
from sklearn.preprocessing import LabelEncoder
Explanation: Let's talk about Tree now :
End of explanation
categoricals = [x for x in data.columns if data[x].dtype == 'object']
categoricals
data.SaleCondition.head()
lbl = LabelEncoder() # Initialisation
lbl.fit(data['SaleCondition'].values)
test = lbl.transform(data['SaleCondition'].values)
test[0:5]
Explanation: Problème avec nos données en string...
End of explanation
def dt_prepare_data_plus(data):
    
    features = [col for col in  data.columns if col not in "SalePrice"]
    
    train = data[features]
    y = data.SalePrice
    
    # String problem
    categoricals = [x for x in train.columns if train[x].dtype == 'object']
    for col in categoricals:
        lbl = LabelEncoder()
        lbl.fit(train[col].values)
        train[col] = lbl.transform(train[col].values)
    
    # Replace Nan value by mean of the column
    train = train.fillna(train.mean())
    
    print train.shape
    return train, y
train, y = dt_prepare_data_plus(data.copy())
train.head()
data.head()
score = cross_validation(dt, train, y)
print score.mean()
def dt_prepare_data_plus_log(data):
    
    features = [col for col in  data.columns if col not in "SalePrice"]
    
    train = data[features]
    y = data.SalePrice
    # Transforme log
    y = np.log1p(y)
    
    # String problem
    categoricals = [x for x in train.columns if train[x].dtype == 'object']
    for col in categoricals:
        lbl = LabelEncoder()
        lbl.fit(train[col].values)
        train[col] = lbl.transform(train[col].values)
    
    # Replace Nan value by mean of the column
    train = train.fillna(train.mean())
    
    print train.shape
    return train, y
train, y = dt_prepare_data_plus_log(data.copy())
score = cross_validation(dt, train, y)
print score.mean()
from sklearn.ensemble import RandomForestRegressor
rfr = RandomForestRegressor(random_state=0)
score = cross_validation(rfr, train, y)
print score.mean()
X_train, X_validation, y_train, y_validation = train_test_split(train, y, random_state = 3)
rfr.fit(X_train, y_train)
mes_predictions_exp = np.expm1(rfr.predict(X_validation))
mes_predictions_exp[0:5]
y_validation_exp = np.exp(y_validation)
y_validation_exp[0:5]
mean_absolute_error(y_validation_exp, mes_predictions_exp)
plt.scatter(mes_predictions_exp, y_validation_exp)
plt.plot([min(mes_predictions_exp),max(mes_predictions_exp)], [min(mes_predictions_exp),max(mes_predictions_exp)]
         , c="red")
plt.xlabel('Mes predicitons')
plt.ylabel('Vrai valeurs')
def dt_prepare_data_plus_log_1(data):
    
    #on enleve les id qui sont trop extreme
    data = data.drop(data.index[[523,898, 1298]])
    data['1stFlr_2ndFlr_Sf'] = np.log1p(data['1stFlrSF'] + data['2ndFlrSF'])
    
    data.loc[data.MasVnrType == 'None', 'MasVnrArea'] = 0
    
    data.loc[pd.isnull(data.BsmtFinType2), 'BsmtFinSF2'] = 0
    
    features = [col for col in  data.columns if col not in "SalePrice"]
    
    train = data[features]
    y = data.SalePrice
    # Transforme log
    y = np.log1p(y)
    
    # String problem
    categoricals = [x for x in train.columns if train[x].dtype == 'object']
    for col in categoricals:
        lbl = LabelEncoder()
        lbl.fit(train[col].values)
        train[col] = lbl.transform(train[col].values)
    
    # Replace Nan value by mean of the column
    train = train.fillna(train.mean())
    
    print train.shape
    return train, y
train, y = dt_prepare_data_plus_log_1(data.copy())
score = cross_validation(rfr, train, y)
print score.mean()
pd.DataFrame?
coef = pd.DataFrame({'col' : X_train.columns,'importance' : rfr.feature_importances_})
coef = coef.sort_values('importance', ascending=False)
top_tree_features = coef.col.head(25)
#plt.figure(figsize=(10, 5))
#coef.head(25).plot(kind='bar')
#plt.title('Feature Significance')
top_tree_features
coef_ridge = pd.DataFrame({'col' : X_train.columns,
                           'importance' : model_ridge.coef_})
coef_ridge[coef_ridge.col.isin(list(top_tree_features))].shape
    
#imp_coef = coef.sort_values().head(nb_important)
#imp_coef.plot(kind = "barh", figsize=(10, 8))
#plt.title("Coefficients in Model")
coef_ridge.tail()
rfr = RandomForestRegressor(n_estimators=100, random_state=0, n_jobs=-1)
rfr
score = cross_validation(rfr, train, y)
print score.mean()
RandomForestRegressor?
cv_rfr = []
n_estimators = [10, 50, 100, 200]
max_depths = [3, 5, 7]
for n_estimator in n_estimators:
    for max_depth in max_depths:
        print "Je lance n_estimator : " + str(n_estimator) + " et "+str(max_depth) + " max_depth."
        score = cross_validation(RandomForestRegressor(n_estimators=n_estimator,
                                                       max_depth=max_depth,
                                                       random_state=0), train, y).mean() 
        cv_rfr.append({'n_estimator' : n_estimator,
                       'max_depths' : max_depth,
                        'score' : score})
cv_rfr_df = pd.DataFrame(cv_rfr)
cv_rfr_df
from sklearn.model_selection import GridSearchCV
param_grid = { "n_estimators"      : [250, 300],
           "max_depth"         : [3, 5, 7, 9]}
#grid_search = GridSearchCV(rfr, param_grid, n_jobs=-1, cv=5)
grid_search = GridSearchCV(rfr,
                           param_grid,
                           n_jobs=-1,
                           cv=5,
                           scoring='neg_mean_absolute_error')
grid_search.fit(train, y)
#print grid_search.best_params_
grid_search.grid_scores_
print grid_search.best_params_
rfr = RandomForestRegressor(n_estimators=300, max_depth=9
                            , random_state=0)
X_train, X_validation, y_train, y_validation = train_test_split(train, y, random_state = 3)
rfr.fit(X_train, y_train)
mes_predictions_exp = np.expm1(rfr.predict(X_validation))
mes_predictions_exp[0:5]
y_validation_exp = np.exp(y_validation)
y_validation_exp[0:5]
mean_absolute_error(y_validation_exp, mes_predictions_exp)
plt.scatter(mes_predictions_exp, y_validation_exp)
plt.plot([min(mes_predictions_exp),max(mes_predictions_exp)], [min(mes_predictions_exp),max(mes_predictions_exp)]
         , c="red")
plt.xlabel('Mes predicitons')
plt.ylabel('Vrai valeurs')
Explanation: On change chaque valeur en string en valeur numérique
End of explanation
model_ridge = Ridge(alpha=13.5, random_state=42)
mes_predictions_ridge = np.expm1(model_ridge.predict(X_validation_ridge))
mes_predictions_ridge[0:5]
mes_predictions_exp[0:5]
resultat = pd.DataFrame({'ridge' : mes_predictions_ridge,
                       'tree' : mes_predictions_exp,
                       'realite' : y_validation_exp})
resultat['moyenne'] = (resultat.ridge+ resultat.tree) / 2.0
resultat.head()
mean_absolute_error(resultat.realite, resultat.ridge)
mean_absolute_error(resultat.realite, resultat.tree)
mean_absolute_error(resultat.realite, resultat.moyenne)
Explanation: yeah mixer 2 algos !!!
End of explanation
coef = pd.Series(rfr.feature_importances_, index = X_train.columns).sort_values(ascending=False)
plt.figure(figsize=(10, 5))
coef.head(25).plot(kind='bar')
plt.title('Feature Significance')
Explanation: Importance des features dans un tree :
End of explanation
<END_TASK>