<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Imputation
Step1:  Create data frame
Step2:  Add some missing values
Step3:  Confirm the presence of null values
Step4:  Create categorical variables
Step5:  Create dummy variables
Step6:  Impute data
Replace null values using MICE model
 MICEData class
Step7:  Imputation for one feature 
The conditional_formula attribute is a dictionary containing the models that will be used to impute the data for each column. This can be updated to change the imputation model.
Step8:  The perturb_params method must be called before running the impute method, that runs the imputation. It updates the specified column in the data attribute.
Step9:  Impute all
Step10:  Validation
<ASSISTANT_TASK:>
Python Code:
import pandas as pd
import numpy as np
import statsmodels
from statsmodels.imputation import mice
import random
random.seed(10)
Explanation: Imputation
End of explanation
df = pd.read_csv("http://goo.gl/19NKXV")
df.head()
original = df.copy()
original.describe().loc['count',:]
Explanation: Create data frame
End of explanation
def add_nulls(df, n):
    new = df.copy()
    new.iloc[random.sample(range(new.shape[0]), n), :] = np.nan
    return new
df.Cholesterol = add_nulls(df[['Cholesterol']], 20)
df.Smoking = add_nulls(df[['Smoking']], 20)
df.Education = add_nulls(df[['Education']], 20)
df.Age = add_nulls(df[['Age']], 5)
df.BMI = add_nulls(df[['BMI']], 5)
Explanation: Add some missing values
End of explanation
df.describe()
Explanation: Confirm the presence of null values
End of explanation
for col in ['Gender', 'Smoking', 'Education']:
    df[col] = df[col].astype('category')
df.dtypes
Explanation: Create categorical variables
End of explanation
df = pd.get_dummies(df);
Explanation: Create dummy variables
End of explanation
imp = mice.MICEData(df)
Explanation: Impute data
Replace null values using MICE model
 MICEData class
End of explanation
imp.conditional_formula['BMI']
before = imp.data.BMI.copy()
Explanation: Imputation for one feature 
The conditional_formula attribute is a dictionary containing the models that will be used to impute the data for each column. This can be updated to change the imputation model.
End of explanation
imp.perturb_params('BMI')
imp.impute('BMI')
after = imp.data.BMI
import matplotlib.pyplot as plt
plt.clf()
fig, ax = plt.subplots(1, 1)
ax.plot(before, 'or', label='before', alpha=1, ms=8)
ax.plot(after, 'ok', label='after', alpha=0.8, mfc='w', ms=8)
plt.legend();
pd.DataFrame(dict(before=before.describe(), after=after.describe()))
before[before != after]
after[before != after]
Explanation: The perturb_params method must be called before running the impute method, that runs the imputation. It updates the specified column in the data attribute.
End of explanation
imp.update_all(2)
imp.plot_fit_obs('BMI');
imp.plot_fit_obs('Age');
Explanation: Impute all
End of explanation
original.mean()
for col in original.mean().index:
    x = original.mean()[col]
    y = imp.data[col].mean()
    e = abs(x - y) / x
    print("{:<12}  mean={:>8.2f}, exact={:>8.2f}, error={:>5.2g}%".format(col, x, y, e * 100))
Explanation: Validation
End of explanation
<END_TASK>