<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Machine Learning in Python
Step1:  Illustration
An image classification example
Python-powered
opencv and scikit-image for feature extraction
keras for feature augmentation
scikit-learn for classification
flask for web application
plus some JavaScript for interactive web features
Types of learning
Unsupervised (clustering, density estimation)
Supervised (classification, regression)
Reinforcement (reward and punishment)
Objective of S/L
Predict outcome from features
y = outcome or label
X = vector of features
y = f(X, Θ) + error
Loss function = g(y, f(X, Θ))
Model evaluation
In-sample (training) and out-of-sample (test) errors
Cross validation
Holdout
K-fold
LOOCV
Note
Step2:  Get data
Step3:  Feature extraction
Split labels and features as plain numpy  arrays
Step4:  Generate polynomial (interaction) features
Step5:  Scale features to have zero mean and unit standard deviation
Step6:  Select "useful" features
Step7:  Split into training and test sets
Step8:  Train an estimator
Step9:  Evaluate estimator
Step10:  Putting it all together in a pipeline
Step11:  Alternative pipeline
Step12:  Getting detailed information from pipeline
<ASSISTANT_TASK:>
Python Code:
%matplotlib inline
import seaborn as sns
sns.set_context('notebook', font_scale=1.5)
Explanation: Machine Learning in Python
End of explanation
from sklearn.preprocessing import PolynomialFeatures, StandardScaler
from sklearn.feature_selection import VarianceThreshold
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.linear_model import RidgeClassifierCV
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report
from sklearn.pipeline import Pipeline
import pandas as pd
Explanation: Illustration
An image classification example
Python-powered
opencv and scikit-image for feature extraction
keras for feature augmentation
scikit-learn for classification
flask for web application
plus some JavaScript for interactive web features
Types of learning
Unsupervised (clustering, density estimation)
Supervised (classification, regression)
Reinforcement (reward and punishment)
Objective of S/L
Predict outcome from features
y = outcome or label
X = vector of features
y = f(X, Θ) + error
Loss function = g(y, f(X, Θ))
Model evaluation
In-sample (training) and out-of-sample (test) errors
Cross validation
Holdout
K-fold
LOOCV
Note: Any step which uses label/outcome information must be included in cross-validation pipeline
S/L training pipeline
Raw data
Extract features from raw data
Normalize/scale features
Select features for use in model
Model selection/evaluation
S/L training process in scikit-learn
Consistent API for scikit-learn classes
fit
transform
predict
fit_transform for transformations
fit_predict for clustering
score for classification and regression
get_params
set_params
Feature extraction
Domain knowledge useful
Consider augmenting with external data sources
More specialized tools
From natural language
From images
From images/video
Image feature augmentation
From audio
Normalize/scale features
Necessary for methods based on measures of distance
Most commonly
Note: Must apply same scaling to training and test data
Feature selection
Note: Include feature selection in a pipeline
Model selection/evaluation
Example
This example is only meant to show the mechanics of using scikit-learn.
End of explanation
iris = pd.read_csv('iris.csv')
iris.head()
sns.pairplot(iris, hue='Species')
pass
Explanation: Get data
End of explanation
X = iris.iloc[:, :4].values
y = iris.iloc[:, 4].astype('category').cat.codes.values
X[:3]
y[:3]
Explanation: Feature extraction
Split labels and features as plain numpy  arrays
End of explanation
poly = PolynomialFeatures(2)
X_poly = poly.fit_transform(X)
X_poly[:3]
Explanation: Generate polynomial (interaction) features
End of explanation
scaler = StandardScaler()
X_poly_scaled = scaler.fit_transform(X_poly)
X_poly_scaled[:3]
Explanation: Scale features to have zero mean and unit standard deviation
End of explanation
selector = VarianceThreshold(threshold=0.1)
X_new = selector.fit_transform(X_poly_scaled)
X_poly_scaled.shape, X_new.shape
Explanation: Select "useful" features
End of explanation
X_train, X_test, y_train, y_test = train_test_split(X_new, y, random_state=1)
X_train[:3]
y_train[:3]
X_test[:3]
y_test[:3]
Explanation: Split into training and test sets
End of explanation
alphas = [1e-5, 1e-4, 1e-3, 1e-2, 1e-1, 1, 10]
clf = RidgeClassifierCV(alphas=alphas, cv=5)
clf.fit(X_train, y_train)
Explanation: Train an estimator
End of explanation
y_pred = clf.predict(X_test)
print(classification_report(y_test, y_pred))
Explanation: Evaluate estimator
End of explanation
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)
pipe = Pipeline([
    ('polynomaial_features', PolynomialFeatures(2)),
    ('standard_scalar', StandardScaler()),
    ('feature_selection', VarianceThreshold(threshold=0.1)),
    ('classification', clf)
])
pipe.fit(X_train, y_train)
y_pred = pipe.predict(X_test)
print(classification_report(y_test, y_pred))
Explanation: Putting it all together in a pipeline
End of explanation
params = {'n_estimators': [5, 10, 25], 'max_depth': [1, 3, None]}
rf = RandomForestClassifier()
clf2 = GridSearchCV(rf, params, cv=5, n_jobs=-1)
pipe2 = Pipeline([
    ('polynomaial_features', PolynomialFeatures(2)),
    ('feature_selection', VarianceThreshold(threshold=0.1)),
    ('classification', clf2)
])
pipe2.fit(X_train, y_train)
y_pred2 = pipe2.predict(X_test)
print(classification_report(y_test, y_pred2))
Explanation: Alternative pipeline
End of explanation
classifier = pipe2.named_steps['classification']
classifier.best_params_
Explanation: Getting detailed information from pipeline
End of explanation
<END_TASK>