<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 SVM Learning
Load Dataset
Step1:  Train-Test Split
Before we construct out model pipeline, we divide the dataset into separate a separate training dataset (75% of the data) and a separate test dataset (25% of the data)
Step2:  K-fold Cross Validation and HyperParameter Tuning
Tuning hyperparameters via Grid Search
Step3:  Setting HyperParameters to tune
Step4:  Show HyperParameters Grid
Step6:  Stratified K-Fold Cross Validation
<ASSISTANT_TASK:>
Python Code:
from coherence import load_coherence_dataset
coherence_ds = load_coherence_dataset()
X = coherence_ds.data
y = coherence_ds.target
Explanation: SVM Learning
Load Dataset
End of explanation
from sklearn.cross_validation import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.25, random_state=1)
Explanation: Train-Test Split
Before we construct out model pipeline, we divide the dataset into separate a separate training dataset (75% of the data) and a separate test dataset (25% of the data):
End of explanation
from sklearn.grid_search import GridSearchCV
from sklearn.svm import SVC
from sklearn.pipeline import Pipeline
Explanation: K-fold Cross Validation and HyperParameter Tuning
Tuning hyperparameters via Grid Search
End of explanation
param_range = [0.0001, 0.001, 0.01, 0.1, 1.0, 10.0, 100.0, 1000.0]
param_grid = [# Parameters for Linear Kernel 
              {'clf__C': param_range,
               'clf__kernel': ['linear']}, 
              # Parameter for RBF Kernel
              {'clf__C': param_range,
               'clf__gamma': param_range,
               'clf__kernel': ['rbf']},
              # Parameter for Polynomial Kernel
              {'clf__C': param_range,
               'clf__degree': [2, 3, 5, 8, 16],
               'clf__coef0': [0, 1],
               'clf__kernel': ['poly']},
            ]
# So far, the `Pipeline` is not actually needed, but
# this code is ready in case multiple (features selection)
# steps would be considered for inclusion
pipe_svc = Pipeline([('clf', SVC(random_state=1))]) 
Explanation: Setting HyperParameters to tune
End of explanation
from sklearn.grid_search import ParameterGrid
hyperp_grid = list(ParameterGrid(param_grid))
# Pretty Printing Grid
kernel_combinations = dict()
for hp in hyperp_grid:
    kernel = hp['clf__kernel']
    kernel_combinations.setdefault(kernel, list())
    kernel_combinations[kernel].append(hp)
    
for kernel in kernel_combinations:
    print('Hyperparamters for Kernel: ', kernel.title())
    for hp in kernel_combinations[kernel]:
        print('\t', hp)
    print('-'*80)
Explanation: Show HyperParameters Grid
End of explanation
from sklearn.cross_validation import cross_val_score
from sklearn.metrics import confusion_matrix
def classify_samples(pipe_svc, param_grid, score, scoring='accuracy', **extra_score_params):
    
    
    
    print('-'*80)
    print('PERFORMING MACHINE LEARNING ANALYSIS WITH SCORING METRIC: ', scoring)
    
    # GridSearch CV
    gs = GridSearchCV(estimator=pipe_svc,
                      param_grid=param_grid,
                      scoring=scoring,
                      cv=10, n_jobs=-1)
    
    gs = gs.fit(X_train, y_train)  # Fit the GridSearchCV Estimator 
    # Printing training results 
    print('Best Score: ', gs.best_score_)
    print('Best Parameters: ', gs.best_params_)
    #print('Grid Scores: ', gs.grid_scores_)
    
    # Predict with Best Estimator
    clf = gs.best_estimator_
    clf.fit(X_train, y_train)
    y_pred = clf.predict(X_test)
    print('Mean Accuracy on Test set: %.3f' %  (clf.score(X_test, y_test)))
    
    print('Calculating Score on Prediction: ', score(y_test, y_pred, **extra_score_params))
    
    #Confusion Matrix
    cm = confusion_matrix(y_test, y_pred)
    print(cm)
    
    scores = cross_val_score(gs, X, y, scoring=scoring, cv=10, n_jobs=-1)
    print('All CV Scores')
    for i, score in enumerate(scores):
        print(i, '): ', score)
    print('CV %s : %.3f +/- %.3f' % (scoring, np.mean(scores), np.std(scores)))
    
    print('-'*80)
# Accuracy (replication)
from sklearn.metrics import accuracy_score
classify_samples(pipe_svc, param_grid, accuracy_score, scoring='accuracy')
# F1
from sklearn.metrics import f1_score
classify_samples(pipe_svc, param_grid, f1_score, scoring='f1', average='binary')
# F1 Macro-averaged
classify_samples(pipe_svc, param_grid, f1_score, scoring='f1', average='macro')
# F1 Weighted Average
classify_samples(pipe_svc, param_grid, f1_score, scoring='f1', average='weighted')
# ROC AUC Score
from sklearn.metrics import roc_auc_score
classify_samples(pipe_svc, param_grid, roc_auc_score, scoring='roc_auc', average='macro')  # default
# ROC AUC Score - Weighted Average
classify_samples(pipe_svc, param_grid, roc_auc_score, scoring='roc_auc', average='weighted')  # Roc AUC Weighted
Explanation: Stratified K-Fold Cross Validation
End of explanation
<END_TASK>