<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 CIFAR10 classification using HOG features
Load CIFAR10 dataset
Step1:  Preprocessing
Step2:  Lets look how images looks like
Step3:  Feature extraction
We use HOG descriptor from scikit-image library.
Step4:  Visualization of HOG histograms
Step5:  Short look at the training data
Step6:  PCA 2D decomposition
Step7:  Classification
We simply use LinearSVC from scikit-learn library.
Step8:  Training
Step9:  Testing and score
We obtain 0.4914 score on CIFAR10 testing dataset.
Step10:  Short look at the prediction
Step11:  Our results for different parameters
TODO
Step12:  SVC(kernel=linear) and SVC()
<ASSISTANT_TASK:>
Python Code:
import myutils
raw_data_training, raw_data_testing = myutils.load_CIFAR_dataset(shuffle=False)
# raw_data_training = raw_data_training[:5000]
class_names = myutils.load_CIFAR_classnames()
n_training = len( raw_data_training )
n_testing = len( raw_data_testing )
print('Loaded CIFAR10 database with {} training and {} testing samples'.format(n_training, n_testing))
Explanation: CIFAR10 classification using HOG features
Load CIFAR10 dataset
End of explanation
# Converting to greyscale
def rgb2gray(image):
    import cv2
    return cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
Xdata_training = [ rgb2gray(raw_data_training[i][0]) for i in range(n_training)]
Xdata_testing  = [ rgb2gray(raw_data_testing[i][0]) for i in range(n_testing)]
Explanation: Preprocessing
End of explanation
import random
import matplotlib.pyplot as plt
%matplotlib inline
# lets choose some random sample of 10 training images
examples_id = random.sample(range(n_training), 10)
fig, axarr = plt.subplots(2,len(examples_id), figsize=(15,3))
for i in range(len(examples_id)):
    id = examples_id[i]
    axarr[0,i].imshow(raw_data_training[id][0][:,:])
    axarr[0,i].axis('off')
    axarr[1,i].imshow(Xdata_training[id],cmap='gray')
    axarr[1,i].axis('off')
print('Few examples after preprocessing')
plt.show()
Explanation: Lets look how images looks like
End of explanation
# Configuring HOG descriptor
# see http://scikit-image.org/docs/dev/api/skimage.feature.html#skimage.feature.hog
# Configuration of HOG descriptor
normalize = True          #  True ==> yields a little bit better score
                          #  
block_norm = 'L2-Hys'     # or 'L1'
orientations = 9          # 
pixels_per_cell = [8, 8]  #  see section 'Additional remarks' for some explanation
cells_per_block = [2, 2]  # 
def extractFeature(img, vis=False):
    from skimage.feature import hog
    return hog(img, orientations, pixels_per_cell, cells_per_block, block_norm, visualise=vis, transform_sqrt=normalize)
Explanation: Feature extraction
We use HOG descriptor from scikit-image library.
End of explanation
# extracting one sample data
nfeatures = extractFeature(Xdata_training[0], vis=False).size
print('Number of features = {}'.format(nfeatures))
fig, axarr = plt.subplots(3,len(examples_id), figsize=(16,5))
for i in range(len(examples_id)):
    id = examples_id[i]
    axarr[0,i].imshow(raw_data_training[id][0][:,:])
    axarr[0,i].axis('off')
    axarr[1,i].imshow(Xdata_training[id],cmap='gray')
    axarr[1,i].axis('off')
    _, hog_vis = extractFeature(Xdata_training[id], vis=True)
    axarr[2,i].imshow(hog_vis,cmap='gray')
    axarr[2,i].axis('off')
plt.show()
# feature extraction
import numpy as np
X_training = np.array( [ extractFeature(Xdata_training[i], vis=False) for i in range(n_training) ] )
y_training = np.array( [ raw_data_training[i][1] for i in range(n_training) ] )
X_testing = np.array( [ extractFeature(Xdata_testing[i], vis=False) for i in range(n_testing) ] )
y_testing = np.array( [ raw_data_testing[i][1] for i in range(n_testing) ] )
Explanation: Visualization of HOG histograms
End of explanation
print( 'X_training shape is {}'.format( X_training.shape ) )
print( 'y_training shape is {}'.format( y_training.shape ) )
print( 'X_testing shape is {}'.format( X_testing.shape ) )
print( 'y_testing shape is {}'.format( y_testing.shape ) )
import pandas as pd
print( 'X_training data description')
pd.DataFrame( X_training ).describe()
print( 'y_training data description')
pd.DataFrame( y_training ).describe()
Explanation: Short look at the training data
End of explanation
from sklearn import decomposition
pca = decomposition.PCA(n_components=2)
pca.fit(X_training)
X = pca.transform(X_training)
print(pca.explained_variance_ratio_)
plt.figure( figsize=(15,15) )
plt.scatter( X[:, 0], X[:, 1], c=y_training, cmap='tab10' )
# plt.colorbar()
plt.show()
# TODO: remove outliers
Explanation: PCA 2D decomposition
End of explanation
from sklearn.svm import LinearSVC
# parameter C chosen experimentally (see explanation below)
C = 1.0
clf = LinearSVC(C=C)
Explanation: Classification
We simply use LinearSVC from scikit-learn library.
End of explanation
# this may take some time
clf.fit(X_training, y_training)
Explanation: Training
End of explanation
clf.score( X_testing, y_testing )
Explanation: Testing and score
We obtain 0.4914 score on CIFAR10 testing dataset.
End of explanation
y_predict = clf.predict( X_testing )
import numpy as np
np.unique( y_predict )
Explanation: Short look at the prediction
End of explanation
for C in [ 0.001, 0.01, 0.1, 1.0, 1.2, 1.5, 2.0, 10.0 ]:
    clf = LinearSVC(C=C)
    clf.fit(X_training, y_training)
    print( 'normalize={norm}, C={C}, score={score}'.format(norm=normalize, C=C, score=clf.score( X_testing, y_testing )) )
Explanation: Our results for different parameters
TODO: 
* show misclassifications
Additional remarks
Tuning C parameter
End of explanation
from sklearn.svm import SVC
svc_lin_clf = SVC(kernel='linear', C=1)
svc_lin_clf.fit(X_training, y_training)
svc_lin_clf.score(X_testing, y_testing)
from sklearn.svm import SVC
svc_clf = SVC(C=1)
svc_clf.fit(X_training, y_training)
svc_clf.score(X_testing, y_testing)
Explanation: SVC(kernel=linear) and SVC()
End of explanation
<END_TASK>