<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 3.1 Problem description
Try to build a classifier for the MNIST dataset that achieves over 97% accuracy
on the test set. Hint
Step1:  Split test and training data
Step2:  3.2 Training a Random Forest Classifier for baseline
The reason to use Random Forest Classifier is it runs faster than linear model
Step3:  3.3 Training a KNeighborsClassifier Classifier with default settings
Seems like we have to have n_jobs = 1 so the prediction runs within reasonable time.
Step4:  3.4 GridSearchCV
<ASSISTANT_TASK:>
Python Code:
from scipy.io import loadmat
mnist = loadmat('./datasets/mnist-original.mat')
mnist
X, y = mnist['data'], mnist['label']
X = X.T
X.shape
y = y.T
y.shape
type(y)
%matplotlib inline
import matplotlib
import matplotlib.pyplot as plt
Explanation: 3.1 Problem description
Try to build a classifier for the MNIST dataset that achieves over 97% accuracy
on the test set. Hint: the KNeighborsClassifier works quite well for this task;
you just need to find good hyperparameter values (try a grid search on the
weights and n_neighbors hyperparameters).
Load the data
End of explanation
X_train, X_test, y_train, y_test = X[:60000], X[60000:], y[:60000], y[60000:]
len(X_train)
shuffle_index = np.random.permutation(len(X_train))
X_train, y_train = X_train[shuffle_index], y_train[shuffle_index]
Explanation: Split test and training data
End of explanation
from sklearn.ensemble import RandomForestClassifier
forest_clf = RandomForestClassifier(random_state=42)
forest_clf.fit(X_train, y_train)
forest_pred = forest_clf.predict(X_test)
forest_pred = forest_pred.reshape(10000,1)
accuracy = (forest_pred == y_test).sum() / len(y_test)
print(accuracy)
Explanation: 3.2 Training a Random Forest Classifier for baseline
The reason to use Random Forest Classifier is it runs faster than linear model
End of explanation
from sklearn.neighbors import KNeighborsClassifier
knn_clf = KNeighborsClassifier(n_jobs=-1)
knn_clf.fit(X_train, y_train)
knn_clf.predict([X_test[0]])
# for i in range(1000):
#     knn_clf.predict([X_test[i]])
knn_pred = knn_clf.predict(X_test)
knn_pred = knn_pred.reshape(10000, 1)
accuracy = (knn_pred == y_test).sum() / len(y_test)
print(accuracy)
Explanation: 3.3 Training a KNeighborsClassifier Classifier with default settings
Seems like we have to have n_jobs = 1 so the prediction runs within reasonable time.
End of explanation
from sklearn.model_selection import GridSearchCV
param_grid = [
    {'n_jobs': [-1], 'n_neighbors': [3, 5, 11, 19], 'weights': ['uniform', 'distance']}
]
grid_search = GridSearchCV(knn_clf, param_grid, cv=3, scoring='accuracy', n_jobs=-1)
grid_search.fit(X_train, y_train)
Explanation: 3.4 GridSearchCV
End of explanation
<END_TASK>