<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 K-Means Algorithm, K-Means Clustering - Unsupervised Learning
cf. Clustering (Coursera Ng, Machine Learning Introduction)
cf. Clustering K-Means Algorithm
cf Ritchie Ng's example, K-nearest Neighbors (KNN) Classification Model  
cf. scikit learn's K-means Clustering
Step1:  1b. KNN (K=5)
cf. Machine Learning with Scikit-Learn
Step2:  2. Evaluation procedure 2 - Train/test split
Data is randomly assigned unless you use random_state hyperparameter
* If you use random_state=4
    - Your data will be split exactly the same way
<ASSISTANT_TASK:>
Python Code:
%matplotlib inline
# sanity check for python setup
import sys
print(sys.executable)
print(sys.path)
import numpy as np
import matplotlib.pyplot as plt  
from mpl_toolkits.mplot3d import Axes3D  
from sklearn.cluster import KMeans 
from sklearn import datasets  
np.random.seed(5)
centers=[[1,1],[-1,-1],[1,-1]]
iris=datasets.load_iris()
X=iris.data
y=iris.target
estimators = {'k_means_iris_3': KMeans(n_clusters=3), 
                'k_means_iris_8':KMeans(n_clusters=8),
                 'k_means_iris_bad_init': KMeans(n_clusters=3, n_init=1,init='random')}
fignum = 1
for name, est in estimators.items():
    fig = plt.figure(fignum, figsize=(4,3))
    plt.clf()
    ax=Axes3D(fig,rect=[0,0,.95,1],elev=48,azim=134)
    
    plt.cla()
    est.fit(X)
    labels=est.labels_
    
    ax.scatter(X[:,3],X[:,0],X[:,2],c=labels.astype(np.float))
    
    ax.w_xaxis.set_ticklabels([])
    ax.w_yaxis.set_ticklabels([])
    ax.w_zaxis.set_ticklabels([])
    ax.set_xlabel('Petal width')
    ax.set_ylabel('Sepal length')
    ax.set_zlabel('Petal length')
    fignum=fignum+1
    
    
# Plot the ground truth
fig=plt.figure(fignum,figsize=(4,3))
plt.clf()
ax=Axes3D(fig,rect=[0,0,.95,1],elev=48,azim=134)
plt.cla()
for name,label in [('Setosa',0),('Versicolour',1),('Virginica',2)]:
    ax.text3D(X[y==label,3].mean(), 
                X[y==label,0].mean()+1.5,
                 X[y==label,2].mean(),name,
                 horizontalalignment='center',
                 bbox=dict(alpha=.5,edgecolor='w',facecolor='w'))
# Reorder the labels to have colors matching the cluster results
# y = np.choose(y,[1,2,0]).astype(np.float)
yint=np.choose(y, [1,2,0])#.astype(np.float)
ax.scatter(X[:,3],X[:,0],X[:,2],c=yint)
ax.w_xaxis.set_ticklabels([])
ax.w_yaxis.set_ticklabels([])
ax.w_zaxis.set_ticklabels([])
ax.set_xlabel('Petal width')
ax.set_ylabel('Sepal length')
ax.set_zlabel('Petal length')
plt.show()
import pandas as pd
print(type(X))
print(type(y))
print(X.shape)
print(y.shape)
print(pd.DataFrame(y).iloc[:,0].unique())
Explanation: K-Means Algorithm, K-Means Clustering - Unsupervised Learning
cf. Clustering (Coursera Ng, Machine Learning Introduction)
cf. Clustering K-Means Algorithm
cf Ritchie Ng's example, K-nearest Neighbors (KNN) Classification Model  
cf. scikit learn's K-means Clustering
End of explanation
from sklearn.neighbors import KNeighborsClassifier
knn = KNeighborsClassifier(n_neighbors=5)
knn.fit(X,y)
y_pred=knn.predict(X)
# compute classification accuracy for the logistic regression model
from sklearn import metrics
print(metrics.accuracy_score(y,y_pred))
### 1c. KNN (K=1)
knn = KNeighborsClassifier(n_neighbors=1)
knn.fit(X,y)
y_pred=knn.predict(X)
print(metrics.accuracy_score(y,y_pred))
Explanation: 1b. KNN (K=5)
cf. Machine Learning with Scikit-Learn
End of explanation
# STEP 1: split X and y into training and testing sets
from sklearn.cross_validation import train_test_split
X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.4,random_state=4)
knn=KNeighborsClassifier(n_neighbors=5)
knn.fit(X_train,y_train)
y_pred=knn.predict(X_test)
print(metrics.accuracy_score(y_test,y_pred))
knn=KNeighborsClassifier(n_neighbors=1)
knn.fit(X_train,y_train)
y_pred=knn.predict(X_test)
print(metrics.accuracy_score(y_test,y_pred))
# try K=1 through K=25 and record testing accuracy  
k_range=range(1,26)
# We can create Python dictionary using [] or dict()
scores=[]
# We use a loop through the range 1 to 26
# We append the scores in the dictionary
for k in k_range:
    knn=KNeighborsClassifier(n_neighbors=k)
    knn.fit(X_train,y_train)
    y_pred=knn.predict(X_test)
    scores.append(metrics.accuracy_score(y_test,y_pred))
print(scores)
# import Matplotlib (scientific plotting library)
#import matplotlib.pyplot as plt
# allow plots to appear within the notebook
#%matplotlib inline
# plot the relationship between $K$ and testing accuracy
# plt.plot(x_axis,y_axis)
plt.plot(k_range,scores)
plt.xlabel('Value of K for KNN')
plt.ylabel('Testing Accuracy')
print(pd.DataFrame(X).describe())
print(pd.DataFrame(X).head())
pd.DataFrame(X).head(10)
Explanation: 2. Evaluation procedure 2 - Train/test split
Data is randomly assigned unless you use random_state hyperparameter
* If you use random_state=4
    - Your data will be split exactly the same way
End of explanation
<END_TASK>