<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Machine Learning Basics with Sklearn
First some imports for the notebook and visualization.
Step1:  Choosing a dataset
First of all you need a dataset to work on. To keep things simple we will use the iris dataset provided with scikit.
Step2:  Splitting the dataset
The dataset needs to be split into a training and test dataset.
This is done to first train our model and then test how good it is on data it has never seen before.
Step3:  Decision Tree Classifier
We will use a simple decision tree as the first model and train it.
Step4:  Visualize the decision tree
Whereas the decision tree is a simple graph we can visualize it quite simple.
Step5:  Evaluating the model
After the model is trained it has to be evaluated.
Step6:  KNN-Classifier
Let's try another classifier. Initialize, train, print error.
Step7:  Implementing your own KNN
Let's implement a simple knn classifier with k=1.
We have to implement the fit method and the predict method.
Then initialize, train and print error.
<ASSISTANT_TASK:>
Python Code:
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
Explanation: Machine Learning Basics with Sklearn
First some imports for the notebook and visualization.
End of explanation
from sklearn.datasets import load_iris
iris = load_iris()
Explanation: Choosing a dataset
First of all you need a dataset to work on. To keep things simple we will use the iris dataset provided with scikit.
End of explanation
test_idx = [0, 50, 100]
train_y = np.delete(iris.target, test_idx)
train_X = np.delete(iris.data, test_idx, axis=0)
test_y = iris.target[test_idx]
test_X = iris.data[test_idx]
Explanation: Splitting the dataset
The dataset needs to be split into a training and test dataset.
This is done to first train our model and then test how good it is on data it has never seen before.
End of explanation
from sklearn import tree
clf = tree.DecisionTreeClassifier()
clf = clf.fit(train_X,train_y)
Explanation: Decision Tree Classifier
We will use a simple decision tree as the first model and train it.
End of explanation
from sklearn.externals.six import StringIO
import pydot
import matplotlib.image as mpimg
dot_data = StringIO()
tree.export_graphviz(clf, out_file=dot_data, feature_names=iris.feature_names,
        class_names=iris.target_names,
        filled=True, rounded=True, impurity=False)
pydot_graph = pydot.graph_from_dot_data(dot_data.getvalue())
png_str = pydot_graph.create_png(prog='dot')
# treat the dot output string as an image file
sio = StringIO()
sio.write(png_str)
sio.seek(0)
img = mpimg.imread(sio)
# plot the image
f, axes = plt.subplots(1, 1, figsize=(12,12))
imgplot = axes.imshow(img, aspect='equal')
plt.show()
Explanation: Visualize the decision tree
Whereas the decision tree is a simple graph we can visualize it quite simple.
End of explanation
from sklearn.metrics import accuracy_score
print(accuracy_score(test_y, clf.predict(test_X)))
Explanation: Evaluating the model
After the model is trained it has to be evaluated.
End of explanation
from sklearn.neighbors import KNeighborsClassifier
clf = KNeighborsClassifier()
clf = clf.fit(train_X,train_y)
print(accuracy_score(test_y, clf.predict(test_X)))
Explanation: KNN-Classifier
Let's try another classifier. Initialize, train, print error.
End of explanation
from scipy.spatial import distance
class ScrappyKNN(object):
    def fit(self, X_train, y_train):
        self.X_train = X_train
        self.y_train = y_train
        return self
    def predict(self, X_test):
        predictions = []
        for row in X_test:
            label = self.closest(row)
            predictions.append(label)
        return predictions
    def closest(self, row):
        best_dist = distance.euclidean(row, self.X_train[0])
        best_index = 0
        for i in range(1, len(self.X_train)):
            dist = distance.euclidean(row, self.X_train[i])
            if dist < best_dist:
                best_dist = dist
                best_index = i
        return self.y_train[best_index]
    
clf = ScrappyKNN()
clf = clf.fit(train_X,train_y)
print(accuracy_score(test_y, clf.predict(test_X)))
Explanation: Implementing your own KNN
Let's implement a simple knn classifier with k=1.
We have to implement the fit method and the predict method.
Then initialize, train and print error.
End of explanation
<END_TASK>