<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Udacity Machine Learning Evaluation mini-project
Prep stuff
Step1:  Training a decision tree on this starter data
Step2:  Counts of actual and predicted values
Step3:  Which turn out to match up very poorly. No true positives. Just guessing 0 for everyone would in fact be more accurate.
Step4:  Precision and Recall
Step5:  These are not even slightly good news
Step6:  Same thing with some fake data for comparison
<ASSISTANT_TASK:>
Python Code:
import pickle
import sys
sys.path.append("../tools/")
from feature_format import featureFormat, targetFeatureSplit
data_dict = pickle.load(open("../final_project/final_project_dataset.pkl", "r") )
features_list = ["poi", "salary"]
data = featureFormat(data_dict, features_list)
labels, features = targetFeatureSplit(data)
Explanation: Udacity Machine Learning Evaluation mini-project
Prep stuff:
End of explanation
from sklearn.tree import DecisionTreeClassifier
from sklearn.cross_validation import train_test_split
features_train, features_test, labels_train, labels_test = train_test_split(features,labels,test_size=0.3,random_state=42)
clf = DecisionTreeClassifier()
clf.fit(features_train,labels_train)
pred = clf.predict(features_test)
Explanation: Training a decision tree on this starter data:
End of explanation
# ref http://stackoverflow.com/questions/10741346
import numpy as np
unique, counts = np.unique(labels_test, return_counts=True)
print "true labels"
print np.asarray((unique, counts)).T
print "predicted labels"
unique, counts = np.unique(pred, return_counts=True)
print np.asarray((unique, counts)).T
Explanation: Counts of actual and predicted values:
End of explanation
print "number of true positives:",sum((labels_test==1) & (pred ==1))
Explanation: Which turn out to match up very poorly. No true positives. Just guessing 0 for everyone would in fact be more accurate.
End of explanation
from sklearn.metrics import precision_score, recall_score
Explanation: Precision and Recall:
End of explanation
print "precision:",precision_score(labels_test,pred)
print "recall:",recall_score(labels_test,pred)
Explanation: These are not even slightly good news:
End of explanation
predictions = np.array([0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 1, 0, 1, 0, 1])
true_labels = np.array([0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 0, 1, 0, 0])
print "number of true positives:",sum((true_labels==1) & (predictions==1))
print "number of false positives:",sum((true_labels==0) & (predictions==1))
print "number of true negatives:",sum((true_labels==0) & (predictions==0))
print "number of false negatives:",sum((true_labels==1) & (predictions==0))
print "precision:", 6/(6+3.)
print "recall:", 6/(6+2.)
Explanation: Same thing with some fake data for comparison:
End of explanation
<END_TASK>