<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 <!--BOOK_INFORMATION-->
<a href="https
Step1:  As in previous examples, all data is contained in a 2-D feature matrix data.data, where the rows represent data samples, and the columns are the feature values
Step2:  With a look at the provided feature names, we recognize some that we mentioned above
Step3:  Since this is a binary classification task, we expect to find exactly two target names
Step4:  Let's split the dataset into training and test sets using a healthy 80-20 split
Step5:  Building the decision tree
Step6:  Since we did not specify any pre-pruning parameters, we would expect this decision tree to grow quite large and result in a perfect score on the training set
Step7:  However, to our surprise, the test error is not too shabby, either
Step8:  Now we want to do some model exploration. For example, we mentioned above that the depth of a tree influences its performance. If we wanted to study this dependency more systematically, we could repeat building the tree for different values of max_depth
Step9:  For each of these values, we want to run the full model cascade from start to finish. We also want to record the train and test scores. We do this in a for loop
Step10:  We can plot the scores as a function of the tree depth using Matplotlib
Step11:  Let's do one more. What about the minimum numbers of samples required to make a node a leaf node?
We repeat the procedure from above
Step12:  This leads to a plot that looks quite different from the one before
<ASSISTANT_TASK:>
Python Code:
from sklearn import datasets
data = datasets.load_breast_cancer()
Explanation: <!--BOOK_INFORMATION-->
<a href="https://www.packtpub.com/big-data-and-business-intelligence/machine-learning-opencv" target="_blank"><img align="left" src="data/cover.jpg" style="width: 76px; height: 100px; background: white; padding: 1px; border: 1px solid black; margin-right:10px;"></a>
This notebook contains an excerpt from the book Machine Learning for OpenCV by Michael Beyeler.
The code is released under the MIT license,
and is available on GitHub.
Note that this excerpt contains only the raw code - the book is rich with additional explanations and illustrations.
If you find this content useful, please consider supporting the work by
buying the book!
<!--NAVIGATION-->
< Building Our First Decision Tree | Contents | Using Decision Trees for Regression >
Using Decision Trees to Diagnose Breast Cancer
Now that we have built our first decision trees, it's time to turn our attention to a real dataset: The Breast Cancer Wisconsin dataset https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+(Diagnostic).
In order to make the take feasible, the researchers performed feature extraction on the images, like we did in Chapter 4, Representing Data and Engineering Features. They went through a total of 569 images, and extracted 30 different features that describe the characteristics of the cell nuclei present in the images, including:
cell nucleus texture (represented by the standard deviation of the gray-scale values)
cell nucleus size (calculated as the mean of distances from center to points on the perimeter)
tissue smoothness (local variation in radius lengths)
tissue compactness
The goal of the research was then to classify tissue samples into benign and malignant (a binary classification task).
Loading the dataset
The full dataset is part of Scikit-Learn's example datasets:
End of explanation
data.data.shape
Explanation: As in previous examples, all data is contained in a 2-D feature matrix data.data, where the rows represent data samples, and the columns are the feature values:
End of explanation
data.feature_names
Explanation: With a look at the provided feature names, we recognize some that we mentioned above:
End of explanation
data.target_names
Explanation: Since this is a binary classification task, we expect to find exactly two target names:
End of explanation
import sklearn.model_selection as ms
X_train, X_test, y_train, y_test = ms.train_test_split(data.data, data.target, test_size=0.2, random_state=42)
X_train.shape, X_test.shape
Explanation: Let's split the dataset into training and test sets using a healthy 80-20 split:
End of explanation
from sklearn import tree
dtc = tree.DecisionTreeClassifier(random_state=42)
dtc.fit(X_train, y_train)
Explanation: Building the decision tree
End of explanation
dtc.score(X_train, y_train)
Explanation: Since we did not specify any pre-pruning parameters, we would expect this decision tree to grow quite large and result in a perfect score on the training set:
End of explanation
dtc.score(X_test, y_test)
with open("tree.dot", 'w') as f:
    f = tree.export_graphviz(dtc, out_file=f,
                             feature_names=data.feature_names,
                             class_names=data.target_names)
Explanation: However, to our surprise, the test error is not too shabby, either:
End of explanation
import numpy as np
max_depths = np.array([1, 2, 3, 5, 7, 9, 11])
Explanation: Now we want to do some model exploration. For example, we mentioned above that the depth of a tree influences its performance. If we wanted to study this dependency more systematically, we could repeat building the tree for different values of max_depth:
End of explanation
train_score = []
test_score = []
for d in max_depths:
    dtc = tree.DecisionTreeClassifier(max_depth=d, random_state=42)
    dtc.fit(X_train, y_train)
    train_score.append(dtc.score(X_train, y_train))
    test_score.append(dtc.score(X_test, y_test))
Explanation: For each of these values, we want to run the full model cascade from start to finish. We also want to record the train and test scores. We do this in a for loop:
End of explanation
import matplotlib.pyplot as plt
%matplotlib inline
plt.style.use('ggplot')
plt.figure(figsize=(10, 6))
plt.plot(max_depths, train_score, 'o-', linewidth=3, label='train')
plt.plot(max_depths, test_score, 's-', linewidth=3, label='test')
plt.xlabel('max_depth')
plt.ylabel('score')
plt.ylim(0.85, 1.1)
plt.legend()
Explanation: We can plot the scores as a function of the tree depth using Matplotlib:
End of explanation
train_score = []
test_score = []
min_samples = np.array([2, 4, 8, 16, 32])
for s in min_samples:
    dtc = tree.DecisionTreeClassifier(min_samples_leaf=s, random_state=42)
    dtc.fit(X_train, y_train)
    train_score.append(dtc.score(X_train, y_train))
    test_score.append(dtc.score(X_test, y_test))
Explanation: Let's do one more. What about the minimum numbers of samples required to make a node a leaf node?
We repeat the procedure from above:
End of explanation
plt.figure(figsize=(10, 6))
plt.plot(min_samples, train_score, 'o-', linewidth=3, label='train')
plt.plot(min_samples, test_score, 's-', linewidth=3, label='test')
plt.xlabel('min_samples_leaf')
plt.ylabel('score')
plt.ylim(0.9, 1)
plt.legend()
Explanation: This leads to a plot that looks quite different from the one before:
End of explanation
<END_TASK>