<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Decision Trees and Random Forests in Python
Learning Objectives
Explore and analyze data using a Pairplot
Train a single Decision Tree
Predict and evaluate the Decision Tree
Compare the Decision Tree model to a Random Forest
Introduction
In this lab, you explore and analyze data using a Pairplot, train a single Decision Tree, predict and evaluate the Decision Tree, and compare the Decision Tree model to a Random Forest.  Recall that the Decision Tree algorithm belongs to the family of supervised learning algorithms. Unlike other supervised learning algorithms, the decision tree algorithm can be used for solving both regression and classification problems too.  Simply, the goal of using a Decision Tree is to create a training model that can use to predict the class or value of the target variable by learning simple decision rules inferred from prior data(training data).
Each learning objective will correspond to a #TODO in the student lab notebook -- try to complete that notebook first before reviewing this solution notebook.
Load necessary libraries
We will start by importing the necessary libraries for this lab.
Step1:  Get the Data
Step2:  Exploratory Data Analysis
We'll just check out a simple pairplot for this small dataset.
Step3:  Train Test Split
Let's split up the data into a training set and a test set!
Step4:  Decision Trees
We'll start just by training a single decision tree.
Step5:  Prediction and Evaluation
Let's evaluate our decision tree.
Step6:  Tree Visualization
Scikit learn actually has some built-in visualization capabilities for decision trees, you won't use this often and it requires you to install the pydot library, but here is an example of what it looks like and the code to execute this
Step7:  Random Forests
Now let's compare the decision tree model to a random forest.
<ASSISTANT_TASK:>
Python Code:
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
%matplotlib inline
Explanation: Decision Trees and Random Forests in Python
Learning Objectives
Explore and analyze data using a Pairplot
Train a single Decision Tree
Predict and evaluate the Decision Tree
Compare the Decision Tree model to a Random Forest
Introduction
In this lab, you explore and analyze data using a Pairplot, train a single Decision Tree, predict and evaluate the Decision Tree, and compare the Decision Tree model to a Random Forest.  Recall that the Decision Tree algorithm belongs to the family of supervised learning algorithms. Unlike other supervised learning algorithms, the decision tree algorithm can be used for solving both regression and classification problems too.  Simply, the goal of using a Decision Tree is to create a training model that can use to predict the class or value of the target variable by learning simple decision rules inferred from prior data(training data).
Each learning objective will correspond to a #TODO in the student lab notebook -- try to complete that notebook first before reviewing this solution notebook.
Load necessary libraries
We will start by importing the necessary libraries for this lab.
End of explanation
df = pd.read_csv("../kyphosis.csv")
df.head()
Explanation: Get the Data
End of explanation
# TODO 1
sns.pairplot(df, hue="Kyphosis", palette="Set1")
Explanation: Exploratory Data Analysis
We'll just check out a simple pairplot for this small dataset.
End of explanation
from sklearn.model_selection import train_test_split
X = df.drop("Kyphosis", axis=1)
y = df["Kyphosis"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30)
Explanation: Train Test Split
Let's split up the data into a training set and a test set!
End of explanation
from sklearn.tree import DecisionTreeClassifier
dtree = DecisionTreeClassifier()
# TODO 2
dtree.fit(X_train, y_train)
Explanation: Decision Trees
We'll start just by training a single decision tree.
End of explanation
predictions = dtree.predict(X_test)
from sklearn.metrics import classification_report, confusion_matrix
# TODO 3a
print(classification_report(y_test, predictions))
# TODO 3b
print(confusion_matrix(y_test, predictions))
Explanation: Prediction and Evaluation
Let's evaluate our decision tree.
End of explanation
import pydot
from IPython.display import Image
from six import StringIO
from sklearn.tree import export_graphviz
features = list(df.columns[1:])
features
dot_data = StringIO()
export_graphviz(
    dtree, out_file=dot_data, feature_names=features, filled=True, rounded=True
)
graph = pydot.graph_from_dot_data(dot_data.getvalue())
Image(graph[0].create_png())
Explanation: Tree Visualization
Scikit learn actually has some built-in visualization capabilities for decision trees, you won't use this often and it requires you to install the pydot library, but here is an example of what it looks like and the code to execute this:
End of explanation
from sklearn.ensemble import RandomForestClassifier
rfc = RandomForestClassifier(n_estimators=100)
rfc.fit(X_train, y_train)
rfc_pred = rfc.predict(X_test)
# TODO 4a
print(confusion_matrix(y_test, rfc_pred))
# TODO 4b
print(classification_report(y_test, rfc_pred))
Explanation: Random Forests
Now let's compare the decision tree model to a random forest.
End of explanation
<END_TASK>