<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Key Requirements for the iRF scikit-learn implementation
The following is a documentation of the main requirements for the iRF implementation
Typical Setup
Step1:  Step 1
Step2:  Check out the data
Step3:  Step 2
Step4:  Design the single function to get the key tree information
Get data from the first and second decision tree
Step5:  Decision Tree 0 (First) - Get output
Check the output against the decision tree graph
Step6:  Step 3
Step7:  Step 3.2 Display Feature Importances Graphically (just for interest)
Step8:  Putting it all together
Create a dictionary object to include all of the random forest objects
Step9:  Check the final dictionary of outputs
Step10:  Now we can start setting up the RIT class
Overview
At it's core, the RIT is comprised of 3 main modules
* FILTERING
Step11:  Get the leaf node 1's paths
Get the unique feature paths where the leaf node predicted class is just 1
<ASSISTANT_TASK:>
Python Code:
%matplotlib inline
import matplotlib.pyplot as plt
import pydotplus
import numpy as np
import pprint
from sklearn import metrics
from sklearn import tree
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn import tree
from sklearn.tree import _tree
from IPython.display import display, Image
from sklearn.datasets import load_iris
from sklearn.datasets import load_breast_cancer
from functools import reduce
# Import our custom utilities
from imp import reload
from utils import utils
reload(utils)
Explanation: Key Requirements for the iRF scikit-learn implementation
The following is a documentation of the main requirements for the iRF implementation
Typical Setup
End of explanation
%timeit
X_train, X_test, y_train, y_test, rf = utils.generate_rf_example(sklearn_ds = load_breast_cancer()
                                                           , train_split_propn = 0.9
                                                           , n_estimators = 3
                                                           , random_state_split = 2017
                                                           , random_state_classifier = 2018)
Explanation: Step 1: Fit the Initial Random Forest
Just fit every feature with equal weights per the usual random forest code e.g. DecisionForestClassifier in scikit-learn
End of explanation
print("Training feature dimensions", X_train.shape, sep = ":\n")
print("\n")
print("Training outcome dimensions", y_train.shape, sep = ":\n")
print("\n")
print("Test feature dimensions", X_test.shape, sep = ":\n")
print("\n")
print("Test outcome dimensions", y_test.shape, sep = ":\n")
print("\n")
print("first 5 rows of the training set features", X_train[:5], sep = ":\n")
print("\n")
print("first 5 rows of the training set outcomes", y_train[:5], sep = ":\n")
X_train.shape[0]
breast_cancer = load_breast_cancer()
breast_cancer.data.shape[0]
Explanation: Check out the data
End of explanation
# Import our custom utilities
rf.n_estimators
estimator0 = rf.estimators_[0] # First tree
estimator1 = rf.estimators_[1] # Second tree
estimator2 = rf.estimators_[2] # Second tree
Explanation: Step 2: For each Tree get core leaf node features
For each decision tree in the classifier, get:
The list of leaf nodes
Depth of the leaf node    
Leaf node predicted class i.e. {0, 1}
Probability of predicting class in leaf node
Number of observations in the leaf node i.e. weight of node
Get the 2 Decision trees to use for testing
End of explanation
tree_dat0 = utils.getTreeData(X_train = X_train, dtree = estimator0, root_node_id = 0)
tree_dat1 = utils.getTreeData(X_train = X_train, dtree = estimator1, root_node_id = 0)
tree_dat1 = utils.getTreeData(X_train = X_train, dtree = estimator2, root_node_id = 0)
Explanation: Design the single function to get the key tree information
Get data from the first and second decision tree
End of explanation
# Now plot the trees individually
utils.draw_tree(decision_tree = estimator0)
utils.prettyPrintDict(inp_dict = tree_dat0)
# Count the number of samples passing through the leaf nodes
sum(tree_dat0['tot_leaf_node_values'])
Explanation: Decision Tree 0 (First) - Get output
Check the output against the decision tree graph
End of explanation
feature_importances = rf.feature_importances_
std = np.std([dtree.feature_importances_ for dtree in rf.estimators_]
             , axis=0)
feature_importances_rank_idx = np.argsort(feature_importances)[::-1]
# Check that the feature importances are standardized to 1
print(sum(feature_importances))
Explanation: Step 3: Get the Gini Importance of Weights for the Random Forest
For the first random forest we just need to get the Gini Importance of Weights
Step 3.1 Get them numerically - most important
End of explanation
# Print the feature ranking
print("Feature ranking:")
for f in range(X_train.shape[1]):
    print("%d. feature %d (%f)" % (f + 1
                                   , feature_importances_rank_idx[f]
                                   , feature_importances[feature_importances_rank_idx[f]]))
    
# Plot the feature importances of the forest
plt.figure()
plt.title("Feature importances")
plt.bar(range(X_train.shape[1])
        , feature_importances[feature_importances_rank_idx]
        , color="r"
        , yerr = std[feature_importances_rank_idx], align="center")
plt.xticks(range(X_train.shape[1]), feature_importances_rank_idx)
plt.xlim([-1, X_train.shape[1]])
plt.show()
Explanation: Step 3.2 Display Feature Importances Graphically (just for interest)
End of explanation
# CHECK: If the random forest objects are going to be really large in size
#        we could just omit them and only return our custom summary outputs
rf_metrics = utils.getValidationMetrics(rf, y_true = y_test, X_test = X_test)
all_rf_outputs = {"rf_obj" : rf,
                 "feature_importances" : feature_importances,
                 "feature_importances_rank_idx" : feature_importances_rank_idx,
                 "rf_metrics" : rf_metrics}
# CHECK: The following should be paralellized!
# CHECK: Whether we can maintain X_train correctly as required
for idx, dtree in enumerate(rf.estimators_):
    dtree_out = utils.getTreeData(X_train = X_train, dtree = dtree, root_node_id = 0)
    # Append output to dictionary
    all_rf_outputs["dtree" + str(idx)] = dtree_out
Explanation: Putting it all together
Create a dictionary object to include all of the random forest objects
End of explanation
utils.prettyPrintDict(inp_dict = all_rf_outputs)
Explanation: Check the final dictionary of outputs
End of explanation
utils.prettyPrintDict(inp_dict = all_rf_outputs['rf_metrics'])
all_rf_outputs['dtree0']
Explanation: Now we can start setting up the RIT class
Overview
At it's core, the RIT is comprised of 3 main modules
* FILTERING: Subsetting to either the 1's or the 0's
* RANDOM SAMPLING: The path-nodes in a weighted manner, with/ without replacement, within tree/ outside tree
* INTERSECTION: Intersecting the selected node paths in a systematic manner
For now we will just work with a single decision tree outputs
End of explanation
uniq_feature_paths = all_rf_outputs['dtree0']['all_uniq_leaf_paths_features']
leaf_node_classes  = all_rf_outputs['dtree0']['all_leaf_node_classes']
ones_only = [i for i, j in zip(uniq_feature_paths, leaf_node_classes) 
               if j == 1]
ones_only
print("Number of leaf nodes", len(all_rf_outputs['dtree0']['all_uniq_leaf_paths_features']), sep = ":\n")
print("Number of leaf nodes with 1 class", len(ones_only), sep = ":\n")
# Just pick the last seven cases, we are going to manually construct
# binary RIT of depth 3 i.e. max 2**3 -1 = 7 intersecting nodes
ones_only_seven = ones_only[-7:]
ones_only_seven
# Construct a binary version of the RIT manually!
# This should come in useful for unit tests!
node0 = ones_only_seven[-1]
node1 = np.intersect1d(node0, ones_only_seven[-2])
node2 = np.intersect1d(node1, ones_only_seven[-3])
node3 = np.intersect1d(node1, ones_only_seven[-4])
node4 = np.intersect1d(node0, ones_only_seven[-5])
node5 = np.intersect1d(node4, ones_only_seven[-6])
node6 = np.intersect1d(node4, ones_only_seven[-7])
intersected_nodes_seven = [node0, node1, node2, node3, node4, node5, node6]
for idx, node in enumerate(intersected_nodes_seven):
    print("node" + str(idx), node)
rit_output = reduce(np.union1d, (node2, node3, node5, node6))
rit_output
Explanation: Get the leaf node 1's paths
Get the unique feature paths where the leaf node predicted class is just 1
End of explanation
<END_TASK>