<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 <!--BOOK_INFORMATION-->
<a href="https
Step1:  Although the original images consisted of 92 x 112 pixel images, the version available
through scikit-learn contains images downscaled to 64 x 64 pixels.
To get a sense of the dataset, we can plot some example images. Let's pick eight indices
from the dataset in a random order
Step2:  We can plot these example images using Matplotlib, but we need to make sure we reshape
the column vectors to 64 x 64 pixel images before plotting
Step3:  You can see how all the faces are taken against a dark background and are upright. The
facial expression varies drastically from image to image, making this an interesting
classification problem. Try not to laugh at some of them!
Preprocessing the dataset
Before we can pass the dataset to the classifier, we need to preprocess it following the best
practices from Chapter 4, Representing Data and Engineering Features.
Specifically, we want to make sure that all example images have the same mean grayscale
level
Step4:  We repeat this procedure for every image to make sure the feature values of every data
point (that is, a row in X) are centered around zero
Step5:  The preprocessed data can be visualized using the preceding code
Step6:  Training and testing the random forest
We continue to follow our best practice to split the data into training and test sets
Step7:  Then we are ready to apply a random forest to the data
Step8:  Here we want to create an ensemble with 50 decision trees
Step9:  Because we have a large number of categories (that is, 40), we want to make sure the
random forest is set up to handle them accordingly
Step10:  We can play with other optional arguments, such as the number of data points required in a
node before it can be split
Step11:  However, we might not want to limit the depth of each tree. This is again, a parameter we
will have to experiment with in the end. But for now, let's set it to a large integer value,
making the depth effectively unconstrained
Step12:  Then we can fit the classifier to the training data
Step13:  We can check the resulting depth of the tree using the following function
Step14:  This means that although we allowed the tree to go up to depth 1000, in the end only 25
layers were needed.
The evaluation of the classifier is done once again by predicting the labels first (y_hat) and
then passing them to the accuracy_score function
Step15:  We find 87% accuracy, which turns out to be much better than with a single decision tree
Step16:  Not bad! We can play with the optional parameters to see if we get better. The most
important one seems to be the number of trees in the forest. We can repeat the experiment
with a forest made from 100 trees
<ASSISTANT_TASK:>
Python Code:
from sklearn.datasets import fetch_olivetti_faces
dataset = fetch_olivetti_faces()
X = dataset.data
y = dataset.target
Explanation: <!--BOOK_INFORMATION-->
<a href="https://www.packtpub.com/big-data-and-business-intelligence/machine-learning-opencv" target="_blank"><img align="left" src="data/cover.jpg" style="width: 76px; height: 100px; background: white; padding: 1px; border: 1px solid black; margin-right:10px;"></a>
This notebook contains an excerpt from the book Machine Learning for OpenCV by Michael Beyeler.
The code is released under the MIT license,
and is available on GitHub.
Note that this excerpt contains only the raw code - the book is rich with additional explanations and illustrations.
If you find this content useful, please consider supporting the work by
buying the book!
<!--NAVIGATION-->
< Combining Decision Trees Into a Random Forest | Contents | Implementing AdaBoost >
Using Random Forests for Face Recognition
A popular dataset that we haven't talked much about yet is the Olivetti face dataset.
The Olivetti face dataset was collected in 1990 by AT&T Laboratories Cambridge. The
dataset comprises facial images of 40 distinct subjects, taken at different times and under
different lighting conditions. In addition, subjects varied their facial expression
(open/closed eyes, smiling/not smiling) and their facial details (glasses/no glasses).
Images were then quantized to 256 grayscale levels and stored as unsigned 8-bit integers.
Because there are 40 distinct subjects, the dataset comes with 40 distinct target labels.
Recognizing faces thus constitutes an example of a multiclass classification task.
Loading the dataset
Like many other classic datasets, the Olivetti face dataset can be loaded using scikit-learn:
End of explanation
import numpy as np
np.random.seed(21)
idx_rand = np.random.randint(len(X), size=8)
Explanation: Although the original images consisted of 92 x 112 pixel images, the version available
through scikit-learn contains images downscaled to 64 x 64 pixels.
To get a sense of the dataset, we can plot some example images. Let's pick eight indices
from the dataset in a random order:
End of explanation
import matplotlib.pyplot as plt
%matplotlib inline
plt.figure(figsize=(14, 8))
for p, i in enumerate(idx_rand):
    plt.subplot(2, 4, p + 1)
    plt.imshow(X[i, :].reshape((64, 64)), cmap='gray')
    plt.axis('off')
Explanation: We can plot these example images using Matplotlib, but we need to make sure we reshape
the column vectors to 64 x 64 pixel images before plotting:
End of explanation
n_samples, n_features = X.shape
X -= X.mean(axis=0)
Explanation: You can see how all the faces are taken against a dark background and are upright. The
facial expression varies drastically from image to image, making this an interesting
classification problem. Try not to laugh at some of them!
Preprocessing the dataset
Before we can pass the dataset to the classifier, we need to preprocess it following the best
practices from Chapter 4, Representing Data and Engineering Features.
Specifically, we want to make sure that all example images have the same mean grayscale
level:
End of explanation
X -= X.mean(axis=1).reshape(n_samples, -1)
Explanation: We repeat this procedure for every image to make sure the feature values of every data
point (that is, a row in X) are centered around zero:
End of explanation
plt.figure(figsize=(14, 8))
for p, i in enumerate(idx_rand):
    plt.subplot(2, 4, p + 1)
    plt.imshow(X[i, :].reshape((64, 64)), cmap='gray')
    plt.axis('off')
plt.savefig('olivetti-pre.png')
Explanation: The preprocessed data can be visualized using the preceding code:
End of explanation
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, random_state=21
)
Explanation: Training and testing the random forest
We continue to follow our best practice to split the data into training and test sets:
End of explanation
import cv2
rtree = cv2.ml.RTrees_create()
Explanation: Then we are ready to apply a random forest to the data:
End of explanation
num_trees = 50
eps = 0.01
criteria = (cv2.TERM_CRITERIA_MAX_ITER + cv2.TERM_CRITERIA_EPS,
            num_trees, eps)
rtree.setTermCriteria(criteria)
Explanation: Here we want to create an ensemble with 50 decision trees:
End of explanation
rtree.setMaxCategories(len(np.unique(y)))
Explanation: Because we have a large number of categories (that is, 40), we want to make sure the
random forest is set up to handle them accordingly:
End of explanation
rtree.setMinSampleCount(2)
Explanation: We can play with other optional arguments, such as the number of data points required in a
node before it can be split:
End of explanation
rtree.setMaxDepth(1000)
Explanation: However, we might not want to limit the depth of each tree. This is again, a parameter we
will have to experiment with in the end. But for now, let's set it to a large integer value,
making the depth effectively unconstrained:
End of explanation
rtree.train(X_train, cv2.ml.ROW_SAMPLE, y_train);
Explanation: Then we can fit the classifier to the training data:
End of explanation
rtree.getMaxDepth()
Explanation: We can check the resulting depth of the tree using the following function:
End of explanation
_, y_hat = rtree.predict(X_test)
from sklearn.metrics import accuracy_score
accuracy_score(y_test, y_hat)
Explanation: This means that although we allowed the tree to go up to depth 1000, in the end only 25
layers were needed.
The evaluation of the classifier is done once again by predicting the labels first (y_hat) and
then passing them to the accuracy_score function:
End of explanation
from sklearn.tree import DecisionTreeClassifier
tree = DecisionTreeClassifier(random_state=21, max_depth=25)
tree.fit(X_train, y_train)
tree.score(X_test, y_test)
Explanation: We find 87% accuracy, which turns out to be much better than with a single decision tree:
End of explanation
num_trees = 100
eps = 0.01
criteria = (cv2.TERM_CRITERIA_MAX_ITER + cv2.TERM_CRITERIA_EPS,
            num_trees, eps)
rtree.setTermCriteria(criteria)
rtree.train(X_train, cv2.ml.ROW_SAMPLE, y_train);
_, y_hat = rtree.predict(X_test)
accuracy_score(y_test, y_hat)
Explanation: Not bad! We can play with the optional parameters to see if we get better. The most
important one seems to be the number of trees in the forest. We can repeat the experiment
with a forest made from 100 trees:
End of explanation
<END_TASK>