<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 <h1>gcForest Algorithm</h1>
<p>The gcForest algorithm was suggested in Zhou and Feng 2017 ( https
Step1:  <h2>Before starting, a word about sizes.</h2>
<p>*Note* 
Step2:  <p>First calling and training the algorithm.
A specificity here is the presence of the 'shape_1X' keyword to specify the shape of a single sample.
I have added it as pictures fed to the machinery might not be square.<br>
Obviously it is not very relevant for the iris data set but still, it has to be defined.</p>
<p>**New in version 0.1.3** 
Step3:  <p>Now checking the prediction for the test set
Step4:  <h2>Digits Example</h2>
<p>A much better example is the digits data set containing images of hand written digits.
The scikit data set can be viewed as a mini-MNIST for training purpose.</p>
Step5:  <p> ... taining gcForest ... (can take some time...) </p>
Step6:  <p> ... and predicting classes ... </p>
Step7:  <h2>Saving Models to Disk</h2>
<p>You probably don't want to re-train your classifier every day especially if you're using it on large data sets.
Fortunately there is a very easy way to save and load models to disk using ```sklearn.externals.joblib```</p>
<p>__Saving model
Step8:  <p>__Loading model__
Step9:  <h2>Using mg-scanning and cascade_forest Sperately</h2>
<p>As the Multi-Grain scanning and the cascade forest modules are quite independent it is possible to use them seperately.<br>
If a target `y` is given the code automaticcaly use it for training otherwise it recalls the last trained Random Forests to slice the data.</p>
Step10:  <p>It is now possible to use the mg_scanning output as input for cascade forests using different parameters. Note that the cascade forest module does not directly return predictions but probability predictions from each Random Forest in the last layer of the cascade. Hence the need to first take the mean of the output and then find the max.</p>
Step11:  <h3>Skipping mg_scanning</h3>
<p>It is also possible to directly use the cascade forest and skip the multi grain scanning step.</p>
<ASSISTANT_TASK:>
Python Code:
from GCForest import gcForest
from sklearn.datasets import load_iris, load_digits
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
Explanation: <h1>gcForest Algorithm</h1>
<p>The gcForest algorithm was suggested in Zhou and Feng 2017 ( https://arxiv.org/abs/1702.08835 , refer for this paper for technical details) and I provide here a python3 implementation of this algorithm.<br>
I chose to adopt the scikit-learn syntax for ease of use and hereafter I present how it can be used.</p>
End of explanation
# loading the data
iris = load_iris()
X = iris.data
y = iris.target
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.33)
Explanation: <h2>Before starting, a word about sizes.</h2>
<p>*Note* : I recommend the reader to look at this section with the original paper next to the computer to see what I am talking about.</p>
<p>The main technical problem in the present gcForest implementation so far is the memory usage when slicing the input data.
A naive calculation can actually give you an idea of the number and sizes of objects the algorithm will be dealing with.</p>
<p>Starting with a dataset of $N$ samples of size $[l,L]$ and with $C$ classes, the initial "size" is:<br><br>
$S_{D} = N.l.L$</p>
<p>**Slicing Step**<br>
If my window is of size $[w_l,w_L]$ and the chosen stride are $[s_l,s_L]$ then the number of slices per sample is :<br>
<br>
$n_{slices} = \left(\frac{l-w_l}{s_l}+1\right)\left(\frac{L-w_L}{s_L}+1\right)$<br><br>
Obviously the size of slice is $w_l.w_L$ hence the total size of the sliced data set is :<br><br>
$S_{sliced} = N.w_l.w_L.\left(\frac{l-w_l}{s_l}+1\right)\left(\frac{L-w_L}{s_L}+1\right)$<br>
This is when the memory consumption is its peak maximum.</p>
<p>**Class Vector after Multi-Grain Scanning**<br>
Now all slices are fed to the random forest to generate *class vectors*.
The number of class vector per random forest per window per sample is simply equal to the number of slices given to the random forest $n_{cv}(w) = n_{slices}(w)$.
Hence, if we have $N_{RF}$ random forest per window the size of a class vector is (recall we have $N$ samples and $C$ classes):<br><br>
$S_{cv}(w) = N.n_{cv}(w).N_{RF}.C$<br><br>
And finally the total size of the Multi-Grain Scanning output will be:<br><br>
$S_{mgs} = N.\sum_{w} N_{RF}.C.n_{cv}(w)$
</p>
<p>This short calculation is just meant to give you an idea of the data processing during the Multi-Grain Scanning phase. The actual memory consumption depends on the format given (aka float, int, double, etc.) and it might be worth looking at it carefully when dealing with large datasets.</p>
<h2>Iris example</h2>
<p>The iris data set is actually not a very good example as the gcForest algorithm is better suited for time series and images where informations can be found at different scales in one sample.<br>
Nonetheless it is still an easy way to test the method.</p>
End of explanation
gcf = gcForest(shape_1X=4, window=2, tolerance=0.0)
gcf.fit(X_tr, y_tr)
Explanation: <p>First calling and training the algorithm.
A specificity here is the presence of the 'shape_1X' keyword to specify the shape of a single sample.
I have added it as pictures fed to the machinery might not be square.<br>
Obviously it is not very relevant for the iris data set but still, it has to be defined.</p>
<p>**New in version 0.1.3** : possibility to directly use an int as shape_1X for sequence data.</p>
End of explanation
pred_X = gcf.predict(X_te)
print(pred_X)
# evaluating accuracy
accuracy = accuracy_score(y_true=y_te, y_pred=pred_X)
print('gcForest accuracy : {}'.format(accuracy))
Explanation: <p>Now checking the prediction for the test set:<p>
End of explanation
# loading the data
digits = load_digits()
X = digits.data
y = digits.target
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.4)
Explanation: <h2>Digits Example</h2>
<p>A much better example is the digits data set containing images of hand written digits.
The scikit data set can be viewed as a mini-MNIST for training purpose.</p>
End of explanation
gcf = gcForest(shape_1X=[8,8], window=[4,6], tolerance=0.0, min_samples_mgs=10, min_samples_cascade=7)
gcf.fit(X_tr, y_tr)
Explanation: <p> ... taining gcForest ... (can take some time...) </p>
End of explanation
pred_X = gcf.predict(X_te)
print(pred_X)
# evaluating accuracy
accuracy = accuracy_score(y_true=y_te, y_pred=pred_X)
print('gcForest accuracy : {}'.format(accuracy))
Explanation: <p> ... and predicting classes ... </p>
End of explanation
from sklearn.externals import joblib
joblib.dump(gcf, 'gcf_model.sav')
Explanation: <h2>Saving Models to Disk</h2>
<p>You probably don't want to re-train your classifier every day especially if you're using it on large data sets.
Fortunately there is a very easy way to save and load models to disk using ```sklearn.externals.joblib```</p>
<p>__Saving model:__</p>
End of explanation
gcf = joblib.load('gcf_model.sav')
Explanation: <p>__Loading model__:</p>
End of explanation
gcf = gcForest(shape_1X=[8,8], window=5, min_samples_mgs=10, min_samples_cascade=7)
X_tr_mgs = gcf.mg_scanning(X_tr, y_tr)
X_te_mgs = gcf.mg_scanning(X_te)
Explanation: <h2>Using mg-scanning and cascade_forest Sperately</h2>
<p>As the Multi-Grain scanning and the cascade forest modules are quite independent it is possible to use them seperately.<br>
If a target `y` is given the code automaticcaly use it for training otherwise it recalls the last trained Random Forests to slice the data.</p>
End of explanation
gcf = gcForest(tolerance=0.0, min_samples_mgs=10, min_samples_cascade=7)
_ = gcf.cascade_forest(X_tr_mgs, y_tr)
pred_proba = gcf.cascade_forest(X_te_mgs)
tmp = np.mean(pred_proba, axis=0)
preds = np.argmax(tmp, axis=1)
accuracy_score(y_true=y_te, y_pred=preds)
gcf = gcForest(tolerance=0.0, min_samples_mgs=20, min_samples_cascade=10)
_ = gcf.cascade_forest(X_tr_mgs, y_tr)
pred_proba = gcf.cascade_forest(X_te_mgs)
tmp = np.mean(pred_proba, axis=0)
preds = np.argmax(tmp, axis=1)
accuracy_score(y_true=y_te, y_pred=preds)
Explanation: <p>It is now possible to use the mg_scanning output as input for cascade forests using different parameters. Note that the cascade forest module does not directly return predictions but probability predictions from each Random Forest in the last layer of the cascade. Hence the need to first take the mean of the output and then find the max.</p>
End of explanation
gcf = gcForest(tolerance=0.0, min_samples_cascade=20)
_ = gcf.cascade_forest(X_tr, y_tr)
pred_proba = gcf.cascade_forest(X_te)
tmp = np.mean(pred_proba, axis=0)
preds = np.argmax(tmp, axis=1)
accuracy_score(y_true=y_te, y_pred=preds)
Explanation: <h3>Skipping mg_scanning</h3>
<p>It is also possible to directly use the cascade forest and skip the multi grain scanning step.</p>
End of explanation
<END_TASK>