<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 H2O Tutorial
Step1:  If you already have an H2O cluster running that you'd like to connect to (for example, in a multi-node Hadoop environment), then you can specify the IP and port of that cluster as follows
Step2:  Download Data
The following code downloads a copy of the Wisconsin Diagnostic Breast Cancer dataset.
We can import the data directly into H2O using the Python API.
Step3:  Explore Data
Once we have loaded the data, let's take a quick look.  First the dimension of the frame
Step4:  Now let's take a look at the top of the frame
Step5:  The first two columns contain an ID and the resposne.  The "diagnosis" column is the response.  Let's take a look at the column names.  The data contains derived features from the medical images of the tumors.
Step6:  To select a subset of the columns to look at, typical Pandas indexing applies
Step7:  Now let's select a single column, for example -- the response column, and look at the data more closely
Step8:  It looks like a binary response, but let's validate that assumption
Step9:  We can query the categorical "levels" as well ('B' and 'M' stand for "Benign" and "Malignant" diagnosis)
Step10:  Since "diagnosis" column is the response we would like to predict, we may want to check if there are any missing values, so let's look for NAs.  To figure out which, if any, values are missing, we can use the isna method on the diagnosis column.  The columns in an H2O Frame are also H2O Frames themselves, so all the methods that apply to a Frame also apply to a single column.
Step11:  The isna method doesn't directly answer the question, "Does the diagnosis column contain any NAs?", rather it returns a 0 if that cell is not missing (Is NA? FALSE == 0) and a 1 if it is missing (Is NA? TRUE == 1).  So if there are no missing values, then summing over the whole column should produce a summand equal to 0.0.  Let's take a look
Step12:  Great, no missing labels.  
Out of curiosity, let's see if there is any missing data in this frame
Step13:  The next thing I may wonder about in a binary classification problem is the distribution of the response in the training data.  Is one of the two outcomes under-represented in the training set?  Many real datasets have what's called an "imbalanace" problem, where one of the classes has far fewer training examples than the other class.  Let's take a look at the distribution, both visually and numerically.
Step14:  Ok, the data is not exactly evenly distributed between the two classes -- there are almost twice as many Benign samples as there are Malicious samples.  However, this level of imbalance shouldn't be much of an issue for the machine learning algos.  (We will revisit this later in the modeling section below).
Step15:  Machine Learning in H2O
We will do a quick demo of the H2O software -- trying to predict malignant tumors using various machine learning algorithms.
Specify the predictor set and response
The response, y, is the 'diagnosis' column, and the predictors, x, are all the columns aside from the first two columns ('id' and 'diagnosis').
Step16:  Split H2O Frame into a train and test set
Step17:  Train and Test a GBM model
Step18:  We first create a model object of class, "H2OGradientBoostingEstimator".  This does not actually do any training, it just sets the model up for training by specifying model parameters.
Step19:  The model object, like all H2O estimator objects, has a train method, which will actually perform model training.  At this step we specify the training and (optionally) a validation set, along with the response and predictor variables.
Step20:  Inspect Model
The type of results shown when you print a model, are determined by the following
Step21:  Model Performance on a Test Set
Once a model has been trained, you can also use it to make predictions on a test set.  In the case above, we passed the test set as the validation_frame in training, so we have technically already created test set predictions and performance.  
However, when performing model selection over a variety of model parameters, it is common for users to break their dataset into three pieces
Step22:  Cross-validated Performance
To perform k-fold cross-validation, you use the same code as above, but you specify nfolds as an integer greater than 1, or add a "fold_column" to your H2O Frame which indicates a fold ID for each row.
Unless you have a specific reason to manually assign the observations to folds, you will find it easiest to simply use the nfolds argument.
When performing cross-validation, you can still pass a validation_frame, but you can also choose to use the original dataset that contains all the rows.  We will cross-validate a model below using the original H2O Frame which we call data.
Step23:  Grid Search
One way of evaluting models with different parameters is to perform a grid search over a set of parameter values.  For example, in GBM, here are three model parameters that may be useful to search over
Step24:  Define an "H2OGridSearch" object by specifying the algorithm (GBM) and the hyper parameters
Step25:  An "H2OGridSearch" object also has a train method, which is used to train all the models in the grid.
Step26:  Compare Models
<ASSISTANT_TASK:>
Python Code:
import h2o
# Start an H2O Cluster on your local machine
h2o.init()
Explanation: H2O Tutorial: Breast Cancer Classification
Author: Erin LeDell
Contact: erin@h2o.ai
This tutorial steps through a quick introduction to H2O's Python API. The goal of this tutorial is to introduce through a complete example H2O's capabilities from Python. Also, to help those that are accustomed to Scikit Learn and Pandas, the demo will be specific call outs for differences between H2O and those packages; this is intended to help anyone that needs to do machine learning on really Big Data make the transition. It is not meant to be a tutorial on machine learning or algorithms.
Detailed documentation about H2O's and the Python API is available at http://docs.h2o.ai.
Install H2O in Python
Prerequisites
This tutorial assumes you have Python 2.7 installed.  The h2o Python package has a few dependencies which can be installed using pip. The packages that are required are (which also have their own dependencies):
bash
pip install requests
pip install tabulate
pip install scikit-learn
If you have any problems (for example, installing the scikit-learn package), check out this page for tips.
Install h2o
Once the dependencies are installed, you can install H2O.  We will use the latest stable version of the h2o package, which is called "Tibshirani-3."  The installation instructions are on the "Install in Python" tab on this page.
```bash
The following command removes the H2O module for Python (if it already exists).
pip uninstall h2o
Next, use pip to install this version of the H2O Python module.
pip install http://h2o-release.s3.amazonaws.com/h2o/rel-tibshirani/3/Python/h2o-3.6.0.3-py2.py3-none-any.whl
```
Start up an H2O cluster
In a Python terminal, we can import the h2o package and start up an H2O cluster.
End of explanation
# This will not actually do anything since it's a fake IP address
# h2o.init(ip="123.45.67.89", port=54321)
Explanation: If you already have an H2O cluster running that you'd like to connect to (for example, in a multi-node Hadoop environment), then you can specify the IP and port of that cluster as follows:
End of explanation
csv_url = "https://h2o-public-test-data.s3.amazonaws.com/smalldata/wisc/wisc-diag-breast-cancer-shuffled.csv"
data = h2o.import_file(csv_url)
Explanation: Download Data
The following code downloads a copy of the Wisconsin Diagnostic Breast Cancer dataset.
We can import the data directly into H2O using the Python API.
End of explanation
data.shape
Explanation: Explore Data
Once we have loaded the data, let's take a quick look.  First the dimension of the frame:
End of explanation
data.head()
Explanation: Now let's take a look at the top of the frame:
End of explanation
data.columns
Explanation: The first two columns contain an ID and the resposne.  The "diagnosis" column is the response.  Let's take a look at the column names.  The data contains derived features from the medical images of the tumors.
End of explanation
columns = ["id", "diagnosis", "area_mean"]
data[columns].head()
Explanation: To select a subset of the columns to look at, typical Pandas indexing applies:
End of explanation
data['diagnosis']
Explanation: Now let's select a single column, for example -- the response column, and look at the data more closely:
End of explanation
data['diagnosis'].unique()
data['diagnosis'].nlevels()
Explanation: It looks like a binary response, but let's validate that assumption:
End of explanation
data['diagnosis'].levels()
Explanation: We can query the categorical "levels" as well ('B' and 'M' stand for "Benign" and "Malignant" diagnosis):
End of explanation
data.isna()
data['diagnosis'].isna()
Explanation: Since "diagnosis" column is the response we would like to predict, we may want to check if there are any missing values, so let's look for NAs.  To figure out which, if any, values are missing, we can use the isna method on the diagnosis column.  The columns in an H2O Frame are also H2O Frames themselves, so all the methods that apply to a Frame also apply to a single column.
End of explanation
data['diagnosis'].isna().sum()
Explanation: The isna method doesn't directly answer the question, "Does the diagnosis column contain any NAs?", rather it returns a 0 if that cell is not missing (Is NA? FALSE == 0) and a 1 if it is missing (Is NA? TRUE == 1).  So if there are no missing values, then summing over the whole column should produce a summand equal to 0.0.  Let's take a look:
End of explanation
data.isna().sum()
Explanation: Great, no missing labels.  
Out of curiosity, let's see if there is any missing data in this frame:
End of explanation
# TO DO: Insert a bar chart or something showing the proportion of M to B in the response.
data['diagnosis'].table()
Explanation: The next thing I may wonder about in a binary classification problem is the distribution of the response in the training data.  Is one of the two outcomes under-represented in the training set?  Many real datasets have what's called an "imbalanace" problem, where one of the classes has far fewer training examples than the other class.  Let's take a look at the distribution, both visually and numerically.
End of explanation
n = data.shape[0]  # Total number of training samples
data['diagnosis'].table()['Count']/n
Explanation: Ok, the data is not exactly evenly distributed between the two classes -- there are almost twice as many Benign samples as there are Malicious samples.  However, this level of imbalance shouldn't be much of an issue for the machine learning algos.  (We will revisit this later in the modeling section below).
End of explanation
y = 'diagnosis'
x = data.columns
del x[0:1]
x
Explanation: Machine Learning in H2O
We will do a quick demo of the H2O software -- trying to predict malignant tumors using various machine learning algorithms.
Specify the predictor set and response
The response, y, is the 'diagnosis' column, and the predictors, x, are all the columns aside from the first two columns ('id' and 'diagnosis').
End of explanation
train, test = data.split_frame(ratios=[0.75], seed=1)
train.shape
test.shape
Explanation: Split H2O Frame into a train and test set
End of explanation
# Import H2O GBM:
from h2o.estimators.gbm import H2OGradientBoostingEstimator
Explanation: Train and Test a GBM model
End of explanation
model = H2OGradientBoostingEstimator(distribution='bernoulli',
                                    ntrees=100,
                                    max_depth=4,
                                    learn_rate=0.1)
Explanation: We first create a model object of class, "H2OGradientBoostingEstimator".  This does not actually do any training, it just sets the model up for training by specifying model parameters.
End of explanation
model.train(x=x, y=y, training_frame=train, validation_frame=test)
Explanation: The model object, like all H2O estimator objects, has a train method, which will actually perform model training.  At this step we specify the training and (optionally) a validation set, along with the response and predictor variables.
End of explanation
print(model)
Explanation: Inspect Model
The type of results shown when you print a model, are determined by the following:
- Model class of the estimator (e.g. GBM, RF, GLM, DL)
- The type of machine learning problem (e.g. binary classification, multiclass classification, regression)
- The data you specify (e.g. training_frame only, training_frame and validation_frame, or training_frame and nfolds)
Below, we see a GBM Model Summary, as well as training and validation metrics since we supplied a validation_frame.  Since this a binary classification task, we are shown the relevant performance metrics, which inclues: MSE, R^2, LogLoss, AUC and Gini.  Also, we are shown a Confusion Matrix, where the threshold for classification is chosen automatically (by H2O) as the threshold which maximizes the F1 score.
The scoring history is also printed, which shows the performance metrics over some increment such as "number of trees" in the case of GBM and RF.
Lastly, for tree-based methods (GBM and RF), we also print variable importance.
End of explanation
perf = model.model_performance(test)
perf.auc()
Explanation: Model Performance on a Test Set
Once a model has been trained, you can also use it to make predictions on a test set.  In the case above, we passed the test set as the validation_frame in training, so we have technically already created test set predictions and performance.  
However, when performing model selection over a variety of model parameters, it is common for users to break their dataset into three pieces:  Training, Validation and Test.
After training a variety of models using different parameters (and evaluating them on a validation set), the user may choose a single model and then evaluate model performance on a separate test set.  This is when the model_performance method, shown below, is most useful.
End of explanation
cvmodel = H2OGradientBoostingEstimator(distribution='bernoulli',
                                       ntrees=100,
                                       max_depth=4,
                                       learn_rate=0.1,
                                       nfolds=5)
cvmodel.train(x=x, y=y, training_frame=data)
Explanation: Cross-validated Performance
To perform k-fold cross-validation, you use the same code as above, but you specify nfolds as an integer greater than 1, or add a "fold_column" to your H2O Frame which indicates a fold ID for each row.
Unless you have a specific reason to manually assign the observations to folds, you will find it easiest to simply use the nfolds argument.
When performing cross-validation, you can still pass a validation_frame, but you can also choose to use the original dataset that contains all the rows.  We will cross-validate a model below using the original H2O Frame which we call data.
End of explanation
ntrees_opt = [5,50,100]
max_depth_opt = [2,3,5]
learn_rate_opt = [0.1,0.2]
hyper_params = {'ntrees': ntrees_opt, 
                'max_depth': max_depth_opt,
                'learn_rate': learn_rate_opt}
Explanation: Grid Search
One way of evaluting models with different parameters is to perform a grid search over a set of parameter values.  For example, in GBM, here are three model parameters that may be useful to search over:
- ntrees: Number of trees
- max_depth: Maximum depth of a tree
- learn_rate: Learning rate in the GBM
We will define a grid as follows:
End of explanation
from h2o.grid.grid_search import H2OGridSearch
gs = H2OGridSearch(H2OGradientBoostingEstimator, hyper_params = hyper_params)
Explanation: Define an "H2OGridSearch" object by specifying the algorithm (GBM) and the hyper parameters:
End of explanation
gs.train(x=x, y=y, training_frame=train, validation_frame=test)
Explanation: An "H2OGridSearch" object also has a train method, which is used to train all the models in the grid.
End of explanation
print(gs)
# print out the auc for all of the models
for g in gs:
    print(g.model_id + " auc: " + str(g.auc()))
#TO DO: Compare grid search models
Explanation: Compare Models
End of explanation
<END_TASK>