<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Text Classification
This text classification example
Step1:  Set Up Verta
Step2:  Imports
Step3:  Create a helper function to plot graphs
Step4:  Setup input pipeline
The IMDB large movie review dataset is a binary classification dataset—all the reviews have either a positive or negative sentiment.
Download the dataset using TFDS. The dataset comes with an inbuilt subword tokenizer.
Step5:  As this is a subwords tokenizer, it can be passed any string and the tokenizer will tokenize it.
Step6:  The tokenizer encodes the string by breaking it into subwords if the word is not in its dictionary.
Step7:  Create the model
Build a tf.keras.Sequential model and start with an embedding layer. An embedding layer stores one vector per word. When called, it converts the sequences of word indices to sequences of vectors. These vectors are trainable. After training (on enough data), words with similar meanings often have similar vectors.
This index-lookup is much more efficient than the equivalent operation of passing a one-hot encoded vector through a tf.keras.layers.Dense layer.
A recurrent neural network (RNN) processes sequence input by iterating through the elements. RNNs pass the outputs from one timestep to their input—and then to the next.
The tf.keras.layers.Bidirectional wrapper can also be used with an RNN layer. This propagates the input forward and backwards through the RNN layer and then concatenates the output. This helps the RNN to learn long range dependencies.
Step8:  Compile the Keras model to configure the training process
Step9:  Train the model
Step10:  Testing and Prediction
Step11:  The above model does not mask the padding applied to the sequences. This can lead to skewness if we train on padded sequences and test on un-padded sequences. Ideally the model would learn to ignore the padding, but as you can see below it does have a small effect on the output.
If the prediction is >= 0.5, it is positive else it is negative.
Step12:  Saving Models
<ASSISTANT_TASK:>
Python Code:
# Python 3.6
!pip install verta
!pip install matplotlib==3.1.1
!pip install tensorflow==2.0.0-beta1
!pip install tensorflow-hub==0.5.0
!pip install tensorflow-datasets==1.0.2
Explanation: Text Classification
This text classification example:
* trains a recurrent neural network on the IMDB large movie review dataset for sentiment analysis. 
* uses verta's Python client logging observations and artifacts
Set Up Environment
This notebook has been tested with the following package versions:
(you may need to change pip to pip3, depending on your own Python environment)
End of explanation
HOST = 'app.verta.ai'
PROJECT_NAME = 'Text-Classification'
EXPERIMENT_NAME = 'RNN'
# import os
# os.environ['VERTA_EMAIL'] = 
# os.environ['VERTA_DEV_KEY'] = 
from verta import Client
from verta.utils import ModelAPI
client = Client(HOST, use_git=False)
proj = client.set_project(PROJECT_NAME)
expt = client.set_experiment(EXPERIMENT_NAME)
run = client.set_experiment_run()
Explanation: Set Up Verta
End of explanation
from __future__ import absolute_import, division, print_function, unicode_literals
import warnings
warnings.filterwarnings("ignore", category=FutureWarning)
import json
import matplotlib.pyplot as plt
import tensorflow_datasets as tfds
import tensorflow as tf
Explanation: Imports
End of explanation
def plot_graphs(history, string, run, plot_title):
    plt.plot(history.history[string])
    plt.plot(history.history['val_'+string])
    plt.xlabel('Epochs')
    plt.ylabel(string)
    plt.legend([string, 'val_'+string])
    run.log_image(plot_title, plt)
    plt.show()
Explanation: Create a helper function to plot graphs:
End of explanation
# loading the dataset
dataset, info = tfds.load('imdb_reviews/subwords8k', with_info=True,
                          as_supervised=True)
train_dataset, test_dataset = dataset['train'], dataset['test']
Explanation: Setup input pipeline
The IMDB large movie review dataset is a binary classification dataset—all the reviews have either a positive or negative sentiment.
Download the dataset using TFDS. The dataset comes with an inbuilt subword tokenizer.
End of explanation
tokenizer = info.features['text'].encoder
print ('Vocabulary size: {}'.format(tokenizer.vocab_size))
sample_string = 'The latest Marvel movie - Endgame was amazing!'
tokenized_string = tokenizer.encode(sample_string)
print ('Tokenized string is {}'.format(tokenized_string))
original_string = tokenizer.decode(tokenized_string)
print ('The original string: {}'.format(original_string))
assert original_string == sample_string
Explanation: As this is a subwords tokenizer, it can be passed any string and the tokenizer will tokenize it.
End of explanation
for ts in tokenized_string:
    print ('{} ----> {}'.format(ts, tokenizer.decode([ts])))
BUFFER_SIZE = 10000
BATCH_SIZE = 64
train_dataset = train_dataset.shuffle(BUFFER_SIZE)
train_dataset = train_dataset.padded_batch(BATCH_SIZE, train_dataset.output_shapes)
test_dataset = test_dataset.padded_batch(BATCH_SIZE, test_dataset.output_shapes)
Explanation: The tokenizer encodes the string by breaking it into subwords if the word is not in its dictionary.
End of explanation
hyperparams = {
    'num_epochs': 10,
    'optimizer': 'adam',
    'loss': 'binary_crossentropy',
    'vocab_size': tokenizer.vocab_size,
    'metrics': 'accuracy'
}
# logging hyperparameters
run.log_hyperparameters(hyperparams)
model = tf.keras.Sequential([
    tf.keras.layers.Embedding(tokenizer.vocab_size, 64),
    tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64)),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.Dense(1, activation='sigmoid')
])
Explanation: Create the model
Build a tf.keras.Sequential model and start with an embedding layer. An embedding layer stores one vector per word. When called, it converts the sequences of word indices to sequences of vectors. These vectors are trainable. After training (on enough data), words with similar meanings often have similar vectors.
This index-lookup is much more efficient than the equivalent operation of passing a one-hot encoded vector through a tf.keras.layers.Dense layer.
A recurrent neural network (RNN) processes sequence input by iterating through the elements. RNNs pass the outputs from one timestep to their input—and then to the next.
The tf.keras.layers.Bidirectional wrapper can also be used with an RNN layer. This propagates the input forward and backwards through the RNN layer and then concatenates the output. This helps the RNN to learn long range dependencies.
End of explanation
model.compile(loss=hyperparams['loss'],
              optimizer=hyperparams['optimizer'],
              metrics=[hyperparams['metrics']])
Explanation: Compile the Keras model to configure the training process:
End of explanation
# called at the end of each epoch - logging loss, accuracy as observations for the run
class LossAndErrorLoggingCallback(tf.keras.callbacks.Callback):
    def on_epoch_end(self, epoch, logs=None):
        print('The average loss for epoch {} is {:7.2f}, accuracy is {:7.2f}.'.format(epoch, logs['loss'], logs['accuracy']))
        run.log_observation("train_loss", float(logs['loss']))
        run.log_observation("train_acc", float(logs['accuracy']))
        run.log_observation("val_loss", float(logs['val_loss']))
        run.log_observation("val_acc", float(logs['val_accuracy']))
history = model.fit(train_dataset,
                    epochs=hyperparams['num_epochs'],
                    validation_data=test_dataset,
                    callbacks=[LossAndErrorLoggingCallback()])
Explanation: Train the model
End of explanation
test_loss, test_acc = model.evaluate(test_dataset)
# logging metrics
run.log_metric('test_loss', float(test_loss))
run.log_metric('test_accuracy', float(test_acc))
print('Test Loss: {}'.format(test_loss))
print('Test Accuracy: {}'.format(test_acc))
Explanation: Testing and Prediction
End of explanation
def pad_to_size(vec, size):
    zeros = [0] * (size - len(vec))
    vec.extend(zeros)
    return vec
def sample_predict(sentence, pad):
    tokenized_sample_pred_text = tokenizer.encode(sample_pred_text)
    if pad:
        tokenized_sample_pred_text = pad_to_size(tokenized_sample_pred_text, 64)
    predictions = model.predict(tf.expand_dims(tokenized_sample_pred_text, 0))
    return (predictions)
# predict on a sample text without padding.
sample_pred_text = ('Spiderman: Far From Home did not disappoint! I loved it!')
predictions = sample_predict(sample_pred_text, pad=False)
print (predictions)
predictions = sample_predict(sample_pred_text, pad=True)
print (predictions)
# plotting graphs to see variation in accuracy and loss
plot_graphs(history, 'accuracy', run, 'epochs_vs_acc')
plot_graphs(history, 'loss', run, 'epochs_vs_loss')
Explanation: The above model does not mask the padding applied to the sequences. This can lead to skewness if we train on padded sequences and test on un-padded sequences. Ideally the model would learn to ignore the padding, but as you can see below it does have a small effect on the output.
If the prediction is >= 0.5, it is positive else it is negative.
End of explanation
# serialize model to JSON
model_json = model.to_json()
with open("model.json", "w") as json_file:
    json_file.write(model_json)
# serialize weights to HDF5
model.save_weights("model.h5")
print("Saved model to disk")
run.log_artifact('model_summary_json', 'model.json')
run.log_artifact('model_weights', 'model.h5')
Explanation: Saving Models
End of explanation
<END_TASK>