<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Vertex AI
Step1:  Please ignore any incompatibility warnings.
Restart the kernel before proceeding further (On the Notebook menu - Kernel - Restart Kernel).
Set your project ID
Finally, you must initialize the client library before you can send requests to the Vertex AI service. With the Python SDK, you initialize the client library as shown in the following cell. This tutorial also uses the Cloud Storage Python library for accessing batch prediction results.
Be sure to provide the ID for your Google Cloud project in the project variable. This notebook uses the us-central1 region, although you can change it to another region. 
If you don't know your project ID, you may be able to get your project ID using gcloud.
Step2:  Create a dataset and import your data
The notebook uses the 'Happy Moments' dataset for demonstration purposes. You can change it to another text classification dataset that conforms to the data preparation requirements.
Using the Python SDK, you can create a dataset and import the dataset in one call to TextDataset.create(), as shown in the following cell.
Creating and importing data is a long-running operation. This next step can take a while. The sample waits for the operation to complete, outputting statements as the operation progresses. The statements contain the full name of the dataset that you will use in the following section.
Note
Step3:  Train your text classification model
Once your dataset has finished importing data, you are ready to train your model. To do this, you first need the full resource name of your dataset, where the full name has the format projects/[YOUR_PROJECT]/locations/us-central1/datasets/[YOUR_DATASET_ID]. If you don't have the resource name handy, you can list all of the datasets in your project using TextDataset.list(). 
As shown in the following code block, you can pass in the display name of your dataset in the call to list() to filter the results.
Step4:  When you create a new model, you need a reference to the TextDataset object that corresponds to your dataset. You can use the ds variable you created previously when you created the dataset or you can also list all of your datasets to get a reference to your dataset. Each item returned from TextDataset.list() is an instance of TextDataset.
The following code block shows how to instantiate a TextDataset object using a dataset ID. Note that this code is intentionally verbose for demonstration purposes.
Step5:  Now you can begin training your model. Training the model is a two part process
Step6:  Get and review model evaluation scores
After your model has finished training, you can review the evaluation scores for it.
First, you need to get a reference to the new model.  As with datasets, you can either use the reference to the model variable you created when deployed the model or you can list all of the models in your project. When listing your models, you can provide filter criteria to narrow down your search.
Step7:  Using the model name (in the format projects/[PROJECT_NAME]/locations/us-central1/models/[MODEL_ID]), you can get its model evaluations. To get model evaluations, you must use the underlying service client.
Building a service client requires that you provide the name of the regionalized hostname used for your model. In this tutorial, the hostname is us-central1-aiplatform.googleapis.com because the model was created in the us-central1 location.
Step8:  Before you can view the model evaluation you must first list all of the evaluations for that model. Each model can have multiple evaluations, although a new model is likely to only have one.
Step9:  Now that you have the model evaluation, you can look at your model's scores. If you have questions about what the scores mean, review the public documentation.
The results returned from the service are formatted as google.protobuf.Value objects. You can transform the return object as a dict for easier reading and parsing.
Step10:  Deploy your text classification model
Once your model has completed training, you must deploy it to an endpoint to get online predictions from it. When you deploy the model to an endpoint, a copy of the model is made on the endpoint with a new resource name and display name.
You can deploy multiple models to the same endpoint and split traffic between the various models assigned to the endpoint. However, you must deploy one model at a time to the endpoint. To change the traffic split percentages, you must assign new values on your second (and subsequent) models each time you deploy a new model.
The following code block demonstrates how to deploy a model. The code snippet relies on the Python SDK to create a new endpoint for deployment. The call to model.deploy() returns a reference to an Endpoint object--you need this reference for online predictions in the next section.
Step11:  In case you didn't record the name of the new endpoint, you can get a list of all your endpoints as you did before with datasets and models. For each endpoint, you can list the models deployed to that endpoint. To get a reference to the model that you just deployed, you can check the display_name of each model deployed to the endpoint against the model you're looking for.
Step12:  Get online predictions from your model
Now that you have your endpoint's resource name, you can get online predictions from the text classification model. To get the online prediction, you send a prediction request to your endpoint.
Step13:  Get batch predictions from your model
You can get batch predictions from a text classification model without deploying it. You must first format all of your prediction instances (prediction input) in JSONL format and you must store the JSONL file in a Google Cloud Storage bucket. You must also provide a Google Cloud Storage bucket to hold your prediction output.
To start, you must first create your predictions input file in JSONL format. Each line in the JSONL document needs to be formatted like so
Step14:  For batch prediction, you must supply the following
Step15:  Now that you have the bucket with the prediction instances ready, you can send a batch prediction request to Vertex AI. When you send a request to the service, you must provide the URI of your JSONL file and your output bucket, including the gs
Step16:  Once the batch prediction job completes, the Python SDK prints out the resource name of the batch prediction job in the format projects/[PROJECT_ID]/locations/[LOCATION]/batchPredictionJobs/[BATCH_PREDICTION_JOB_ID]. You can query the Vertex AI service for the status of the batch prediction job using its ID.
The following code snippet demonstrates how to create an instance of the BatchPredictionJob class to review its status. Note that you need the full resource name printed out from the Python SDK for this snippet.
Step17:  After the batch job has completed, you can view the results of the job in your output Storage bucket. You might want to first list all of the files in your output bucket to find the URI of the output file.
Step18:  The output from the batch prediction job should be contained in a folder (or prefix) that includes the name of the batch prediction job plus a time stamp for when it was created.
For example, if your batch prediction job name is my-job and your bucket name is my-bucket, the URI of the folder containing your output might look like the following
Step19:  With all of the results files downloaded locally, you can open them and read the results. In this tutorial, you use the jsonlines library to read the output results.
The following cell opens up the JSONL output file and then prints the predictions for each instance.
Step20:  Cleaning up
To clean up all Google Cloud resources used in this project, you can delete the Google Cloud project you used for the tutorial.
Otherwise, you can delete the individual resources you created in this tutorial
<ASSISTANT_TASK:>
Python Code:
# Setup your dependencies
import os
# The Google Cloud Notebook product has specific requirements
IS_GOOGLE_CLOUD_NOTEBOOK = os.path.exists("/opt/deeplearning/metadata/env_version")
# Google Cloud Notebook requires dependencies to be installed with '--user'
USER_FLAG = ""
if IS_GOOGLE_CLOUD_NOTEBOOK:
    USER_FLAG = "--user"
# Upgrade the specified package to the newest available version
! pip install {USER_FLAG} --upgrade google-cloud-aiplatform google-cloud-storage jsonlines
Explanation: Vertex AI: Create, Train, and Deploy an AutoML Text Classification Model
Learning Objective
In this notebook, you learn how to:
Create a dataset and import data.
Train an AutoML model.
Get and review evaluations for the model.
Deploy a model to an endpoint.
Get online predictions.
Get batch predictions.
Introduction
In this notebook, you use the "Happy Moments" sample dataset to train a model. The resulting model classifies happy moments into categories that reflect the causes of happiness. This notebook walks you through the major phases of building and using a text classification model on Vertex AI.
Each learning objective will correspond to a #TODO in this student lab notebook -- try to complete this notebook first and then review the solution notebook.  
Make sure to enable the Vertex AI, Cloud Storage, and Compute Engine APIs.
Install additional packages
This notebook uses the Python SDK for Vertex AI, which is contained in the python-aiplatform package. You must first install the package into your development environment.
End of explanation
# import necessary libraries
import os
from datetime import datetime
import jsonlines
from google.cloud import aiplatform, storage
from google.protobuf import json_format
PROJECT_ID = "[your-project-id]"
REGION = "us-central1"
# Get your Google Cloud project ID from gcloud
if not os.getenv("IS_TESTING"):
    shell_output=!gcloud config list --format 'value(core.project)' 2>/dev/null
    PROJECT_ID = shell_output[0]
    print("Project ID: ", PROJECT_ID)
aiplatform.init(project=PROJECT_ID, location=REGION)
Explanation: Please ignore any incompatibility warnings.
Restart the kernel before proceeding further (On the Notebook menu - Kernel - Restart Kernel).
Set your project ID
Finally, you must initialize the client library before you can send requests to the Vertex AI service. With the Python SDK, you initialize the client library as shown in the following cell. This tutorial also uses the Cloud Storage Python library for accessing batch prediction results.
Be sure to provide the ID for your Google Cloud project in the project variable. This notebook uses the us-central1 region, although you can change it to another region. 
If you don't know your project ID, you may be able to get your project ID using gcloud.
End of explanation
# TODO
# Use a timestamp to ensure unique resources
TIMESTAMP = # TODO: Your code goes here
src_uris = "gs://cloud-ml-data/NL-classification/happiness.csv"
display_name = f"e2e-text-dataset-{TIMESTAMP}"
# TODO
# create a dataset and import the dataset
ds = # TODO: Your code goes here(
    display_name=display_name,
    gcs_source=src_uris,
    import_schema_uri=aiplatform.schema.dataset.ioformat.text.single_label_classification,
    sync=True,
)
Explanation: Create a dataset and import your data
The notebook uses the 'Happy Moments' dataset for demonstration purposes. You can change it to another text classification dataset that conforms to the data preparation requirements.
Using the Python SDK, you can create a dataset and import the dataset in one call to TextDataset.create(), as shown in the following cell.
Creating and importing data is a long-running operation. This next step can take a while. The sample waits for the operation to complete, outputting statements as the operation progresses. The statements contain the full name of the dataset that you will use in the following section.
Note: You can close the noteboook while you wait for this operation to complete.
End of explanation
# TODO
# list all of the datasets in your project
datasets = # TODO: Your code goes here(filter=f'display_name="{display_name}"')
print(datasets)
Explanation: Train your text classification model
Once your dataset has finished importing data, you are ready to train your model. To do this, you first need the full resource name of your dataset, where the full name has the format projects/[YOUR_PROJECT]/locations/us-central1/datasets/[YOUR_DATASET_ID]. If you don't have the resource name handy, you can list all of the datasets in your project using TextDataset.list(). 
As shown in the following code block, you can pass in the display name of your dataset in the call to list() to filter the results.
End of explanation
# Get the dataset ID if it's not available
dataset_id = "[your-dataset-id]"
if dataset_id == "[your-dataset-id]":
    # Use the reference to the new dataset captured when we created it
    dataset_id = ds.resource_name.split("/")[-1]
    print(f"Dataset ID: {dataset_id}")
text_dataset = aiplatform.TextDataset(dataset_id)
Explanation: When you create a new model, you need a reference to the TextDataset object that corresponds to your dataset. You can use the ds variable you created previously when you created the dataset or you can also list all of your datasets to get a reference to your dataset. Each item returned from TextDataset.list() is an instance of TextDataset.
The following code block shows how to instantiate a TextDataset object using a dataset ID. Note that this code is intentionally verbose for demonstration purposes.
End of explanation
# Define the training job
training_job_display_name = f"e2e-text-training-job-{TIMESTAMP}"
# TODO
# constructs a AutoML Text Training Job
job = # TODO: Your code goes here(
    display_name=training_job_display_name,
    prediction_type="classification",
    multi_label=False,
)
model_display_name = f"e2e-text-classification-model-{TIMESTAMP}"
# TODO
# Run the training job
model = # TODO: Your code goes here(
    dataset=text_dataset,
    model_display_name=model_display_name,
    training_fraction_split=0.7,
    validation_fraction_split=0.2,
    test_fraction_split=0.1,
    sync=True,
)
Explanation: Now you can begin training your model. Training the model is a two part process:
Define the training job. You must provide a display name and the type of training you want when you define the training job.
Run the training job. When you run the training job, you need to supply a reference to the dataset to use for training. At this step, you can also configure the data split percentages.
You do not need to specify data splits. The training job has a default setting of  training 80%/ testing 10%/ validate 10% if you don't provide these values.
To train your model, you call AutoMLTextTrainingJob.run() as shown in the following snippets. The method returns a reference to your new Model object.
As with importing data into the dataset, training your model can take a substantial amount of time. The client library prints out operation status messages while the training pipeline operation processes. You must wait for the training process to complete before you can get the resource name and ID of your new model, which is required for model evaluation and model deployment.
Note: You can close the notebook while you wait for the operation to complete.
End of explanation
# TODO
# list the aiplatform model
models = # TODO: Your code goes here(filter=f'display_name="{model_display_name}"')
print(models)
Explanation: Get and review model evaluation scores
After your model has finished training, you can review the evaluation scores for it.
First, you need to get a reference to the new model.  As with datasets, you can either use the reference to the model variable you created when deployed the model or you can list all of the models in your project. When listing your models, you can provide filter criteria to narrow down your search.
End of explanation
# Get the ID of the model
model_name = "[your-model-resource-name]"
if model_name == "[your-model-resource-name]":
    # Use the `resource_name` of the Model instance you created previously
    model_name = model.resource_name
    print(f"Model name: {model_name}")
# Get a reference to the Model Service client
client_options = {"api_endpoint": "us-central1-aiplatform.googleapis.com"}
model_service_client = aiplatform.gapic.ModelServiceClient(
    client_options=client_options
)
Explanation: Using the model name (in the format projects/[PROJECT_NAME]/locations/us-central1/models/[MODEL_ID]), you can get its model evaluations. To get model evaluations, you must use the underlying service client.
Building a service client requires that you provide the name of the regionalized hostname used for your model. In this tutorial, the hostname is us-central1-aiplatform.googleapis.com because the model was created in the us-central1 location.
End of explanation
model_evaluations = model_service_client.list_model_evaluations(parent=model_name)
model_evaluation = list(model_evaluations)[0]
Explanation: Before you can view the model evaluation you must first list all of the evaluations for that model. Each model can have multiple evaluations, although a new model is likely to only have one.
End of explanation
model_eval_dict = json_format.MessageToDict(model_evaluation._pb)
metrics = model_eval_dict["metrics"]
confidence_metrics = metrics["confidenceMetrics"]
print(f'Area under precision-recall curve (AuPRC): {metrics["auPrc"]}')
for confidence_scores in confidence_metrics:
    metrics = confidence_scores.keys()
    print("\n")
    for metric in metrics:
        print(f"\t{metric}: {confidence_scores[metric]}")
Explanation: Now that you have the model evaluation, you can look at your model's scores. If you have questions about what the scores mean, review the public documentation.
The results returned from the service are formatted as google.protobuf.Value objects. You can transform the return object as a dict for easier reading and parsing.
End of explanation
deployed_model_display_name = f"e2e-deployed-text-classification-model-{TIMESTAMP}"
# TODO
# deploy a model
endpoint = # TODO: Your code goes here(
    deployed_model_display_name=deployed_model_display_name, sync=True
)
Explanation: Deploy your text classification model
Once your model has completed training, you must deploy it to an endpoint to get online predictions from it. When you deploy the model to an endpoint, a copy of the model is made on the endpoint with a new resource name and display name.
You can deploy multiple models to the same endpoint and split traffic between the various models assigned to the endpoint. However, you must deploy one model at a time to the endpoint. To change the traffic split percentages, you must assign new values on your second (and subsequent) models each time you deploy a new model.
The following code block demonstrates how to deploy a model. The code snippet relies on the Python SDK to create a new endpoint for deployment. The call to model.deploy() returns a reference to an Endpoint object--you need this reference for online predictions in the next section.
End of explanation
endpoints = aiplatform.Endpoint.list()
endpoint_with_deployed_model = []
for endpoint_ in endpoints:
    for model in endpoint_.list_models():
        if model.display_name.find(deployed_model_display_name) == 0:
            endpoint_with_deployed_model.append(endpoint_)
print(endpoint_with_deployed_model)
Explanation: In case you didn't record the name of the new endpoint, you can get a list of all your endpoints as you did before with datasets and models. For each endpoint, you can list the models deployed to that endpoint. To get a reference to the model that you just deployed, you can check the display_name of each model deployed to the endpoint against the model you're looking for.
End of explanation
endpoint_name = "[your-endpoint-name]"
if endpoint_name == "[your-endpoint-name]":
    endpoint_name = endpoint.resource_name
print(f"Endpoint name: {endpoint_name}")
endpoint = aiplatform.Endpoint(endpoint_name)
content = "I got a high score on my math final!"
# TODO
# send a prediction request to your endpoint
response = # TODO: Your code goes here(instances=[{"content": content}])
for prediction_ in response.predictions:
    ids = prediction_["ids"]
    display_names = prediction_["displayNames"]
    confidence_scores = prediction_["confidences"]
    for count, id in enumerate(ids):
        print(f"Prediction ID: {id}")
        print(f"Prediction display name: {display_names[count]}")
        print(f"Prediction confidence score: {confidence_scores[count]}")
Explanation: Get online predictions from your model
Now that you have your endpoint's resource name, you can get online predictions from the text classification model. To get the online prediction, you send a prediction request to your endpoint.
End of explanation
instances = [
    "We hiked through the woods and up the hill to the ice caves",
    "My kitten is so cute",
]
input_file_name = "batch-prediction-input.jsonl"
Explanation: Get batch predictions from your model
You can get batch predictions from a text classification model without deploying it. You must first format all of your prediction instances (prediction input) in JSONL format and you must store the JSONL file in a Google Cloud Storage bucket. You must also provide a Google Cloud Storage bucket to hold your prediction output.
To start, you must first create your predictions input file in JSONL format. Each line in the JSONL document needs to be formatted like so:
{ "content": "gs://sourcebucket/datasets/texts/source_text.txt", "mimeType": "text/plain"}
The content field in the JSON structure must be a Google Cloud Storage URI to another document that contains the text input for prediction.
See the documentation for more information.
End of explanation
TIMESTAMP = datetime.now().strftime("%Y%m%d%H%M%S")
BUCKET_NAME = "[your-bucket-name]"
if BUCKET_NAME == "" or BUCKET_NAME is None or BUCKET_NAME == "[your-bucket-name]":
    BUCKET_NAME = f"automl-text-notebook-{TIMESTAMP}"
BUCKET_URI = f"gs://{BUCKET_NAME}"
! gsutil mb -l $REGION $BUCKET_URI
# Instantiate the Storage client and create the new bucket
storage = storage.Client()
bucket = storage.bucket(BUCKET_NAME)
# Iterate over the prediction instances, creating a new TXT file
# for each.
input_file_data = []
for count, instance in enumerate(instances):
    instance_name = f"input_{count}.txt"
    instance_file_uri = f"{BUCKET_URI}/{instance_name}"
    # Add the data to store in the JSONL input file.
    tmp_data = {"content": instance_file_uri, "mimeType": "text/plain"}
    input_file_data.append(tmp_data)
    # Create the new instance file
    blob = bucket.blob(instance_name)
    blob.upload_from_string(instance)
input_str = "\n".join([str(d) for d in input_file_data])
file_blob = bucket.blob(f"{input_file_name}")
file_blob.upload_from_string(input_str)
Explanation: For batch prediction, you must supply the following:
All of your prediction instances as individual files on Google Cloud Storage, as TXT files for your instances
A JSONL file that lists the URIs of all your prediction instances
A Google Cloud Storage bucket to hold the output from batch prediction
For this tutorial, the following cells create a new Storage bucket, upload individual prediction instances as text files to the bucket, and then create the JSONL file with the URIs of your prediction instances.
End of explanation
job_display_name = "e2e-text-classification-batch-prediction-job"
model = aiplatform.Model(model_name=model_name)
# TODO
# create a batch prediction job
batch_prediction_job = # TODO: Your code goes here(
    job_display_name=job_display_name,
    gcs_source=f"{BUCKET_URI}/{input_file_name}",
    gcs_destination_prefix=f"{BUCKET_URI}/output",
    sync=True,
)
batch_prediction_job_name = batch_prediction_job.resource_name
Explanation: Now that you have the bucket with the prediction instances ready, you can send a batch prediction request to Vertex AI. When you send a request to the service, you must provide the URI of your JSONL file and your output bucket, including the gs:// protocols.
With the Python SDK, you can create a batch prediction job by calling Model.batch_predict().
End of explanation
from google.cloud.aiplatform import jobs
batch_job = jobs.BatchPredictionJob(batch_prediction_job_name)
print(f"Batch prediction job state: {str(batch_job.state)}")
Explanation: Once the batch prediction job completes, the Python SDK prints out the resource name of the batch prediction job in the format projects/[PROJECT_ID]/locations/[LOCATION]/batchPredictionJobs/[BATCH_PREDICTION_JOB_ID]. You can query the Vertex AI service for the status of the batch prediction job using its ID.
The following code snippet demonstrates how to create an instance of the BatchPredictionJob class to review its status. Note that you need the full resource name printed out from the Python SDK for this snippet.
End of explanation
BUCKET_OUTPUT = f"{BUCKET_URI}/output"
! gsutil ls -a $BUCKET_OUTPUT
Explanation: After the batch job has completed, you can view the results of the job in your output Storage bucket. You might want to first list all of the files in your output bucket to find the URI of the output file.
End of explanation
RESULTS_DIRECTORY = "prediction_results"
RESULTS_DIRECTORY_FULL = f"{RESULTS_DIRECTORY}/output"
# Create missing directories
os.makedirs(RESULTS_DIRECTORY, exist_ok=True)
# Get the Cloud Storage paths for each result
! gsutil -m cp -r $BUCKET_OUTPUT $RESULTS_DIRECTORY
# Get most recently modified directory
latest_directory = max(
    [
        os.path.join(RESULTS_DIRECTORY_FULL, d)
        for d in os.listdir(RESULTS_DIRECTORY_FULL)
    ],
    key=os.path.getmtime,
)
print(f"Local results folder: {latest_directory}")
Explanation: The output from the batch prediction job should be contained in a folder (or prefix) that includes the name of the batch prediction job plus a time stamp for when it was created.
For example, if your batch prediction job name is my-job and your bucket name is my-bucket, the URI of the folder containing your output might look like the following:
gs://my-bucket/output/prediction-my-job-2021-06-04T19:54:25.889262Z/
To read the batch prediction results, you must download the file locally and open the file. The next cell copies all of the files in the BUCKET_OUTPUT_FOLDER into a local folder.
End of explanation
# Get downloaded results in directory
results_files = []
for dirpath, subdirs, files in os.walk(latest_directory):
    for file in files:
        if file.find("predictions") >= 0:
            results_files.append(os.path.join(dirpath, file))
# Consolidate all the results into a list
results = []
for results_file in results_files:
    # Open each result
    with jsonlines.open(results_file) as reader:
        for result in reader.iter(type=dict, skip_invalid=True):
            instance = result["instance"]
            prediction = result["prediction"]
            print(f"\ninstance: {instance['content']}")
            for key, output in prediction.items():
                print(f"\n{key}: {output}")
Explanation: With all of the results files downloaded locally, you can open them and read the results. In this tutorial, you use the jsonlines library to read the output results.
The following cell opens up the JSONL output file and then prints the predictions for each instance.
End of explanation
if os.getenv("IS_TESTING"):
    ! gsutil rm -r $BUCKET_URI
batch_job.delete()
# `force` parameter ensures that models are undeployed before deletion
endpoint.delete(force=True)
model.delete()
text_dataset.delete()
# Training job
job.delete()
Explanation: Cleaning up
To clean up all Google Cloud resources used in this project, you can delete the Google Cloud project you used for the tutorial.
Otherwise, you can delete the individual resources you created in this tutorial:
Dataset
Training job
Model
Endpoint
Batch prediction
Batch prediction bucket
End of explanation
<END_TASK>