<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Notebook is a revised version of notebook from Amy Wu and Shen Zhimo
E2E ML on GCP
Step1:  Restart the kernel
After you install the additional packages, you need to restart the notebook kernel so it can find the packages.
Step2:  Before you begin
Set up your Google Cloud project
The following steps are required, regardless of your notebook environment.
Select or create a Google Cloud project. When you first create an account, you get a $300 free credit towards your compute/storage costs.
Make sure that billing is enabled for your project.
Enable the Vertex AI API.
If you are running this notebook locally, you will need to install the Cloud SDK.
Enter your project ID in the cell below. Then run the cell to make sure the
Cloud SDK uses the right project for all the commands in this notebook.
Note
Step3:  Get your project number
Now that the project ID is set, you get your corresponding project number.
Step4:  Region
You can also change the REGION variable, which is used for operations
throughout the rest of this notebook.  Below are regions supported for Vertex AI. We recommend that you choose the region closest to you.
Americas
Step5:  Timestamp
If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a timestamp for each instance session, and append it onto the name of resources you create in this tutorial.
Step6:  Authenticate your Google Cloud account
If you are using Vertex AI Workbench Notebooks, your environment is already
authenticated. Skip this step.
If you are using Colab, run the cell below and follow the instructions
when prompted to authenticate your account via oAuth.
Otherwise, follow these steps
Step7:  Create a Cloud Storage bucket
The following steps are required, regardless of your notebook environment.
Before you submit a training job for the two-tower model, you need to upload your training data and schema to Cloud Storage. Vertex AI trains the model using this input data. In this tutorial, the Two-Tower built-in algorithm also saves the trained model that results from your job in the same bucket. Using this model artifact, you can then create Vertex AI model and endpoint resources in order to serve online predictions.
Set the name of your Cloud Storage bucket below. It must be unique across all
Cloud Storage buckets.
Step8:  Only if your bucket doesn't already exist
Step9:  Finally, validate access to your Cloud Storage bucket by examining its contents
Step10:  Import libraries and define constants
Step11:  Initialize Vertex AI SDK for Python
Initialize the Vertex AI SDK for Python for your project and corresponding bucket.
Step12:  Set machine type
Next, set the machine type to use for prediction.
Set the variable DEPLOY_COMPUTE to configure  the compute resources for the VMs you will use for for prediction.
machine type
n1-standard
Step13:  Introduction to Two-Tower algorithm
Two-tower models learn to represent two items of various types (such as user profiles, search queries, web documents, answer passages, or images) in the same vector space, so that similar or related items are close to each other. These two items are referred to as the query and candidate object, since when paired with a nearest neighbor search service such as Vertex Matching Engine, the two-tower model can retrieve candidate objects related to an input query object. These objects are encoded by a query and candidate encoder (the two "towers") respectively, which are trained on pairs of relevant items. This built-in algorithm exports trained query and candidate encoders as model artifacts, which can be deployed in Vertex Prediction for usage in a recommendation system.
Configure training parameters for the Two-Tower builtin algorithm
The following table shows parameters that are common to all Vertex AI Training jobs created using the gcloud ai custom-jobs create command. See the official documentation for all the possible arguments.
| Parameter | Data type | Description | Required |
|--|--|--|--|
| display-name | string | Name of the job. | Yes |
| worker-pool-spec | string | Comma-separated list of arguments specifying a worker pool configuration (see below). | Yes |
| region | string | Region to submit the job to. | No |
The worker-pool-spec flag can be specified multiple times, one for each worker pool. The following table shows the arguments used to specify a worker pool.
| Parameter | Data type | Description | Required |
|--|--|--|--|
| machine-type | string | Machine type for the pool. See the official documentation for supported machines. | Yes |
| replica-count | int | The number of replicas of the machine in the pool. | No |
| container-image-uri | string | Docker image to run on each worker. | No |
The following table shows the parameters for the two-tower model training job
Step14:  Train on Vertex AI Training with CPU
Submit the Two-Tower training job to Vertex AI Training. The following command uses a single CPU machine for training. When using single node training, training_steps_per_epoch and eval_steps_per_epoch do not need to be set.
Prepare your machine specification
Now define the machine specification for your custom hyperparameter tuning job. This tells Vertex what type of machine instance to provision for the hyperparameter tuning.
  - machine_type
Step15:  Prepare your disk specification
(optional) Now define the disk specification for your custom hyperparameter tuning job. This tells Vertex what type and size of disk to provision in each machine instance for the hyperparameter tuning.
boot_disk_type
Step16:  Define the worker pool specification
Next, you define the worker pool specification for your custom hyperparameter tuning job. The worker pool specification will consist of the following
Step17:  Create a custom job
Use the class CustomJob to create a custom job, such as for hyperparameter tuning, with the following parameters
Step18:  Execute the custom job
Next, execute your custom job using the method run().
Step19:  View output
After the job finishes successfully, you can view the output directory.
Step20:  Train on Vertex AI Training with GPU
Next, train the Two Tower model using a GPU.
Step21:  Create and execute the custom job
Next, create and execute the custom job.
Step22:  View output
After the job finishes successfully, you can view the output directory.
Step23:  Train on Vertex AI Training with TFRecords
Next, train the Two Tower model using TFRecords
Step24:  Create and execute the custom job
Next, create and execute the custom job.
Step25:  View output
After the job finishes successfully, you can view the output directory.
Step26:  Tensorboard
When the training starts, you can view the logs in TensorBoard. Colab users can use the TensorBoard widget below
Step27:  Hyperparameter tuning
You may want to optimize the hyperparameters used during training to improve your model's accuracy and performance. 
For this example, the following command runs a Vertex AI hyperparameter tuning job with 8 trials that attempts to maximize the validation AUC metric. The hyperparameters it optimizes are the number of hidden layers, the size of the hidden layers, and the learning rate.
Learn more about Hyperparameter tuning overview.
Step28:  Run the hyperparameter tuning job
Use the run() method to execute the hyperparameter tuning job.
Step29:  Display the hyperparameter tuning job trial results
After the hyperparameter tuning job has completed, the property trials will return the results for each trial.
Step30:  Best trial
Now look at which trial was the best
Step31:  Delete the hyperparameter tuning job
The method 'delete()' will delete the hyperparameter tuning job.
Step32:  View output
After the job finishes successfully, you can view the output directory.
Step33:  Upload the model to Vertex AI Model resource
Your training job will export two TF SavedModels under gs
Step34:  Deploy the model to Vertex AI Endpoint
Deploying the Vertex AI Model resoure to a Vertex AI Endpoint for online predictions
Step35:  Deploying Model resources to an Endpoint resource.
You can deploy one of more Vertex AI Model resource instances to the same endpoint. Each Vertex AI Model resource that is deployed will have its own deployment container for the serving binary. 
In the next example, you deploy the Vertex AI Model resource to a Vertex AI Endpoint resource. The Vertex AI Model resource already has defined for it the deployment container image. To deploy, you specify the following additional configuration settings
Step36:  Creating embeddings
Now that you have deployed the query/candidate encoder model on Vertex AI Prediction, you can call the model to generate embeddings for new data. 
Make an online prediction with SDK
Online prediction is used to synchronously query a model on a small batch of instances with minimal latency. The following function calls the deployed model using Vertex AI SDK for Python.
The input data you want predicted embeddings on should be provided as a stringified JSON in the data field. Note that you should also provide a unique key field (of type str) for each input instance so that you can associate each output embedding with its corresponding input.
Step37:  Make an online prediction with gcloud
You can also do online prediction using the gcloud CLI.
Step38:  Make a batch prediction
Batch prediction is used to asynchronously make predictions on a batch of input data.  This is recommended if you have a large input size and do not need an immediate response, such as getting embeddings for candidate objects in order to create an index for a nearest neighbor search service such as Vertex Matching Engine.
Create the batch input file
Next, you generate the batch input file to generate embeddings for the dataset, which you subsequently use to create an index with Vertex AI Matching Engine. In this example, the dataset contains a 1000 unique identifiers (0...999). You will use the trained encoder to generate a predicted embedding for each unique identifier.
The input data needs to be on Cloud Storage and in JSONL format. You can use the sample query object file provided below. Like with online prediction, it's recommended to have the key field so that you can associate each output embedding with its corresponding input.
Step39:  Send the prediction request
To make a batch prediction request, call the model object's batch_predict method with the following parameters
Step40:  Get the predicted embeddings
Next, get the results from the completed batch prediction job.
The results are written to the Cloud Storage output bucket you specified in the batch prediction request. You call the method iter_outputs() to get a list of each Cloud Storage file generated with the results. Each file contains one or more prediction requests in a JSON format
Step41:  Save the embeddings in JSONL format
Next, you store the predicted embeddings as a JSONL formatted file. Each embedding is stored as
Step42:  Store the JSONL formatted embeddings in Cloud Storage
Next, you upload the training data to your Cloud Storage bucket.
Step43:  Create Matching Engine Index
Next, you create the index for your embeddings. Currently, two indexing algorithms are supported
Step44:  Setup VPC peering network
To use a Matching Engine Index, you setup a VPC peering network between your project and the Vertex AI Matching Engine service project. This eliminates additional hops in network traffic and allows using efficient gRPC protocol.
Learn more about VPC peering.
IMPORTANT
Step45:  Create the VPC connection
Next, create the connection for VPC peering.
Note
Step46:  Check the status of your peering connections.
Step47:  Construct the full network name
You need to have the full network resource name when you subsequently create an Matching Engine Index Endpoint resource for VPC peering.
Step48:  Create an IndexEndpoint with VPC Network
Next, you create a Matching Engine Index Endpoint, similar to the concept of creating a Private Endpoint for prediction with a peer-to-peer network.
To create the Index Endpoint resource, you call the method create() with the following parameters
Step49:  Deploy the Matching Engine Index to the Index Endpoint resource
Next, deploy your index to the Index Endpoint using the method deploy_index() with the following parameters
Step50:  Create and execute an online query
Now that your index is deployed, you can make queries.
First, you construct a vector query using synthetic data, to use as the example to return matches for.
Next, you make the matching request using the method match(), with the following parameters
Step51:  Cleaning up
To clean up all Google Cloud resources used in this project, you can delete the Google Cloud
project you used for the tutorial.
Otherwise, you can delete the individual resources you created in this tutorial
<ASSISTANT_TASK:>
Python Code:
import os
# The Vertex AI Workbench Notebook product has specific requirements
IS_WORKBENCH_NOTEBOOK = os.getenv("DL_ANACONDA_HOME")
IS_USER_MANAGED_WORKBENCH_NOTEBOOK = os.path.exists(
    "/opt/deeplearning/metadata/env_version"
)
# Vertex AI Notebook requires dependencies to be installed with '--user'
USER_FLAG = ""
if IS_WORKBENCH_NOTEBOOK:
    USER_FLAG = "--user"
! pip3 install {USER_FLAG} --upgrade tensorflow -q
! pip3 install {USER_FLAG} --upgrade google-cloud-aiplatform tensorboard-plugin-profile -q
! gcloud components update --quiet
Explanation: Notebook is a revised version of notebook from Amy Wu and Shen Zhimo
E2E ML on GCP: MLOps stage 6 : serving: get started with Vertex AI Matching Engine and Two Towers builtin algorithm
<table align="left">
  <td>
    <a href="https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/ml_ops/stage6/get_started_with_matching_engine_twotowers.ipynb">
      <img src="https://cloud.google.com/ml-engine/images/colab-logo-32px.png" alt="Colab logo"> Run in Colab
    </a>
  </td>
  <td>
    <a href="https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/ocommunity/ml_ops/stage6/get_started_with_matching_engine_twotowers.ipynb">
      <img src="https://cloud.google.com/ml-engine/images/github-logo-32px.png" alt="GitHub logo">
      View on GitHub
    </a>
  </td>
  <td>
    <a href="https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/ml_ops/stage6/get_started_with_matching_engine_twotowers.ipynb">
      <img src="https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32" alt="Vertex AI logo">
      Run in Vertex Workbench
    </a>
  </td>
</table>
Overview
This tutorial demonstrates how to use the Vertex AI Two-Tower built-in algorithm with Vertex AI Matching Engine.
Dataset
This tutorial uses the movielens_100k sample dataset in the public bucket gs://cloud-samples-data/vertex-ai/matching-engine/two-tower, which was generated from the MovieLens movie rating dataset. For this tutorial, the data only includes the user id feature for users, and the movie id and movie title features for movies. In this example, the user is the query object and the movie is the candidate object, and each training example in the dataset contains a user and a movie they rated (we only include positive ratings in the dataset). The two-tower model will embed the user and the movie in the same embedding space, so that given a user, the model will recommend movies it thinks the user will like.
Objective
In this notebook, you will learn how to use the Two-Tower builtin algorithms for generating embeddings for a dataset, for use with generating an Matching Engine Index, with the Vertex AI Matching Engine service.
This tutorial uses the following Google Cloud ML services:
Vertex AI Two-Towers builtin algorithm
Vertex AI Matching Engine
Vertex AI Batch Prediction
The tutorial covers the following steps:
Train the Two-Tower algorithm to generate embeddings (encoder) for the dataset.
Hyperparameter tune the trained Two-Tower encoder.
Make example predictions (embeddings) from then trained encoder.
Generate embeddings using the trained Two-Tower builtin algorithm.
Store embeddings to format supported by Matching Engine.
Create a Matching Engine Index for the embeddings.
Deploy the Matching Engine Index to a Index Endpoint.
Make a matching engine prediction request.
Costs
This tutorial uses billable components of Google Cloud:
Vertex AI
Cloud Storage
Learn about Vertex AI
pricing and Cloud Storage
pricing, and use the Pricing
Calculator
to generate a cost estimate based on your projected usage.
Set up your local development environment
If you are using Colab or Google Cloud Notebooks, your environment already meets
all the requirements to run this notebook. You can skip this step.
Otherwise, make sure your environment meets this notebook's requirements.
You need the following:
The Google Cloud SDK
Git
Python 3
virtualenv
Jupyter notebook running in a virtual environment with Python 3
The Google Cloud guide to Setting up a Python development
environment and the Jupyter
installation guide provide detailed instructions
for meeting these requirements. The following steps provide a condensed set of
instructions:
Install and initialize the Cloud SDK.
Install Python 3.
Install
   virtualenv
   and create a virtual environment that uses Python 3. Activate the virtual environment.
To install Jupyter, run pip3 install jupyter on the
command-line in a terminal shell.
To launch Jupyter, run jupyter notebook on the command-line in a terminal shell.
Open this notebook in the Jupyter Notebook Dashboard.
Installation
Install the packages required for executing this notebook.
End of explanation
# Automatically restart kernel after installs
import os
if not os.getenv("IS_TESTING"):
    # Automatically restart kernel after installs
    import IPython
    app = IPython.Application.instance()
    app.kernel.do_shutdown(True)
Explanation: Restart the kernel
After you install the additional packages, you need to restart the notebook kernel so it can find the packages.
End of explanation
PROJECT_ID = "[your-project-id]"  # @param {type:"string"}
if PROJECT_ID == "" or PROJECT_ID is None or PROJECT_ID == "[your-project-id]":
    # Get your GCP project id from gcloud
    shell_output = ! gcloud config list --format 'value(core.project)' 2>/dev/null
    PROJECT_ID = shell_output[0]
    print("Project ID:", PROJECT_ID)
! gcloud config set project $PROJECT_ID
Explanation: Before you begin
Set up your Google Cloud project
The following steps are required, regardless of your notebook environment.
Select or create a Google Cloud project. When you first create an account, you get a $300 free credit towards your compute/storage costs.
Make sure that billing is enabled for your project.
Enable the Vertex AI API.
If you are running this notebook locally, you will need to install the Cloud SDK.
Enter your project ID in the cell below. Then run the cell to make sure the
Cloud SDK uses the right project for all the commands in this notebook.
Note: Jupyter runs lines prefixed with ! as shell commands, and it interpolates Python variables prefixed with $ into these commands.
Set your project ID
If you do not know your project ID, you may be able to get your project ID using gcloud.
End of explanation
shell_output = ! gcloud projects list --filter="PROJECT_ID:'{PROJECT_ID}'" --format='value(PROJECT_NUMBER)'
PROJECT_NUMBER = shell_output[0]
print("Project Number:", PROJECT_NUMBER)
Explanation: Get your project number
Now that the project ID is set, you get your corresponding project number.
End of explanation
REGION = "[your-region]"  # @param {type: "string"}
if REGION == "[your-region]":
    REGION = "us-central1"
Explanation: Region
You can also change the REGION variable, which is used for operations
throughout the rest of this notebook.  Below are regions supported for Vertex AI. We recommend that you choose the region closest to you.
Americas: us-central1
Europe: europe-west4
Asia Pacific: asia-east1
You may not use a multi-regional bucket for training with Vertex AI. Not all regions provide support for all Vertex AI services.
Learn more about Vertex AI regions.
End of explanation
from datetime import datetime
TIMESTAMP = datetime.now().strftime("%Y%m%d%H%M%S")
Explanation: Timestamp
If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a timestamp for each instance session, and append it onto the name of resources you create in this tutorial.
End of explanation
# If you are running this notebook in Colab, run this cell and follow the
# instructions to authenticate your GCP account. This provides access to your
# Cloud Storage bucket and lets you submit training jobs and prediction
# requests.
import os
import sys
# If on Vertex AI Workbench, then don't execute this code
IS_COLAB = False
if not os.path.exists("/opt/deeplearning/metadata/env_version") and not os.getenv(
    "DL_ANACONDA_HOME"
):
    if "google.colab" in sys.modules:
        IS_COLAB = True
        from google.colab import auth as google_auth
        google_auth.authenticate_user()
    # If you are running this notebook locally, replace the string below with the
    # path to your service account key and run this cell to authenticate your GCP
    # account.
    elif not os.getenv("IS_TESTING"):
        %env GOOGLE_APPLICATION_CREDENTIALS ''
Explanation: Authenticate your Google Cloud account
If you are using Vertex AI Workbench Notebooks, your environment is already
authenticated. Skip this step.
If you are using Colab, run the cell below and follow the instructions
when prompted to authenticate your account via oAuth.
Otherwise, follow these steps:
In the Cloud Console, go to the Create service account key
   page.
Click Create service account.
In the Service account name field, enter a name, and
   click Create.
In the Grant this service account access to project section, click the Role drop-down list. Type "Vertex AI"
into the filter box, and select
   Vertex AI Administrator. Type "Storage Object Admin" into the filter box, and select Storage Object Admin.
Click Create. A JSON file that contains your key downloads to your
local environment.
Enter the path to your service account key as the
GOOGLE_APPLICATION_CREDENTIALS variable in the cell below and run the cell.
End of explanation
BUCKET_NAME = "[your-bucket-name]"  # @param {type:"string"}
BUCKET_URI = f"gs://{BUCKET_NAME}"
if BUCKET_URI == "" or BUCKET_URI is None or BUCKET_URI == "gs://[your-bucket-name]":
    BUCKET_NAME = PROJECT_ID + "aip-" + TIMESTAMP
    BUCKET_URI = "gs://" + BUCKET_NAME
Explanation: Create a Cloud Storage bucket
The following steps are required, regardless of your notebook environment.
Before you submit a training job for the two-tower model, you need to upload your training data and schema to Cloud Storage. Vertex AI trains the model using this input data. In this tutorial, the Two-Tower built-in algorithm also saves the trained model that results from your job in the same bucket. Using this model artifact, you can then create Vertex AI model and endpoint resources in order to serve online predictions.
Set the name of your Cloud Storage bucket below. It must be unique across all
Cloud Storage buckets.
End of explanation
! gsutil mb -l $REGION $BUCKET_URI
Explanation: Only if your bucket doesn't already exist: Run the following cell to create your Cloud Storage bucket.
End of explanation
! gsutil ls -al $BUCKET_URI
Explanation: Finally, validate access to your Cloud Storage bucket by examining its contents:
End of explanation
import os
from google.cloud import aiplatform
%load_ext tensorboard
Explanation: Import libraries and define constants
End of explanation
aiplatform.init(project=PROJECT_ID, location=REGION, staging_bucket=BUCKET_URI)
Explanation: Initialize Vertex AI SDK for Python
Initialize the Vertex AI SDK for Python for your project and corresponding bucket.
End of explanation
if os.getenv("IS_TESTING_DEPLOY_MACHINE"):
    MACHINE_TYPE = os.getenv("IS_TESTING_DEPLOY_MACHINE")
else:
    MACHINE_TYPE = "n1-standard"
VCPU = "4"
DEPLOY_COMPUTE = MACHINE_TYPE + "-" + VCPU
print("Deploy machine type", DEPLOY_COMPUTE)
Explanation: Set machine type
Next, set the machine type to use for prediction.
Set the variable DEPLOY_COMPUTE to configure  the compute resources for the VMs you will use for for prediction.
machine type
n1-standard: 3.75GB of memory per vCPU.
n1-highmem: 6.5GB of memory per vCPU
n1-highcpu: 0.9 GB of memory per vCPU
vCPUs: number of [2, 4, 8, 16, 32, 64, 96 ]
Note: You may also use n2 and e2 machine types for training and deployment, but they do not support GPUs.
End of explanation
DATASET_NAME = "movielens_100k"  # Change to your dataset name.
# Change to your data and schema paths. These are paths to the movielens_100k
# sample data.
TRAINING_DATA_PATH = f"gs://cloud-samples-data/vertex-ai/matching-engine/two-tower/{DATASET_NAME}/training_data/*"
INPUT_SCHEMA_PATH = f"gs://cloud-samples-data/vertex-ai/matching-engine/two-tower/{DATASET_NAME}/input_schema.json"
# URI of the two-tower training Docker image.
LEARNER_IMAGE_URI = "us-docker.pkg.dev/vertex-ai-restricted/builtin-algorithm/two-tower"
# Change to your output location.
OUTPUT_DIR = f"{BUCKET_URI}/experiment/output"
TRAIN_BATCH_SIZE = 100  # Batch size for training.
NUM_EPOCHS = 3  # Number of epochs for training.
print(f"Dataset name: {DATASET_NAME}")
print(f"Training data path: {TRAINING_DATA_PATH}")
print(f"Input schema path: {INPUT_SCHEMA_PATH}")
print(f"Output directory: {OUTPUT_DIR}")
print(f"Train batch size: {TRAIN_BATCH_SIZE}")
print(f"Number of epochs: {NUM_EPOCHS}")
Explanation: Introduction to Two-Tower algorithm
Two-tower models learn to represent two items of various types (such as user profiles, search queries, web documents, answer passages, or images) in the same vector space, so that similar or related items are close to each other. These two items are referred to as the query and candidate object, since when paired with a nearest neighbor search service such as Vertex Matching Engine, the two-tower model can retrieve candidate objects related to an input query object. These objects are encoded by a query and candidate encoder (the two "towers") respectively, which are trained on pairs of relevant items. This built-in algorithm exports trained query and candidate encoders as model artifacts, which can be deployed in Vertex Prediction for usage in a recommendation system.
Configure training parameters for the Two-Tower builtin algorithm
The following table shows parameters that are common to all Vertex AI Training jobs created using the gcloud ai custom-jobs create command. See the official documentation for all the possible arguments.
| Parameter | Data type | Description | Required |
|--|--|--|--|
| display-name | string | Name of the job. | Yes |
| worker-pool-spec | string | Comma-separated list of arguments specifying a worker pool configuration (see below). | Yes |
| region | string | Region to submit the job to. | No |
The worker-pool-spec flag can be specified multiple times, one for each worker pool. The following table shows the arguments used to specify a worker pool.
| Parameter | Data type | Description | Required |
|--|--|--|--|
| machine-type | string | Machine type for the pool. See the official documentation for supported machines. | Yes |
| replica-count | int | The number of replicas of the machine in the pool. | No |
| container-image-uri | string | Docker image to run on each worker. | No |
The following table shows the parameters for the two-tower model training job:
| Parameter | Data type | Description | Required |
|--|--|--|--|
| training_data_path | string | Cloud Storage pattern where training data is stored. | Yes |
| input_schema_path | string | Cloud Storage path where the JSON input schema is stored. | Yes |
| input_file_format | string | The file format of input. Currently supports jsonl and tfrecord. | No - default is jsonl. |
| job_dir | string | Cloud Storage directory where the model output files will be stored. | Yes |
| eval_data_path | string | Cloud Storage pattern where eval data is stored. | No |
| candidate_data_path | string | Cloud Storage pattern where candidate data is stored. Only used for top_k_categorical_accuracy metrics. If not set, it's generated from training/eval data. | No |
| train_batch_size | int | Batch size for training. | No - Default is 100. |
| eval_batch_size | int | Batch size for evaluation. | No - Default is 100. |
| eval_split | float | Split fraction to use for the evaluation dataset, if eval_data_path is not provided. | No - Default is 0.2 |
| optimizer | string | Training optimizer. Lowercase string name of any TF2.3 Keras optimizer is supported ('sgd', 'nadam', 'ftrl', etc.). See TensorFlow documentation. | No - Default is 'adagrad'. |
| learning_rate | float | Learning rate for training. | No - Default is the default learning rate of the specified optimizer. |
| momentum | float | Momentum for optimizer, if specified. | No - Default is the default momentum value for the specified optimizer. |
| metrics | string | Metrics used to evaluate the model. Can be either auc, top_k_categorical_accuracy or precision_at_1. | No - Default is auc. |
| num_epochs | int | Number of epochs for training. | No - Default is 10. |
| num_hidden_layers | int | Number of hidden layers. | No |
| num_nodes_hidden_layer{index} | int | Num of nodes in hidden layer {index}. The range of index is 1 to 20. | No |
| output_dim | int | The output embedding dimension for each encoder tower of the two-tower model. | No - Default is 64. |
| training_steps_per_epoch | int | Number of steps per epoch to run the training for.  Only needed if you are using more than 1 machine or using a master machine with more than 1 gpu. | No - Default is None. |
| eval_steps_per_epoch | int | Number of steps per epoch to run the evaluation for.  Only needed if you are using more than 1 machine or using a master machine with more than 1 gpu. | No - Default is None. |
| gpu_memory_alloc | int | Amount of memory allocated per GPU (in MB). | No - Default is no limit. |
End of explanation
TRAIN_COMPUTE = "n1-standard-8"
machine_spec = {"machine_type": TRAIN_COMPUTE, "accelerator_count": 0}
Explanation: Train on Vertex AI Training with CPU
Submit the Two-Tower training job to Vertex AI Training. The following command uses a single CPU machine for training. When using single node training, training_steps_per_epoch and eval_steps_per_epoch do not need to be set.
Prepare your machine specification
Now define the machine specification for your custom hyperparameter tuning job. This tells Vertex what type of machine instance to provision for the hyperparameter tuning.
  - machine_type: The type of GCP instance to provision -- e.g., n1-standard-8.
  - accelerator_type: The type, if any, of hardware accelerator. In this tutorial if you previously set the variable TRAIN_GPU != None, you are using a GPU; otherwise you will use a CPU.
  - accelerator_count: The number of accelerators.
End of explanation
DISK_TYPE = "pd-ssd"  # [ pd-ssd, pd-standard]
DISK_SIZE = 200  # GB
disk_spec = {"boot_disk_type": DISK_TYPE, "boot_disk_size_gb": DISK_SIZE}
Explanation: Prepare your disk specification
(optional) Now define the disk specification for your custom hyperparameter tuning job. This tells Vertex what type and size of disk to provision in each machine instance for the hyperparameter tuning.
boot_disk_type: Either SSD or Standard. SSD is faster, and Standard is less expensive. Defaults to SSD.
boot_disk_size_gb: Size of disk in GB.
End of explanation
JOB_NAME = "twotowers_cpu_" + TIMESTAMP
MODEL_DIR = "{}/{}".format(BUCKET_URI, JOB_NAME)
CMDARGS = [
    f"--training_data_path={TRAINING_DATA_PATH}",
    f"--input_schema_path={INPUT_SCHEMA_PATH}",
    f"--job-dir={OUTPUT_DIR}",
    f"--train_batch_size={TRAIN_BATCH_SIZE}",
    f"--num_epochs={NUM_EPOCHS}",
]
worker_pool_spec = [
    {
        "replica_count": 1,
        "machine_spec": machine_spec,
        "disk_spec": disk_spec,
        "container_spec": {
            "image_uri": LEARNER_IMAGE_URI,
            "command": [],
            "args": CMDARGS,
        },
    }
]
Explanation: Define the worker pool specification
Next, you define the worker pool specification for your custom hyperparameter tuning job. The worker pool specification will consist of the following:
replica_count: The number of instances to provision of this machine type.
machine_spec: The hardware specification.
disk_spec : (optional) The disk storage specification.
container_spec: The training container containing the training package.
Let's dive deeper now into the container specification:
image_uri: The training image.
command: The command to invoke in the training image. Defaults to the command entry point specified for the training image.
args: The command line arguments to pass to the corresponding command entry point in training image.
End of explanation
job = aiplatform.CustomJob(
    display_name="twotower_cpu_" + TIMESTAMP, worker_pool_specs=worker_pool_spec
)
Explanation: Create a custom job
Use the class CustomJob to create a custom job, such as for hyperparameter tuning, with the following parameters:
display_name: A human readable name for the custom job.
worker_pool_specs: The specification for the corresponding VM instances.
End of explanation
job.run()
Explanation: Execute the custom job
Next, execute your custom job using the method run().
End of explanation
! gsutil ls {OUTPUT_DIR}
! gsutil rm -rf {OUTPUT_DIR}/*
Explanation: View output
After the job finishes successfully, you can view the output directory.
End of explanation
JOB_NAME = "twotowers_gpu_" + TIMESTAMP
MODEL_DIR = "{}/{}".format(BUCKET_URI, JOB_NAME)
TRAIN_COMPUTE = "n1-highmem-4"
TRAIN_GPU = "NVIDIA_TESLA_K80"
machine_spec = {
    "machine_type": TRAIN_COMPUTE,
    "accelerator_type": TRAIN_GPU,
    "accelerator_count": 1,
}
CMDARGS = [
    f"--training_data_path={TRAINING_DATA_PATH}",
    f"--input_schema_path={INPUT_SCHEMA_PATH}",
    f"--job-dir={OUTPUT_DIR}",
    "--training_steps_per_epoch=1500",
    "--eval_steps_per_epoch=1500",
]
worker_pool_spec = [
    {
        "replica_count": 1,
        "machine_spec": machine_spec,
        "disk_spec": disk_spec,
        "container_spec": {
            "image_uri": LEARNER_IMAGE_URI,
            "command": [],
            "args": CMDARGS,
        },
    }
]
Explanation: Train on Vertex AI Training with GPU
Next, train the Two Tower model using a GPU.
End of explanation
job = aiplatform.CustomJob(
    display_name="twotower_cpu_" + TIMESTAMP, worker_pool_specs=worker_pool_spec
)
job.run()
Explanation: Create and execute the custom job
Next, create and execute the custom job.
End of explanation
! gsutil ls {OUTPUT_DIR}
! gsutil rm -rf {OUTPUT_DIR}/*
Explanation: View output
After the job finishes successfully, you can view the output directory.
End of explanation
TRAINING_DATA_PATH = f"gs://cloud-samples-data/vertex-ai/matching-engine/two-tower/{DATASET_NAME}/tfrecord/*"
JOB_NAME = "twotowers_tfrec_" + TIMESTAMP
MODEL_DIR = "{}/{}".format(BUCKET_URI, JOB_NAME)
TRAIN_COMPUTE = "n1-standard-8"
machine_spec = {"machine_type": TRAIN_COMPUTE, "accelerator_count": 0}
CMDARGS = [
    f"--training_data_path={TRAINING_DATA_PATH}",
    f"--input_schema_path={INPUT_SCHEMA_PATH}",
    f"--job-dir={OUTPUT_DIR}",
    f"--train_batch_size={TRAIN_BATCH_SIZE}",
    f"--num_epochs={NUM_EPOCHS}",
    "--input_file_format=tfrecord",
]
worker_pool_spec = [
    {
        "replica_count": 1,
        "machine_spec": machine_spec,
        "disk_spec": disk_spec,
        "container_spec": {
            "image_uri": LEARNER_IMAGE_URI,
            "command": [],
            "args": CMDARGS,
        },
    }
]
Explanation: Train on Vertex AI Training with TFRecords
Next, train the Two Tower model using TFRecords
End of explanation
job = aiplatform.CustomJob(
    display_name="twotower_cpu_" + TIMESTAMP, worker_pool_specs=worker_pool_spec
)
job.run()
Explanation: Create and execute the custom job
Next, create and execute the custom job.
End of explanation
! gsutil ls {OUTPUT_DIR}
! gsutil rm -rf {OUTPUT_DIR}
Explanation: View output
After the job finishes successfully, you can view the output directory.
End of explanation
try:
    TENSORBOARD_DIR = os.path.join(OUTPUT_DIR, "tensorboard")
    %tensorboard --logdir {TENSORBOARD_DIR}
except Exception as e:
    print(e)
Explanation: Tensorboard
When the training starts, you can view the logs in TensorBoard. Colab users can use the TensorBoard widget below:
For Workbench AI Notebooks users, the TensorBoard widget above won't work. We recommend you to launch TensorBoard through the Cloud Shell.
In your Cloud Shell, launch Tensorboard on port 8080:
export TENSORBOARD_DIR=gs://xxxxx/tensorboard
tensorboard --logdir=${TENSORBOARD_DIR} --port=8080
Click the "Web Preview" button at the top-right of the Cloud Shell window (looks like an eye in a rectangle). 
Select "Preview on port 8080". This should launch the TensorBoard webpage in a new tab in your browser.
After the job finishes successfully, you can view the output directory:
End of explanation
from google.cloud.aiplatform import hyperparameter_tuning as hpt
hpt_job = aiplatform.HyperparameterTuningJob(
    display_name="twotowers_" + TIMESTAMP,
    custom_job=job,
    metric_spec={
        "val_auc": "maximize",
    },
    parameter_spec={
        "learning_rate": hpt.DoubleParameterSpec(min=0.0001, max=0.1, scale="log"),
        "num_hidden_layers": hpt.IntegerParameterSpec(min=0, max=2, scale="linear"),
        "num_nodes_hidden_layer1": hpt.IntegerParameterSpec(
            min=1, max=128, scale="log"
        ),
        "num_nodes_hidden_layer2": hpt.IntegerParameterSpec(
            min=1, max=128, scale="log"
        ),
    },
    search_algorithm=None,
    max_trial_count=8,
    parallel_trial_count=1,
)
Explanation: Hyperparameter tuning
You may want to optimize the hyperparameters used during training to improve your model's accuracy and performance. 
For this example, the following command runs a Vertex AI hyperparameter tuning job with 8 trials that attempts to maximize the validation AUC metric. The hyperparameters it optimizes are the number of hidden layers, the size of the hidden layers, and the learning rate.
Learn more about Hyperparameter tuning overview.
End of explanation
hpt_job.run()
Explanation: Run the hyperparameter tuning job
Use the run() method to execute the hyperparameter tuning job.
End of explanation
print(hpt_job.trials)
Explanation: Display the hyperparameter tuning job trial results
After the hyperparameter tuning job has completed, the property trials will return the results for each trial.
End of explanation
best = (None, None, None, 0.0)
for trial in hpt_job.trials:
    # Keep track of the best outcome
    if float(trial.final_measurement.metrics[0].value) > best[3]:
        try:
            best = (
                trial.id,
                float(trial.parameters[0].value),
                float(trial.parameters[1].value),
                float(trial.final_measurement.metrics[0].value),
            )
        except:
            best = (
                trial.id,
                float(trial.parameters[0].value),
                None,
                float(trial.final_measurement.metrics[0].value),
            )
print(best)
Explanation: Best trial
Now look at which trial was the best:
End of explanation
hpt_job.delete()
Explanation: Delete the hyperparameter tuning job
The method 'delete()' will delete the hyperparameter tuning job.
End of explanation
BEST_MODEL = OUTPUT_DIR + "/trial_" + best[0]
! gsutil ls {BEST_MODEL}
Explanation: View output
After the job finishes successfully, you can view the output directory.
End of explanation
# The following imports the query (user) encoder model.
MODEL_TYPE = "query"
# Use the following instead to import the candidate (movie) encoder model.
# MODEL_TYPE = 'candidate'
DISPLAY_NAME = f"{DATASET_NAME}_{MODEL_TYPE}"  # The display name of the model.
MODEL_NAME = f"{MODEL_TYPE}_model"  # Used by the deployment container.
model = aiplatform.Model.upload(
    display_name=DISPLAY_NAME,
    artifact_uri=BEST_MODEL,
    serving_container_image_uri="us-central1-docker.pkg.dev/cloud-ml-algos/two-tower/deploy",
    serving_container_health_route=f"/v1/models/{MODEL_NAME}",
    serving_container_predict_route=f"/v1/models/{MODEL_NAME}:predict",
    serving_container_environment_variables={
        "MODEL_BASE_PATH": "$(AIP_STORAGE_URI)",
        "MODEL_NAME": MODEL_NAME,
    },
)
Explanation: Upload the model to Vertex AI Model resource
Your training job will export two TF SavedModels under gs://&lt;job_dir&gt;/query_model and gs://&lt;job_dir&gt;/candidate_model. These exported models can be used for online or batch prediction in Vertex Prediction. 
First, import the query (or candidate) model using the upload() method, with the following parameters:
display_name: A human readable name for the model resource.
artifact_uri: The Cloud Storage location of the model artifacts.
serving_container_image_uri: The deployment container. In this tutorial, you use the prebuilt Two-Tower deployment container.
serving_container_health_route: The URL for the service to periodically ping for a response to verify that the serving binary is running. For Two-Towers, this will be /v1/models/[model_name].
serving_container_predict_route: The URL for the service to periodically ping for a response to verify that the serving binary is running. For Two-Towers, this will be /v1/models/[model_name]:predict.
serving_container_environment_variables: Preset environment variables to pass into the deployment container.
Note: The underlying deployment container is built on TensorFlow Serving.
End of explanation
endpoint = aiplatform.Endpoint.create(display_name=DATASET_NAME)
Explanation: Deploy the model to Vertex AI Endpoint
Deploying the Vertex AI Model resoure to a Vertex AI Endpoint for online predictions:
Create an Endpoint resource exposing an external interface to users consuming the model. 
After the Endpoint is ready, deploy one or more instances of a model to the Endpoint. The deployed model runs the custom container image running Two-Tower encoder to serve embeddings.
Refer to Vertex AI Predictions guide to Deploy a model using the Vertex AI API for more information about the APIs used in the following cells.
Create a Vertex AI Endpoint
Next, you create the Vertex AI Endpoint, from which you subsequently deploy your Vertex AI Model resource to.
End of explanation
response = endpoint.deploy(
    model=model,
    deployed_model_display_name=DISPLAY_NAME,
    machine_type=DEPLOY_COMPUTE,
    traffic_split={"0": 100},
)
print(endpoint)
Explanation: Deploying Model resources to an Endpoint resource.
You can deploy one of more Vertex AI Model resource instances to the same endpoint. Each Vertex AI Model resource that is deployed will have its own deployment container for the serving binary. 
In the next example, you deploy the Vertex AI Model resource to a Vertex AI Endpoint resource. The Vertex AI Model resource already has defined for it the deployment container image. To deploy, you specify the following additional configuration settings:
The machine type.
The (if any) type and number of GPUs.
Static, manual or auto-scaling of VM instances.
In this example, you deploy the model with the minimal amount of specified parameters, as follows:
model: The Model resource.
deployed_model_displayed_name: The human readable name for the deployed model instance.
machine_type: The machine type for each VM instance.
Do to the requirements to provision the resource, this may take upto a few minutes.
End of explanation
# Input items for the query model:
input_items = [
    {"data": '{"user_id": ["1"]}', "key": "key1"},
    {"data": '{"user_id": ["2"]}', "key": "key2"},
]
# Input items for the candidate model:
# input_items = [{
#     'data' : '{"movie_id": ["1"], "movie_title": ["fake title"]}',
#     'key': 'key1'
# }]
encodings = endpoint.predict(input_items)
print(f"Number of encodings: {len(encodings.predictions)}")
print(encodings.predictions[0]["encoding"])
Explanation: Creating embeddings
Now that you have deployed the query/candidate encoder model on Vertex AI Prediction, you can call the model to generate embeddings for new data. 
Make an online prediction with SDK
Online prediction is used to synchronously query a model on a small batch of instances with minimal latency. The following function calls the deployed model using Vertex AI SDK for Python.
The input data you want predicted embeddings on should be provided as a stringified JSON in the data field. Note that you should also provide a unique key field (of type str) for each input instance so that you can associate each output embedding with its corresponding input.
End of explanation
import json
request = json.dumps({"instances": input_items})
with open("request.json", "w") as writer:
    writer.write(f"{request}\n")
ENDPOINT_ID = endpoint.resource_name
! gcloud ai endpoints predict {ENDPOINT_ID} \
  --region={REGION} \
  --json-request=request.json
Explanation: Make an online prediction with gcloud
You can also do online prediction using the gcloud CLI.
End of explanation
QUERY_EMBEDDING_PATH = f"{BUCKET_URI}/embeddings/train.jsonl"
import tensorflow as tf
with tf.io.gfile.GFile(QUERY_EMBEDDING_PATH, "w") as f:
    for i in range(0, 1000):
        query = {"data": '{"user_id": ["' + str(i) + '"]}', "key": f"key{i}"}
        f.write(json.dumps(query) + "\n")
print("\nNumber of embeddings: ")
! gsutil cat {QUERY_EMBEDDING_PATH} | wc -l
Explanation: Make a batch prediction
Batch prediction is used to asynchronously make predictions on a batch of input data.  This is recommended if you have a large input size and do not need an immediate response, such as getting embeddings for candidate objects in order to create an index for a nearest neighbor search service such as Vertex Matching Engine.
Create the batch input file
Next, you generate the batch input file to generate embeddings for the dataset, which you subsequently use to create an index with Vertex AI Matching Engine. In this example, the dataset contains a 1000 unique identifiers (0...999). You will use the trained encoder to generate a predicted embedding for each unique identifier.
The input data needs to be on Cloud Storage and in JSONL format. You can use the sample query object file provided below. Like with online prediction, it's recommended to have the key field so that you can associate each output embedding with its corresponding input.
End of explanation
MIN_NODES = 1
MAX_NODES = 4
batch_predict_job = model.batch_predict(
    job_display_name=f"batch_predict_{DISPLAY_NAME}",
    gcs_source=[QUERY_EMBEDDING_PATH],
    gcs_destination_prefix=f"{BUCKET_URI}/embeddings/output",
    machine_type=DEPLOY_COMPUTE,
    starting_replica_count=MIN_NODES,
    max_replica_count=MAX_NODES,
)
Explanation: Send the prediction request
To make a batch prediction request, call the model object's batch_predict method with the following parameters: 
- instances_format: The format of the batch prediction request file: "jsonl", "csv", "bigquery", "tf-record", "tf-record-gzip" or "file-list"
- prediction_format: The format of the batch prediction response file: "jsonl", "csv", "bigquery", "tf-record", "tf-record-gzip" or "file-list"
- job_display_name: The human readable name for the prediction job.
- gcs_source: A list of one or more Cloud Storage paths to your batch prediction requests.
- gcs_destination_prefix: The Cloud Storage path that the service will write the predictions to.
- model_parameters: Additional filtering parameters for serving prediction results.
- machine_type: The type of machine to use for training.
- accelerator_type: The hardware accelerator type.
- accelerator_count: The number of accelerators to attach to a worker replica.
- starting_replica_count: The number of compute instances to initially provision.
- max_replica_count: The maximum number of compute instances to scale to. In this tutorial, only one instance is provisioned.
Compute instance scaling
You can specify a single instance (or node) to process your batch prediction request. This tutorial uses a single node, so the variables MIN_NODES and MAX_NODES are both set to 1.
If you want to use multiple nodes to process your batch prediction request, set MAX_NODES to the maximum number of nodes you want to use. Vertex AI autoscales the number of nodes used to serve your predictions, up to the maximum number you set. Refer to the pricing page to understand the costs of autoscaling with multiple nodes.
End of explanation
bp_iter_outputs = batch_predict_job.iter_outputs()
prediction_results = list()
for blob in bp_iter_outputs:
    if blob.name.split("/")[-1].startswith("prediction"):
        prediction_results.append(blob.name)
result_files = []
for prediction_result in prediction_results:
    result_file = f"gs://{bp_iter_outputs.bucket.name}/{prediction_result}"
    result_files.append(result_file)
print(result_files)
Explanation: Get the predicted embeddings
Next, get the results from the completed batch prediction job.
The results are written to the Cloud Storage output bucket you specified in the batch prediction request. You call the method iter_outputs() to get a list of each Cloud Storage file generated with the results. Each file contains one or more prediction requests in a JSON format:
instance: The prediction request.
prediction: The prediction response.
End of explanation
embeddings = []
for result_file in result_files:
    with tf.io.gfile.GFile(result_file, "r") as f:
        instances = list(f)
    for instance in instances:
        instance = instance.replace('\\"', "'")
        result = json.loads(instance)
        prediction = result["prediction"]
        key = prediction["key"][3:]
        encoding = prediction["encoding"]
        embedding = {"id": key, "embedding": encoding}
        embeddings.append(embedding)
print("Number of embeddings", len(embeddings))
print("Encoding Dimensions", len(embeddings[0]["embedding"]))
print("Example embedding", embeddings[0])
with open("embeddings.json", "w") as f:
    for i in range(len(embeddings)):
        f.write(json.dumps(embeddings[i]).replace('"', "'"))
        f.write("\n")
! head -n 2 embeddings.json
Explanation: Save the embeddings in JSONL format
Next, you store the predicted embeddings as a JSONL formatted file. Each embedding is stored as:
{ 'id': .., 'embedding': [ ... ] }
The format of the embeddings for the index can be in either CSV, JSON, or Avro format.
Learn more about Embedding Formats for Indexing
End of explanation
EMBEDDINGS_URI = f"{BUCKET_URI}/embeddings/twotower/"
! gsutil cp embeddings.json {EMBEDDINGS_URI}
Explanation: Store the JSONL formatted embeddings in Cloud Storage
Next, you upload the training data to your Cloud Storage bucket.
End of explanation
DIMENSIONS = len(embeddings[0]["embedding"])
DISPLAY_NAME = "movies"
tree_ah_index = aiplatform.MatchingEngineIndex.create_tree_ah_index(
    display_name=DISPLAY_NAME,
    contents_delta_uri=EMBEDDINGS_URI,
    dimensions=DIMENSIONS,
    approximate_neighbors_count=50,
    distance_measure_type="DOT_PRODUCT_DISTANCE",
    description="Two tower generated embeddings",
    labels={"label_name": "label_value"},
    # TreeAH specific parameters
    leaf_node_embedding_count=100,
    leaf_nodes_to_search_percent=7,
)
INDEX_RESOURCE_NAME = tree_ah_index.resource_name
print(INDEX_RESOURCE_NAME)
Explanation: Create Matching Engine Index
Next, you create the index for your embeddings. Currently, two indexing algorithms are supported:
create_tree_ah_index():  Shallow tree + Asymmetric hashing.
create_brute_force_index(): Linear search.
In this tutorial, you use the create_tree_ah_index()for production scale. The method is called with the following parameters:
display_name: A human readable name for the index.
contents_delta_uri: A Cloud Storage location for the embeddings, which are either to be inserted, updated or deleted.
dimensions: The number of dimensions of the input vector
approximate_neighbors_count: (for Tree AH) The default number of neighbors to find via approximate search before exact reordering is performed. Exact reordering is a procedure where results returned by an approximate search algorithm are reordered via a more expensive distance computation.
distance_measure_type: The distance measure used in nearest neighbor search.
SQUARED_L2_DISTANCE: Euclidean (L2) Distance
L1_DISTANCE: Manhattan (L1) Distance
COSINE_DISTANCE: Cosine Distance. Defined as 1 - cosine similarity.
DOT_PRODUCT_DISTANCE: Default value. Defined as a negative of the dot product.
description: A human readble description of the index.
labels: User metadata in the form of a dictionary.
leaf_node_embedding_count: Number of embeddings on each leaf node. The default value is 1000 if not set.
leaf_nodes_to_search_percent: The default percentage of leaf nodes that any query may be searched. Must be in range 1-100, inclusive. The default value is 10 (means 10%) if not set.
This may take upto 30 minutes.
Learn more about Configuring Matching Engine Indexes.
End of explanation
# This is for display only; you can name the range anything.
PEERING_RANGE_NAME = "vertex-ai-prediction-peering-range"
NETWORK = "default"
# NOTE: `prefix-length=16` means a CIDR block with mask /16 will be
# reserved for use by Google services, such as Vertex AI.
! gcloud compute addresses create $PEERING_RANGE_NAME \
  --global \
  --prefix-length=16 \
  --description="peering range for Google service" \
  --network=$NETWORK \
  --purpose=VPC_PEERING
Explanation: Setup VPC peering network
To use a Matching Engine Index, you setup a VPC peering network between your project and the Vertex AI Matching Engine service project. This eliminates additional hops in network traffic and allows using efficient gRPC protocol.
Learn more about VPC peering.
IMPORTANT: you can only setup one VPC peering to servicenetworking.googleapis.com per project.
Create VPC peering for default network
For simplicity, we setup VPC peering to the default network. You can create a different network for your project.
If you setup VPC peering with any other network, make sure that the network already exists and that your VM is running on that network.
End of explanation
! gcloud services vpc-peerings connect \
  --service=servicenetworking.googleapis.com \
  --network=$NETWORK \
  --ranges=$PEERING_RANGE_NAME \
  --project=$PROJECT_ID
Explanation: Create the VPC connection
Next, create the connection for VPC peering.
Note: If you get a PERMISSION DENIED, you may not have the neccessary role 'Compute Network Admin' set for your default service account. In the Cloud Console, do the following steps.
Goto IAM &amp; Admin
Find your service account.
Click edit icon.
Select Add Another Role.
Enter 'Compute Network Admin'.
Select Save
End of explanation
! gcloud compute networks peerings list --network $NETWORK
Explanation: Check the status of your peering connections.
End of explanation
full_network_name = f"projects/{PROJECT_NUMBER}/global/networks/{NETWORK}"
Explanation: Construct the full network name
You need to have the full network resource name when you subsequently create an Matching Engine Index Endpoint resource for VPC peering.
End of explanation
index_endpoint = aiplatform.MatchingEngineIndexEndpoint.create(
    display_name="index_endpoint_for_demo",
    description="index endpoint description",
    network=full_network_name,
)
INDEX_ENDPOINT_NAME = index_endpoint.resource_name
print(INDEX_ENDPOINT_NAME)
Explanation: Create an IndexEndpoint with VPC Network
Next, you create a Matching Engine Index Endpoint, similar to the concept of creating a Private Endpoint for prediction with a peer-to-peer network.
To create the Index Endpoint resource, you call the method create() with the following parameters:
display_name: A human readable name for the Index Endpoint.
description: A description for the Index Endpoint.
network: The VPC network resource name.
End of explanation
DEPLOYED_INDEX_ID = "tree_ah_twotower_deployed_" + TIMESTAMP
MIN_NODES = 1
MAX_NODES = 2
DEPLOY_COMPUTE = "n1-standard-16"
index_endpoint.deploy_index(
    display_name="deployed_index_for_demo",
    index=tree_ah_index,
    deployed_index_id=DEPLOYED_INDEX_ID,
    machine_type=DEPLOY_COMPUTE,
    min_replica_count=MIN_NODES,
    max_replica_count=MAX_NODES,
)
print(index_endpoint.deployed_indexes)
Explanation: Deploy the Matching Engine Index to the Index Endpoint resource
Next, deploy your index to the Index Endpoint using the method deploy_index() with the following parameters:
display_name: A human readable name for the deployed index.
index: Your index.
deployed_index_id: A user assigned identifier for the deployed index.
machine_type: (optional) The VM instance type.
min_replica_count: (optional) Minimum number of VM instances for auto-scaling.
max_replica_count: (optional) Maximum number of VM instances for auto-scaling.
Learn more about Machine resources for Index Endpoint
End of explanation
# The number of nearest neighbors to be retrieved from database for each query.
NUM_NEIGHBOURS = 10
# Test query
queries = [embeddings[0]["embedding"], embeddings[1]["embedding"]]
matches = index_endpoint.match(
    deployed_index_id=DEPLOYED_INDEX_ID, queries=queries, num_neighbors=NUM_NEIGHBOURS
)
for instance in matches:
    print("INSTANCE")
    for match in instance:
        print(match)
Explanation: Create and execute an online query
Now that your index is deployed, you can make queries.
First, you construct a vector query using synthetic data, to use as the example to return matches for.
Next, you make the matching request using the method match(), with the following parameters:
deployed_index_id:  The identifier of the deployed index.
queries: A list of queries (instances).
num_neighbors: The number of closest matches to return.
End of explanation
# Delete endpoint resource
endpoint.delete(force=True)
# Delete model resource
model.delete()
# Force undeployment of indexes and delete endpoint
try:
    index_endpoint.delete(force=True)
except Exception as e:
    print(e)
# Delete indexes
try:
    tree_ah_index.delete()
    brute_force_index.delete()
except Exception as e:
    print(e)
# Delete Cloud Storage objects that were created
delete_bucket = False
if delete_bucket or os.getenv("IS_TESTING"):
    ! gsutil -m rm -r $OUTPUT_DIR
Explanation: Cleaning up
To clean up all Google Cloud resources used in this project, you can delete the Google Cloud
project you used for the tutorial.
Otherwise, you can delete the individual resources you created in this tutorial:
End of explanation
<END_TASK>