<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 CI/CD for TFX pipelines
Learning Objectives
Develop a CI/CD workflow with Cloud Build to build and deploy TFX pipeline code.
Integrate with Github to automatically trigger pipeline deployment with source code repository changes.
In this lab, you will walk through authoring a Cloud Build CI/CD workflow that automatically builds and deploys the same TFX pipeline from lab-02.ipynb. You will also integrate your workflow with GitHub by setting up a trigger that starts the workflow when a new tag is applied to the GitHub repo hosting the pipeline's code.
Setup
Step1:  Note
Step2:  Note
Step3:  CUSTOM_SERVICE_ACCOUNT - In the gcp console Click on the Navigation Menu and navigate to IAM &amp; Admin, then to Service Accounts and use the service account starting with prefix - 'tfx-tuner-caip-service-account'. This enables CloudTuner and the Google Cloud AI Platform extensions Tuner component to work together and allows for distributed and parallel tuning backed by AI Platform Vizier's hyperparameter search algorithm. Please see the lab setup README for setup instructions.
ENDPOINT - set the ENDPOINT constant to the endpoint to your AI Platform Pipelines instance. The endpoint to the AI Platform Pipelines instance can be found on the AI Platform Pipelines page in the Google Cloud Console. Open the SETTINGS for your instance and use the value of the host variable in the Connect to this Kubeflow Pipelines instance from a Python client via Kubeflow Pipelines SKD section of the SETTINGS window. The format is '....[region].pipelines.googleusercontent.com'.
Step4:  Creating the TFX CLI builder
Review the Dockerfile for the TFX CLI builder
Step5:  Build the image and push it to your project's Container Registry
Hint
Step7:  Exercise
Step8:  Hint
<ASSISTANT_TASK:>
Python Code:
import yaml
# Set `PATH` to include the directory containing TFX CLI.
PATH = %env PATH
%env PATH=/home/jupyter/.local/bin:{PATH}
!python -c "import tfx; print('TFX version: {}'.format(tfx.__version__))"
Explanation: CI/CD for TFX pipelines
Learning Objectives
Develop a CI/CD workflow with Cloud Build to build and deploy TFX pipeline code.
Integrate with Github to automatically trigger pipeline deployment with source code repository changes.
In this lab, you will walk through authoring a Cloud Build CI/CD workflow that automatically builds and deploys the same TFX pipeline from lab-02.ipynb. You will also integrate your workflow with GitHub by setting up a trigger that starts the workflow when a new tag is applied to the GitHub repo hosting the pipeline's code.
Setup
End of explanation
%pip install --upgrade --user tfx==0.25.0
Explanation: Note: this lab was built and tested with the following package versions:
TFX version: 0.25.0
(Optional) If the TFX version above does not match the lab tested defaults, run the command below:
End of explanation
# Use the following command to identify the GCS bucket for metadata and pipeline storage.
!gsutil ls
Explanation: Note: you may need to restart the kernel to pick up the correct package versions.
Understanding the Cloud Build workflow
Review the cloudbuild.yaml file to understand how the CI/CD workflow is implemented and how environment specific settings are abstracted using Cloud Build variables.
The Cloud Build CI/CD workflow automates the steps you walked through manually during lab-02:
1. Builds the custom TFX image to be used as a runtime execution environment for TFX components and as the AI Platform Training training container.
1. Compiles the pipeline and uploads the pipeline to the KFP environment
1. Pushes the custom TFX image to your project's Container Registry
The Cloud Build workflow configuration uses both standard and custom Cloud Build builders. The custom builder encapsulates TFX CLI. 
Configuring environment settings
Navigate to AI Platform Pipelines page in the Google Cloud Console.
Create or select an existing Kubernetes cluster (GKE) and deploy AI Platform
Make sure to select "Allow access to the following Cloud APIs https://www.googleapis.com/auth/cloud-platform" to allow for programmatic access to your pipeline by the Kubeflow SDK for the rest of the lab. Also, provide an App instance name such as "tfx" or "mlops". Note you may have already deployed an AI Pipelines instance during the Setup for the lab series. If so, you can proceed using that instance below in the next step.
Validate the deployment of your AI Platform Pipelines instance in the console before proceeding.
Configure environment settings
Update  the below constants  with the settings reflecting your lab environment. 
GCP_REGION - the compute region for AI Platform Training and Prediction
ARTIFACT_STORE - the GCS bucket created during installation of AI Platform Pipelines. The bucket name starts with the kubeflowpipelines- prefix.
End of explanation
# TODO: Set your environment resource settings here for GCP_REGION, ARTIFACT_STORE_URI, ENDPOINT, and CUSTOM_SERVICE_ACCOUNT.
GCP_REGION = ""
ARTIFACT_STORE_URI = ""
CUSTOM_SERVICE_ACCOUNT = ""
ENDPOINT = ""
PROJECT_ID = !(gcloud config get-value core/project)
PROJECT_ID = PROJECT_ID[0]
Explanation: CUSTOM_SERVICE_ACCOUNT - In the gcp console Click on the Navigation Menu and navigate to IAM &amp; Admin, then to Service Accounts and use the service account starting with prefix - 'tfx-tuner-caip-service-account'. This enables CloudTuner and the Google Cloud AI Platform extensions Tuner component to work together and allows for distributed and parallel tuning backed by AI Platform Vizier's hyperparameter search algorithm. Please see the lab setup README for setup instructions.
ENDPOINT - set the ENDPOINT constant to the endpoint to your AI Platform Pipelines instance. The endpoint to the AI Platform Pipelines instance can be found on the AI Platform Pipelines page in the Google Cloud Console. Open the SETTINGS for your instance and use the value of the host variable in the Connect to this Kubeflow Pipelines instance from a Python client via Kubeflow Pipelines SKD section of the SETTINGS window. The format is '....[region].pipelines.googleusercontent.com'.
End of explanation
!cat tfx-cli/Dockerfile
!cat tfx-cli/requirements.txt
Explanation: Creating the TFX CLI builder
Review the Dockerfile for the TFX CLI builder
End of explanation
IMAGE_NAME = "tfx-cli"
TAG = "latest"
IMAGE_URI = f"gcr.io/{PROJECT_ID}/{IMAGE_NAME}:{TAG}"
# TODO: Your gcloud command here to build tfx-cli and submit to Container Registry.
Explanation: Build the image and push it to your project's Container Registry
Hint: Review the Cloud Build gcloud command line reference for builds submit. Your image should follow the format gcr.io/[PROJECT_ID]/[IMAGE_NAME]:latest. Note the source code for the tfx-cli is in the directory ./tfx-cli.
End of explanation
PIPELINE_NAME = "tfx_covertype_continuous_training"
MODEL_NAME = "tfx_covertype_classifier"
DATA_ROOT_URI = "gs://workshop-datasets/covertype/small"
TAG_NAME = "test"
TFX_IMAGE_NAME = "lab-tfx-cicd-image"
PIPELINE_FOLDER = "pipeline"
PIPELINE_DSL = "runner.py"
RUNTIME_VERSION = "2.3"
PYTHON_VERSION = "3.7"
USE_KFP_SA = "False"
ENABLE_TUNING = "True"
SUBSTITUTIONS = 
_GCP_REGION={},\
_ARTIFACT_STORE_URI={},\
_CUSTOM_SERVICE_ACCOUNT={},\
_ENDPOINT={},\
_PIPELINE_NAME={},\
_MODEL_NAME={},\
_DATA_ROOT_URI={},\
_TFX_IMAGE_NAME={},\
TAG_NAME={},\
_PIPELINE_FOLDER={},\
_PIPELINE_DSL={},\
_RUNTIME_VERSION={},\
_PYTHON_VERSION={},\
_USE_KFP_SA={},\
_ENABLE_TUNING={},
.format(
    GCP_REGION,
    ARTIFACT_STORE_URI,
    CUSTOM_SERVICE_ACCOUNT,
    ENDPOINT,
    PIPELINE_NAME,
    MODEL_NAME,
    DATA_ROOT_URI,
    TFX_IMAGE_NAME,
    TAG_NAME,
    PIPELINE_FOLDER,
    PIPELINE_DSL,
    RUNTIME_VERSION,
    PYTHON_VERSION,
    USE_KFP_SA,
    ENABLE_TUNING,
).strip()
Explanation: Exercise: manually trigger CI/CD pipeline run with Cloud Build
You can manually trigger Cloud Build runs using the gcloud builds submit command.
End of explanation
# TODO: write gcloud builds submit command to trigger manual pipeline run.
Explanation: Hint: you can manually trigger Cloud Build runs using the gcloud builds submit command. See the documentation for pass the cloudbuild.yaml file and SUBSTITIONS as arguments.
End of explanation
<END_TASK>