<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Neural Architecture Search with DARTS
In this example you will deploy Katib Experiment with Differentiable Architecture Search (DARTS) algorithm using Jupyter Notebook and Katib SDK. Your Kubernetes cluster must have at least one GPU for this example.
You can read more about how we use DARTS in Katib here.
The notebook shows how to create, get, check status and delete an Experiment.
Step1:  Import required packages
Step2:  Define your Experiment
You have to create your Experiment object before deploying it. This Experiment is similar to this example.
You can read more about DARTS algorithm settings here.
Step3:  You can print the Experiment's info to verify it before submission.
Step4:  Create your Experiment
You have to create Katib client to use the SDK
TODO (andreyvelich)
Step5:  Get your Experiment
You can get your Experiment by name and receive required data.
Step6:  Get the current Experiment status
You can check the current Experiment status.
Step7:  You can check if your Experiment is succeeded.
Step8:  Get the best Genotype
Best Genotype is located in the optimal Trial currently. The latest Genotype is the best.
Check your Trial logs to get more information about the training process.
Step9:  Delete your Experiments
You can delete your Experiments.
<ASSISTANT_TASK:>
Python Code:
# Install required package (Katib SDK).
!pip install kubeflow-katib==0.13.0
Explanation: Neural Architecture Search with DARTS
In this example you will deploy Katib Experiment with Differentiable Architecture Search (DARTS) algorithm using Jupyter Notebook and Katib SDK. Your Kubernetes cluster must have at least one GPU for this example.
You can read more about how we use DARTS in Katib here.
The notebook shows how to create, get, check status and delete an Experiment.
End of explanation
from kubeflow.katib import KatibClient
from kubernetes.client import V1ObjectMeta
from kubeflow.katib import V1beta1Experiment
from kubeflow.katib import V1beta1AlgorithmSpec
from kubeflow.katib import V1beta1AlgorithmSetting
from kubeflow.katib import V1beta1ObjectiveSpec
from kubeflow.katib import V1beta1MetricsCollectorSpec
from kubeflow.katib import V1beta1CollectorSpec
from kubeflow.katib import V1beta1SourceSpec
from kubeflow.katib import V1beta1FilterSpec
from kubeflow.katib import V1beta1FeasibleSpace
from kubeflow.katib import V1beta1ExperimentSpec
from kubeflow.katib import V1beta1NasConfig
from kubeflow.katib import V1beta1GraphConfig
from kubeflow.katib import V1beta1Operation
from kubeflow.katib import V1beta1ParameterSpec
from kubeflow.katib import V1beta1TrialTemplate
from kubeflow.katib import V1beta1TrialParameterSpec
Explanation: Import required packages
End of explanation
# Experiment name and namespace.
namespace = "kubeflow-user-example-com"
experiment_name = "darts-example"
metadata = V1ObjectMeta(
    name=experiment_name,
    namespace=namespace
)
# Algorithm specification.
algorithm_spec=V1beta1AlgorithmSpec(
    algorithm_name="darts",
    algorithm_settings=[
        V1beta1AlgorithmSetting(
            name="num_epochs",
            value="2"
        ),
        V1beta1AlgorithmSetting(
            name="stem_multiplier",
            value="1"
        ),
        V1beta1AlgorithmSetting(
            name="init_channels",
            value="4"
        ),
        V1beta1AlgorithmSetting(
            name="num_nodes",
            value="3"
        ),
        
    ]
)
# Objective specification. For DARTS Goal is omitted.
objective_spec=V1beta1ObjectiveSpec(
    type="maximize",
    objective_metric_name="Best-Genotype",
)
# Metrics collector specification.
# We should specify metrics format to get Genotype from training container.
metrics_collector_spec=V1beta1MetricsCollectorSpec(
    collector=V1beta1CollectorSpec(
        kind="StdOut"
    ),
    source=V1beta1SourceSpec(
        filter=V1beta1FilterSpec(
            metrics_format=[
                "([\\w-]+)=(Genotype.*)"
            ]
        )
    )
)
# Configuration for the Neural Network (NN).
# This NN contains 2 number of layers and 5 various operations with different parameters.
nas_config=V1beta1NasConfig(
    graph_config=V1beta1GraphConfig(
        num_layers=2
    ),
    operations=[
        V1beta1Operation(
            operation_type="separable_convolution",
            parameters=[
                V1beta1ParameterSpec(
                    name="filter_size",
                    parameter_type="categorical",
                    feasible_space=V1beta1FeasibleSpace(
                        list=["3"]
                    ),
                )
            ]
        ),
        V1beta1Operation(
            operation_type="dilated_convolution",
            parameters=[
                V1beta1ParameterSpec(
                    name="filter_size",
                    parameter_type="categorical",
                    feasible_space=V1beta1FeasibleSpace(
                        list=["3", "5"]
                    ),
                )
            ]
        ),
        V1beta1Operation(
            operation_type="avg_pooling",
            parameters=[
                V1beta1ParameterSpec(
                    name="filter_size",
                    parameter_type="categorical",
                    feasible_space=V1beta1FeasibleSpace(
                        list=["3"]
                    ),
                )
            ]
        ),
        V1beta1Operation(
            operation_type="max_pooling",
            parameters=[
                V1beta1ParameterSpec(
                    name="filter_size",
                    parameter_type="categorical",
                    feasible_space=V1beta1FeasibleSpace(
                        list=["3"]
                    ),
                )
            ]
        ),
        V1beta1Operation(
            operation_type="skip_connection",
        ),
    ]
)
# JSON template specification for the Trial's Worker Kubernetes Job.
trial_spec={
    "apiVersion": "batch/v1",
    "kind": "Job",
    "spec": {
        "template": {
            "metadata": {
                "annotations": {
                    "sidecar.istio.io/inject": "false"
                }
            },
            "spec": {
                "containers": [
                    {
                        "name": "training-container",
                        "image": "docker.io/kubeflowkatib/darts-cnn-cifar10:v0.13.0",
                        "command": [
                            'python3',
                            'run_trial.py',
                            '--algorithm-settings="${trialParameters.algorithmSettings}"',
                            '--search-space="${trialParameters.searchSpace}"',
                            '--num-layers="${trialParameters.numberLayers}"'
                        ],
                        # Training container requires 1 GPU.
                        "resources": {
                            "limits": {
                                "nvidia.com/gpu": 1
                            }
                        }
                    }
                ],
                "restartPolicy": "Never"
            }
        }
    }
}
# Template with Trial parameters and Trial spec.
# Set retain to True to save trial resources after completion.
trial_template=V1beta1TrialTemplate(
    retain=True,
    primary_container_name="training-container",
    trial_parameters=[
        V1beta1TrialParameterSpec(
            name="algorithmSettings",
            description=" Algorithm settings of DARTS Experiment",
            reference="algorithm-settings"
        ),
        V1beta1TrialParameterSpec(
            name="searchSpace",
            description="Search Space of DARTS Experiment",
            reference="search-space"
        ),
        V1beta1TrialParameterSpec(
            name="numberLayers",
            description="Number of Neural Network layers",
            reference="num-layers"
        ),
    ],
    trial_spec=trial_spec
)
# Experiment object.
experiment = V1beta1Experiment(
    api_version="kubeflow.org/v1beta1",
    kind="Experiment",
    metadata=metadata,
    spec=V1beta1ExperimentSpec(
        max_trial_count=1,
        parallel_trial_count=1,
        max_failed_trial_count=1,
        algorithm=algorithm_spec,
        objective=objective_spec,
        metrics_collector_spec=metrics_collector_spec,
        nas_config=nas_config,
        trial_template=trial_template,
    )
)
Explanation: Define your Experiment
You have to create your Experiment object before deploying it. This Experiment is similar to this example.
You can read more about DARTS algorithm settings here.
End of explanation
# Print the Trial template container info.
print(experiment.spec.trial_template.trial_spec["spec"]["template"]["spec"]["containers"][0])
Explanation: You can print the Experiment's info to verify it before submission.
End of explanation
# Create client.
kclient = KatibClient()
# Create your Experiment.
kclient.create_experiment(experiment,namespace=namespace)
Explanation: Create your Experiment
You have to create Katib client to use the SDK
TODO (andreyvelich): Current Experiment link for NAS is incorrect.
End of explanation
exp = kclient.get_experiment(name=experiment_name, namespace=namespace)
print(exp)
print("-----------------\n")
# Get the latest status.
print(exp["status"]["conditions"][-1])
Explanation: Get your Experiment
You can get your Experiment by name and receive required data.
End of explanation
kclient.get_experiment_status(name=experiment_name, namespace=namespace)
Explanation: Get the current Experiment status
You can check the current Experiment status.
End of explanation
kclient.is_experiment_succeeded(name=experiment_name, namespace=namespace)
Explanation: You can check if your Experiment is succeeded.
End of explanation
opt_trial = kclient.get_optimal_hyperparameters(name=experiment_name, namespace=namespace)
best_genotype = opt_trial["currentOptimalTrial"]["observation"]["metrics"][0]["latest"]
print(best_genotype)
Explanation: Get the best Genotype
Best Genotype is located in the optimal Trial currently. The latest Genotype is the best.
Check your Trial logs to get more information about the training process.
End of explanation
kclient.delete_experiment(name=experiment_name, namespace=namespace)
Explanation: Delete your Experiments
You can delete your Experiments.
End of explanation
<END_TASK>