<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Reinforcement Learning on Google Cloud
Ran from datalab. You can do this from cloud using datalab create ... Or here are steps to get datalab running locally
Step1:  First run locally to make sure everything is working.
Step2:  Run on ML-Engine
Step3:  TODO
Step4:  Launch tensorboard
Using tensoboard --logdir='gs
<ASSISTANT_TASK:>
Python Code:
%%bash
# Install packages to test model locally.
apt-get update
apt-get install -y python-numpy python-dev cmake zlib1g-dev libjpeg-dev xvfb libav-tools xorg-dev python-opengl libboost-all-dev libsdl2-dev swig libffi-dev
pip install gym
pip install gym[atari]
pip install opencv-python
apt update && apt install -y libsm6 libxext6
apt-get install -y libxrender-dev
pip install keras
Explanation: Reinforcement Learning on Google Cloud
Ran from datalab. You can do this from cloud using datalab create ... Or here are steps to get datalab running locally:
sudo docker run -it -p "127.0.0.1:8081:8080" -v "${HOME}:/content"      -e "PROJECT_ID=[YOUR_PROJECT_ID]"  gcr.io/cloud-datalab/datalab:local
Click user image at top right of datalab, and login to you GCP account. Gives you authentification for everything :)
End of explanation
%%bash
rm -r rl_on_gcp/my_model
gcloud ml-engine local train \
   --module-name=trainer.trainer \
   --package-path=${PWD}/rl_on_gcp/trainer \
   --\
   --steps=100000\
   --start_train=100\
   --buffer_size=100\
   --save_model=True\
   --model_dir='my_model'
Explanation: First run locally to make sure everything is working.
End of explanation
%%bash
JOBNAME=rl_breakout_$(date -u +%y%m%d_%H%M%S)
REGION='us-central1'
BUCKET='dqn-breakout'
gcloud ml-engine jobs submit training $JOBNAME \
        --package-path=$PWD/rl_on_gcp/trainer \
        --module-name=trainer.trainer \
        --region=$REGION \
        --staging-bucket=gs://$BUCKET \
        --scale-tier=BASIC_TPU\
        --runtime-version=1.9 \
        --\
        --steps=5000000\
        --start_train=10000\
        --buffer_size=1000000\
        --save_model=True\
        --model_dir='gs://dqn-breakout/models/'
Explanation: Run on ML-Engine
End of explanation
%%writefile hyperparam.yaml
trainingInput:
  scaleTier: BASIC_GPU
  hyperparameters:
    maxTrials: 40
    maxParallelTrials: 5
    enableTrialEarlyStopping: False
    goal: MAXIMIZE    
    hyperparameterMetricTag: reward
    params:
    - parameterName: update_target
      type: INTEGER
      minValue: 500
      maxValue: 5000
      scaleType: UNIT_LOG_SCALE
    - parameterName: init_eta
      type: DOUBLE
      minValue: 0.8
      maxValue: 0.95
      scaleType: UNIT_LOG_SCALE
    - parameterName: learning_rate
      type: DOUBLE
      minValue: 0.00001
      maxValue: 0.001
      scaleType: UNIT_LOG_SCALE
    - parameterName: batch_size
      type: DISCRETE
      discreteValues:
      - 4
      - 16
      - 32
      - 64
      - 128
      - 256
      - 512
%%bash
JOBNAME=rl_breakout_hp_$(date -u +%y%m%d_%H%M%S)
REGION='us-central1'
BUCKET='dqn-breakout'
gcloud ml-engine jobs submit training $JOBNAME \
        --package-path=$PWD/rl_on_gcp/trainer \
        --module-name=trainer.trainer \
        --region=$REGION \
        --staging-bucket=gs://$BUCKET \
        --config=hyperparam.yaml \
        --runtime-version=1.10 \
        --\
        --steps=100000\
        --start_train=10000\
        --buffer_size=10000\
        --model_dir='gs://dqn-breakout/models/hp/'
Explanation: TODO: why doesn't it use GPU?
From logs using BASIC_GPU:
master-replica-0 | Processes:                                                       GPU Memory | I  master-replica-0
 master-replica-0 |  GPU       PID   Type   Process name                             Usage      | I  master-replica-0
 master-replica-0 |=============================================================================| I  master-replica-0
 master-replica-0 |  No running processes found                                                 | I  master-replica-0
Hyperparameter tuning
End of explanation
from google.datalab.ml import TensorBoard as tb
tb.start('gs://crawles-sandbox/rl_on_gcp/hp/4/')
!gsutil ls  gs://crawles-sandbox/rl_on_gcp/hp/4/
Explanation: Launch tensorboard
Using tensoboard --logdir='gs://dqn-breakout/models/' or within datalab:
End of explanation
<END_TASK>