<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Exploration vs exploitation
Sigurd Carlen, September 2019.
Reformatted by Holger Nahrstaedt 2020
.. currentmodule
Step1:  Toy example
First we define our objective like in the ask-and-tell example notebook and
define a plotting function. We do however only use on initial random point.
All points afterthe first one is therefore choosen by the acquisition
function.
Step2:  We run a an optimization loop with standard settings
Step3:  We see that some minima is found and "exploited"
Now lets try to set kappa and xi using'to other values and
pass it to the optimizer
Step4:  We see that the points are more random now.
This works both for kappa when using acq_func="LCB"
Step5:  And for xi when using acq_func="EI"
Step6:  We can also favor exploitaton
Step7:  Note that negative values does not work with the "PI"-acquisition function
but works with "EI"
Step8:  Changing kappa and xi on the go
If we want to change kappa or ki at any point during our optimization
process we just replace opt.acq_func_kwargs. Remember to call
opt.update_next() after the change, in order for next point to be
recalculated.
<ASSISTANT_TASK:>
Python Code:
print(__doc__)
import numpy as np
np.random.seed(1234)
import matplotlib.pyplot as plt
Explanation: Exploration vs exploitation
Sigurd Carlen, September 2019.
Reformatted by Holger Nahrstaedt 2020
.. currentmodule:: skopt
We can control how much the acqusition function favors exploration and
exploitation by tweaking the two parameters kappa and xi. Higher values
means more exploration and less exploitation and vice versa with low values.
kappa is only used if acq_func is set to "LCB". xi is used when acq_func is
"EI" or "PI". By default the acqusition function is set to "gp_hedge" which
chooses the best of these three. Therefore I recommend not using gp_hedge
when tweaking exploration/exploitation, but instead choosing "LCB",
"EI" or "PI.
The way to pass kappa and xi to the optimizer is to use the named argument
"acq_func_kwargs". This is a dict of extra arguments for the aqcuisittion
function.
If you want opt.ask() to give a new acquisition value imdediatly after
tweaking kappa or xi call opt.update_next(). This ensures that the next
value is updated with the new acquisition parameters.
End of explanation
from skopt.learning import ExtraTreesRegressor
from skopt import Optimizer
noise_level = 0.1
# Our 1D toy problem, this is the function we are trying to
# minimize
def objective(x, noise_level=noise_level):
    return np.sin(5 * x[0]) * (1 - np.tanh(x[0] ** 2)) +\
           np.random.randn() * noise_level
opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points = 1,
                acq_optimizer="sampling")
x = np.linspace(-2, 2, 400).reshape(-1, 1)
fx = np.array([objective(x_i, noise_level=0.0) for x_i in x])
from skopt.acquisition import gaussian_ei
def plot_optimizer(opt, x, fx):
    model = opt.models[-1]
    x_model = opt.space.transform(x.tolist())
    # Plot true function.
    plt.plot(x, fx, "r--", label="True (unknown)")
    plt.fill(np.concatenate([x, x[::-1]]),
             np.concatenate([fx - 1.9600 * noise_level,
                             fx[::-1] + 1.9600 * noise_level]),
             alpha=.2, fc="r", ec="None")
    # Plot Model(x) + contours
    y_pred, sigma = model.predict(x_model, return_std=True)
    plt.plot(x, y_pred, "g--", label=r"$\mu(x)$")
    plt.fill(np.concatenate([x, x[::-1]]),
             np.concatenate([y_pred - 1.9600 * sigma,
                             (y_pred + 1.9600 * sigma)[::-1]]),
             alpha=.2, fc="g", ec="None")
    # Plot sampled points
    plt.plot(opt.Xi, opt.yi,
             "r.", markersize=8, label="Observations")
    acq = gaussian_ei(x_model, model, y_opt=np.min(opt.yi))
    # shift down to make a better plot
    acq = 4 * acq - 2
    plt.plot(x, acq, "b", label="EI(x)")
    plt.fill_between(x.ravel(), -2.0, acq.ravel(), alpha=0.3, color='blue')
    # Adjust plot layout
    plt.grid()
    plt.legend(loc='best')
Explanation: Toy example
First we define our objective like in the ask-and-tell example notebook and
define a plotting function. We do however only use on initial random point.
All points afterthe first one is therefore choosen by the acquisition
function.
End of explanation
for i in range(30):
    next_x = opt.ask()
    f_val = objective(next_x)
    opt.tell(next_x, f_val)
# The same output could be created with opt.run(objective, n_iter=30)
plot_optimizer(opt, x, fx)
Explanation: We run a an optimization loop with standard settings
End of explanation
acq_func_kwargs = {"xi": 10000, "kappa": 10000}
opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=1,
                acq_optimizer="sampling",
                acq_func_kwargs=acq_func_kwargs)
opt.run(objective, n_iter=20)
plot_optimizer(opt, x, fx)
Explanation: We see that some minima is found and "exploited"
Now lets try to set kappa and xi using'to other values and
pass it to the optimizer:
End of explanation
opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=1,
                acq_func="LCB", acq_optimizer="sampling",
                acq_func_kwargs=acq_func_kwargs)
opt.run(objective, n_iter=20)
plot_optimizer(opt, x, fx)
Explanation: We see that the points are more random now.
This works both for kappa when using acq_func="LCB":
End of explanation
opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=1,
                acq_func="PI", acq_optimizer="sampling",
                acq_func_kwargs=acq_func_kwargs)
opt.run(objective, n_iter=20)
plot_optimizer(opt, x, fx)
Explanation: And for xi when using acq_func="EI": or acq_func="PI":
End of explanation
acq_func_kwargs = {"xi": 0.000001, "kappa": 0.001}
opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=1,
                acq_func="LCB", acq_optimizer="sampling",
                acq_func_kwargs=acq_func_kwargs)
opt.run(objective, n_iter=20)
plot_optimizer(opt, x, fx)
opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=1,
                acq_func="EI", acq_optimizer="sampling",
                acq_func_kwargs=acq_func_kwargs)
opt.run(objective, n_iter=20)
plot_optimizer(opt, x, fx)
opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=1,
                acq_func="PI", acq_optimizer="sampling",
                acq_func_kwargs=acq_func_kwargs)
opt.run(objective, n_iter=20)
plot_optimizer(opt, x, fx)
Explanation: We can also favor exploitaton:
End of explanation
acq_func_kwargs = {"xi": -1000000000000}
opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=1,
                acq_func="PI", acq_optimizer="sampling",
                acq_func_kwargs=acq_func_kwargs)
opt.run(objective, n_iter=20)
plot_optimizer(opt, x, fx)
opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=1,
                acq_func="EI", acq_optimizer="sampling",
                acq_func_kwargs=acq_func_kwargs)
opt.run(objective, n_iter=20)
plot_optimizer(opt, x, fx)
Explanation: Note that negative values does not work with the "PI"-acquisition function
but works with "EI":
End of explanation
acq_func_kwargs = {"kappa": 0}
opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=1,
                acq_func="LCB", acq_optimizer="sampling",
                acq_func_kwargs=acq_func_kwargs)
opt.acq_func_kwargs
opt.run(objective, n_iter=20)
plot_optimizer(opt, x, fx)
acq_func_kwargs = {"kappa": 100000}
opt.acq_func_kwargs = acq_func_kwargs
opt.update_next()
opt.run(objective, n_iter=20)
plot_optimizer(opt, x, fx)
Explanation: Changing kappa and xi on the go
If we want to change kappa or ki at any point during our optimization
process we just replace opt.acq_func_kwargs. Remember to call
opt.update_next() after the change, in order for next point to be
recalculated.
End of explanation
<END_TASK>