<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 # Table of Contents
<div class="toc" style="margin-top
Step1:  Convert targets to categorical
We will one-hot-encode the target array
Step2:  Assign observations to bins, part 1
Step3:  Assign observations to bins, part 2
Step4:  Are we overfitting?
Step5:  Only at the end, try a more complex model.
Step6:  Bin size tuning
Step7:  How complex does a network have to be to produce a clean PDF
Step8:  Convert to relative bins
Step9:  Full ensemble features
Step10:  With convolution
Step11:  Full ensemble with mean and std output
Step12:  With station embeddings
Step13:  With relative bins
Step14:  With full ensemble input
Step15:  Standard bins
Step16:  testing a convolution layer
Step17:  Auxiliary data
Mean and std
Step18:  Full ensemble
Step19:  XGBoost
<ASSISTANT_TASK:>
Python Code:
# Imports
from importlib import reload
import utils; reload(utils)
from utils import *
import losses; reload(losses)
from losses import approx_crps_cat, approx_crps_cat, crps_cost_function
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
from collections import OrderedDict
import keras
from keras.layers import Input, Dense, merge, Embedding, Flatten, Dropout
from keras.layers.merge import Concatenate
from keras.models import Model
import keras.backend as K
from keras.callbacks import EarlyStopping
from keras.optimizers import SGD, Adam
from scipy.stats import binned_statistic
# Setup
# DATA_DIR = '/Volumes/STICK/data/ppnn_data/'  # Mac
DATA_DIR = '/project/meteo/w2w/C7/ppnn_data/'   # LMU
fclt = 48
Explanation: # Table of Contents
<div class="toc" style="margin-top: 1em;"><ul class="toc-item" id="toc-level0"><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#Convert-targets-to-categorical" data-toc-modified-id="Convert-targets-to-categorical-1"><span class="toc-item-num">1&nbsp;&nbsp;</span>Convert targets to categorical</a></span><ul class="toc-item"><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#Assign-observations-to-bins,-part-1:-Always-to-the-left" data-toc-modified-id="Assign-observations-to-bins,-part-1:-Always-to-the-left-1.1"><span class="toc-item-num">1.1&nbsp;&nbsp;</span>Assign observations to bins, part 1: Always to the left</a></span></li><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#Assign-observations-to-bins,-part-2:-Nearest-edge" data-toc-modified-id="Assign-observations-to-bins,-part-2:-Nearest-edge-1.2"><span class="toc-item-num">1.2&nbsp;&nbsp;</span>Assign observations to bins, part 2: Nearest edge</a></span></li><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#Are-we-overfitting?" data-toc-modified-id="Are-we-overfitting?-1.3"><span class="toc-item-num">1.3&nbsp;&nbsp;</span>Are we overfitting?</a></span></li><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#Bin-size-tuning" data-toc-modified-id="Bin-size-tuning-1.4"><span class="toc-item-num">1.4&nbsp;&nbsp;</span>Bin size tuning</a></span></li><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#How-complex-does-a-network-have-to-be-to-produce-a-clean-PDF" data-toc-modified-id="How-complex-does-a-network-have-to-be-to-produce-a-clean-PDF-1.5"><span class="toc-item-num">1.5&nbsp;&nbsp;</span>How complex does a network have to be to produce a clean PDF</a></span></li><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#Convert-to-relative-bins" data-toc-modified-id="Convert-to-relative-bins-1.6"><span class="toc-item-num">1.6&nbsp;&nbsp;</span>Convert to relative bins</a></span></li></ul></li><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#Full-ensemble-features" data-toc-modified-id="Full-ensemble-features-2"><span class="toc-item-num">2&nbsp;&nbsp;</span>Full ensemble features</a></span><ul class="toc-item"><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#With-convolution" data-toc-modified-id="With-convolution-2.1"><span class="toc-item-num">2.1&nbsp;&nbsp;</span>With convolution</a></span></li><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#Full-ensemble-with-mean-and-std-output" data-toc-modified-id="Full-ensemble-with-mean-and-std-output-2.2"><span class="toc-item-num">2.2&nbsp;&nbsp;</span>Full ensemble with mean and std output</a></span></li></ul></li><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#With-station-embeddings" data-toc-modified-id="With-station-embeddings-3"><span class="toc-item-num">3&nbsp;&nbsp;</span>With station embeddings</a></span><ul class="toc-item"><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#With-relative-bins" data-toc-modified-id="With-relative-bins-3.1"><span class="toc-item-num">3.1&nbsp;&nbsp;</span>With relative bins</a></span></li><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#With-full-ensemble-input" data-toc-modified-id="With-full-ensemble-input-3.2"><span class="toc-item-num">3.2&nbsp;&nbsp;</span>With full ensemble input</a></span></li><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#Standard-bins" data-toc-modified-id="Standard-bins-3.3"><span class="toc-item-num">3.3&nbsp;&nbsp;</span>Standard bins</a></span></li><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#testing-a-convolution-layer" data-toc-modified-id="testing-a-convolution-layer-3.4"><span class="toc-item-num">3.4&nbsp;&nbsp;</span>testing a convolution layer</a></span></li></ul></li><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#Auxiliary-data" data-toc-modified-id="Auxiliary-data-4"><span class="toc-item-num">4&nbsp;&nbsp;</span>Auxiliary data</a></span><ul class="toc-item"><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#Mean-and-std" data-toc-modified-id="Mean-and-std-4.1"><span class="toc-item-num">4.1&nbsp;&nbsp;</span>Mean and std</a></span></li><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#Full-ensemble" data-toc-modified-id="Full-ensemble-4.2"><span class="toc-item-num">4.2&nbsp;&nbsp;</span>Full ensemble</a></span></li></ul></li><li><span><a href="http://localhost:8888/notebooks/notebooks/categorical_networks.ipynb#XGBoost" data-toc-modified-id="XGBoost-5"><span class="toc-item-num">5&nbsp;&nbsp;</span>XGBoost</a></span></li></ul></div>
Categorical/binned output
Here we will use categorical output instead of the mean and standard deviation.
End of explanation
train_dates = ['2010-01-01', '2016-01-01']
test_dates =  ['2016-01-01', '2017-01-01']
train_set, test_set = get_train_test_sets(DATA_DIR, train_dates, test_dates)
# Define the bins
bin_width = 0.5
bin_edges = np.arange(-30, 35 + bin_width, bin_width)
bin_edges.shape
Explanation: Convert targets to categorical
We will one-hot-encode the target array
End of explanation
def convert_targets(targets, bin_edges):
    bin_idxs = binned_statistic(targets, targets, bins=bin_edges)[-1]
    return keras.utils.to_categorical(bin_idxs, num_classes=bin_edges.shape[0]-1)
train_cat_targets = convert_targets(train_set.targets, bin_edges)
test_cat_targets = convert_targets(test_set.targets, bin_edges)
print('Obs:', test_set.targets[0])
plt.bar(bin_edges[:-1] + 0.25, test_cat_targets[0], width=0.5, zorder=0.1)
plt.axvline(test_set.targets[0], c='r')
plt.show()
def build_cat_model(n_features, hidden_nodes, n_bins):
    inp = Input(shape=(n_features,))
    x = Dense(hidden_nodes[0], activation='relu')(inp)
    if len(hidden_nodes) > 1:
        for h in hidden_nodes[1:]:
            x = Dense(h, activation='relu')(x)
    x = Dense(n_bins, activation='softmax')(x)
    return Model(inputs=inp, outputs=x)
model = build_cat_model(2, [50], bin_edges.shape[0]-1)
model.compile(optimizer=Adam(0.01), loss=approx_crps_cat(bin_width))
model.summary()
model.fit(train_set.features, train_cat_targets, epochs=5, batch_size=4096,
          validation_data=[test_set.features, test_cat_targets])
preds = model.predict(test_set.features, batch_size=4096)
# Get correct CRPS
maybe_correct_cat_crps(preds, test_set.targets, bin_edges)
i = 10
plt.bar(bin_edges[:-1] + 0.25, preds[i], width=0.5, alpha=0.8)
plt.bar(bin_edges[:-1] + 0.25, test_cat_targets[i], width=0.5, zorder=0.1)
plt.axvline(test_set.targets[i], c='r')
plt.show()
Explanation: Assign observations to bins, part 1: Always to the left
End of explanation
def convert_targets_nearest(targets, bin_edges):
    mat_bin_edges = np.repeat(np.atleast_2d(bin_edges), targets.shape[0], axis=0).T
    bin_idxs = np.argmin(np.abs(mat_bin_edges - targets),axis=0)
    return keras.utils.to_categorical(bin_idxs, num_classes=bin_edges.shape[0]-1)
train_cat_targets_nearest = convert_targets_nearest(train_set.targets, bin_edges)
test_cat_targets_nearest = convert_targets_nearest(test_set.targets, bin_edges)
model = build_cat_model(2, [50], bin_edges.shape[0]-1)
model.compile(optimizer=Adam(0.01), loss=approx_crps_cat(bin_width))
model.fit(train_set.features, train_cat_targets_nearest, epochs=5, batch_size=1024,
          validation_data=[test_set.features, test_cat_targets_nearest])
model.optimizer.lr = 1e-3
model.fit(train_set.features, train_cat_targets_nearest, epochs=5, batch_size=1024,
          validation_data=[test_set.features, test_cat_targets_nearest])
preds = model.predict(test_set.features)
maybe_correct_cat_crps(preds, test_set.targets, bin_edges)
for i in np.random.randint(0, 182218, 10):
    plt.bar(bin_edges[:-1] + 0.25, preds[i], width=0.5, alpha=0.8)
    plt.bar(bin_edges[:-1] + 0.25, test_cat_targets_nearest[i], width=0.5, zorder=0.1)
    plt.axvline(test_set.targets[i], c='r')
    plt.axvline(test_set.features[i, 0] * test_set.scale_factors[0], c='g')
    plt.show()
plt.hist(test_set.targets, bins = 50);
Explanation: Assign observations to bins, part 2: Nearest edge
End of explanation
model = build_cat_model(2, [50], bin_edges.shape[0]-1)
model.compile(optimizer=Adam(0.01), loss=approx_crps_cat(bin_width))
model.fit(train_set.features, train_cat_targets_nearest, epochs=5, batch_size=1024,
              validation_split=0.2)
Explanation: Are we overfitting?
End of explanation
model = build_cat_model(2, [500, 500], bin_edges.shape[0]-1)
model.compile(optimizer=Adam(0.01), loss=approx_crps_cat(bin_width))
model.fit(train_set.features, train_cat_targets_nearest, epochs=5, batch_size=1024,
              validation_split=0.2)
Explanation: Only at the end, try a more complex model.
End of explanation
wide_bin_width = 1.5
wide_bin_edges = np.arange(-30, 35 + wide_bin_width, wide_bin_width)
wide_train_cat_targets = convert_targets_nearest(train_set.targets, wide_bin_edges)
wide_test_cat_targets = convert_targets_nearest(test_set.targets, wide_bin_edges)
model = build_cat_model(2, [50], wide_bin_edges.shape[0]-1)
model.summary()
model.compile(optimizer=Adam(0.0001), loss=approx_crps)
model.fit(train_set.features, wide_train_cat_targets, epochs=5, batch_size=1024,
          validation_data=[test_set.features, wide_test_cat_targets])
preds = model.predict(test_set.features)
i = 100447
plt.bar(wide_bin_edges[:-1] + 0.75, preds[i], width=1.5, alpha=0.8)
plt.bar(wide_bin_edges[:-1] + 0.75, wide_test_cat_targets[i], width=1.5, zorder=0.1)
plt.axvline(test_set.targets[i], c='r')
plt.show()
maybe_correct_cat_crps(preds, test_set.targets, wide_bin_edges)
Explanation: Bin size tuning
End of explanation
preds[:, 1].mean()
# Create dummy dataset of means and stds
dummy_means = np.random.rand(100000) * 40. + 5
dummy_std = np.random.rand(100000) * 3.
# Create the corresponding PDFs in bins
dummy_bin_edges = np.arange(0, 50 + bin_width, bin_width)
Explanation: How complex does a network have to be to produce a clean PDF
End of explanation
mean_error = test_set.features[:, 0] * test_set.scale_factors[0] - test_set.targets
mean_error.min(), mean_error.max()
mean_error.argmin()
np.mean(np.abs(mean_error))
train_conv_targets = train_set.targets - train_set.features[:, 0] * train_set.scale_factors[0]
test_conv_targets = test_set.targets - test_set.features[:, 0] * test_set.scale_factors[0]
train_conv_targets.min(), train_conv_targets.max()
conv_bin_edges = np.arange(-15, 20 + bin_width, bin_width)
conv_train_cat_targets_nearest = convert_targets_nearest(train_conv_targets, conv_bin_edges)
conv_test_cat_targets_nearest = convert_targets_nearest(test_conv_targets, conv_bin_edges)
model = build_cat_model(2, [50], conv_bin_edges.shape[0]-1)
model.compile(optimizer=Adam(0.01), loss=approx_crps_cat(bin_width))
model.fit(train_set.features, conv_train_cat_targets_nearest, epochs=5, batch_size=1024,
          validation_data=[test_set.features, conv_test_cat_targets_nearest])
preds = model.predict(test_set.features)
maybe_correct_cat_crps(preds, test_conv_targets, conv_bin_edges)
for i in np.random.randint(0, 182218, 10):
    print(i)
    print(test_set.targets[i], test_set.features[i, 0] * test_set.scale_factors[0])
    plt.bar(conv_bin_edges[:-1] + 0.25, preds[i], width=0.5, alpha=0.8)
    plt.bar(conv_bin_edges[:-1] + 0.25, conv_test_cat_targets_nearest[i], width=0.5, zorder=0.1)
    plt.axvline(test_conv_targets[i], c='r')
    plt.show()
Explanation: Convert to relative bins
End of explanation
train_set_full_ens, test_set_full_ens = get_train_test_sets(DATA_DIR, train_dates, test_dates,
                                                            full_ensemble_t=True)
train_set_full_ens.features.shape
model = build_cat_model(50, [50], bin_edges.shape[0]-1)
model.summary()
model.compile(optimizer=Adam(0.0001), loss=approx_crps)
model.fit(train_set_full_ens.features, train_cat_targets_nearest, epochs=10, batch_size=1024,
          validation_data=[test_set_full_ens.features, test_cat_targets_nearest])
preds = model.predict(test_set_full_ens.features)
maybe_correct_cat_crps(preds, test_set.targets, bin_edges)
i = 10
plt.bar(bin_edges[:-1] + 0.25, preds[i], width=0.5, alpha=0.8)
plt.bar(bin_edges[:-1] + 0.25, test_cat_targets[i], width=0.5, zorder=0.1)
plt.axvline(test_set.targets[i], c='r')
plt.show()
Explanation: Full ensemble features
End of explanation
def build_cat_conv_model(hidden_nodes, feature_size, n_bins, maps):
    features_in = Input(shape=(feature_size, 1))
    conv = Conv1D(maps, feature_size)(features_in)
    conv = Flatten()(conv)
    x = Dense(hidden_nodes, activation='relu')(conv)
    x = Dense(n_bins, activation='softmax')(x)
    model = Model(inputs=features_in, outputs=x)
    return model
model = build_cat_conv_model(50, 50, bin_edges.shape[0]-1, 10)
model.summary()
model.compile(optimizer=Adam(0.0001), loss=approx_crps)
model.fit(np.atleast_3d(train_set_full_ens.features), train_cat_targets_nearest, 
          epochs=10, batch_size=1024,
          validation_data=[np.atleast_3d(test_set_full_ens.features), test_cat_targets_nearest])
Explanation: With convolution
End of explanation
def build_hidden_model(hidden_nodes, feature_size=2):
    inp = Input(shape=(feature_size,))
    x = Dense(hidden_nodes, activation='relu')(inp)
    x = Dense(2, activation='linear')(x)
    return Model(inputs=inp, outputs=x)
model = build_hidden_model(50, 50)
model.compile(optimizer=Adam(0.01), loss=crps_cost_function)
model.fit(train_set_full_ens.features, train_set_full_ens.targets, epochs=5, 
                 batch_size=1024,
                 validation_data=[test_set_full_ens.features, test_set_full_ens.targets])
preds = model.predict(test_set_full_ens.features)
preds[10], test_set_full_ens.targets[10]
Explanation: Full ensemble with mean and std output
End of explanation
def build_cat_emb_model(hidden_nodes, emb_size, max_id, feature_size, n_bins):
    features_in = Input(shape=(feature_size,))
    id_in = Input(shape=(1,))
    emb = Embedding(max_id + 1, emb_size)(id_in)
    emb = Flatten()(emb)
    x = Concatenate()([features_in, emb])
    x = Dense(hidden_nodes, activation='relu')(x)
    x = Dense(n_bins, activation='softmax')(x)
    model = Model(inputs=[features_in, id_in], outputs=x)
    return model
emb_size = 5
max_id = int(np.max([train_set.cont_ids.max(), test_set.cont_ids.max()]))
max_id
Explanation: With station embeddings
End of explanation
model = build_cat_emb_model(100, emb_size, max_id, 2, conv_bin_edges.shape[0]-1)
model.compile(optimizer=Adam(0.001), loss=approx_crps_cat(bin_width))
model.fit([train_set.features, train_set.cont_ids], conv_train_cat_targets_nearest, 
              epochs=5, batch_size=1024, 
              validation_data=[[test_set.features, test_set.cont_ids], conv_test_cat_targets_nearest])
preds = model.predict([test_set.features, test_set.cont_ids])
maybe_correct_cat_crps(preds, test_conv_targets, conv_bin_edges)
for i in np.random.randint(0, 182218, 10):
    print(i)
    print(test_set.targets[i], test_set.features[i, 0] * test_set.scale_factors[0])
    plt.bar(conv_bin_edges[:-1] + 0.25, preds[i], width=0.5, alpha=0.8)
    plt.bar(conv_bin_edges[:-1] + 0.25, conv_test_cat_targets_nearest[i], width=0.5, zorder=0.1)
    plt.axvline(test_conv_targets[i], c='r')
    plt.show()
Explanation: With relative bins
End of explanation
train_set_full_ens, test_set_full_ens = get_train_test_sets(DATA_DIR, train_dates, test_dates,
                                                            full_ensemble_t=True)
model = build_cat_emb_model(100, emb_size, max_id, 50, conv_bin_edges.shape[0]-1)
model.compile(optimizer=Adam(0.001), loss=approx_crps_cat(bin_width))
model.fit([train_set_full_ens.features, train_set_full_ens.cont_ids], conv_train_cat_targets_nearest, 
              epochs=3, batch_size=1024, 
              validation_data=[[test_set_full_ens.features, test_set_full_ens.cont_ids], 
                               conv_test_cat_targets_nearest])
preds = model.predict([test_set_full_ens.features, test_set_full_ens.cont_ids])
maybe_correct_cat_crps(preds, test_conv_targets, conv_bin_edges)
for i in np.random.randint(0, 182218, 10):
    plt.bar(conv_bin_edges[:-1] + 0.25, preds[i], width=0.5, alpha=0.8)
    plt.bar(conv_bin_edges[:-1] + 0.25, conv_test_cat_targets_nearest[i], width=0.5, zorder=0.1)
    plt.axvline(test_conv_targets[i], c='r')
    plt.show()
Explanation: With full ensemble input
End of explanation
model = build_cat_emb_model(100, emb_size, max_id, 2, bin_edges.shape[0]-1)
model.compile(optimizer=Adam(0.00001), loss=approx_crps)
model.fit([train_set.features, train_set.cont_ids], train_cat_targets_nearest, 
              epochs=10, batch_size=4096, 
              validation_data=[[test_set.features, test_set.cont_ids], test_cat_targets_nearest])
preds = model.predict([test_set.features, test_set.cont_ids])
maybe_correct_cat_crps(preds, test_set.targets, bin_edges)
i = 10
plt.bar(bin_edges[:-1] + 0.25, preds[i], width=0.5, alpha=0.8)
plt.bar(bin_edges[:-1] + 0.25, test_cat_targets[i], width=0.5, zorder=0.1)
plt.axvline(test_set.targets[i], c='r')
plt.show()
Explanation: Standard bins
End of explanation
from keras.layers import Conv1D
def build_cat_emb_conv_model(hidden_nodes, emb_size, max_id, feature_size, n_bins, maps):
    features_in = Input(shape=(feature_size, 1))
    id_in = Input(shape=(1,))
    emb = Embedding(max_id + 1, emb_size)(id_in)
    emb = Flatten()(emb)
    conv = Conv1D(maps, feature_size)(features_in)
    conv = Flatten()(conv)
    x = Concatenate()([conv, emb])
    x = Dense(hidden_nodes, activation='relu')(x)
    x = Dense(n_bins, activation='softmax')(x)
    model = Model(inputs=[features_in, id_in], outputs=x)
    return model
model = build_cat_emb_conv_model(100, emb_size, max_id, 50, bin_edges.shape[0]-1, 2)
model.summary()
model.compile(optimizer=Adam(0.0001), loss=approx_crps)
model.fit([np.atleast_3d(train_set_full_ens.features), train_set_full_ens.cont_ids], 
          train_cat_targets_nearest, 
          epochs=10, batch_size=4096, 
          validation_data=[[np.atleast_3d(test_set_full_ens.features), test_set_full_ens.cont_ids], 
                           test_cat_targets_nearest])
Explanation: testing a convolution layer
End of explanation
aux_dict = OrderedDict()
aux_dict['data_aux_geo_interpolated.nc'] = ['orog', 
                                            'station_alt', 
                                            'station_lat', 
                                            'station_lon']
aux_dict['data_aux_pl500_interpolated_00UTC.nc'] = ['u_pl500_fc',
                                                    'v_pl500_fc',
                                                    'gh_pl500_fc']
aux_dict['data_aux_pl850_interpolated_00UTC.nc'] = ['u_pl850_fc',
                                                    'v_pl850_fc',
                                                    'q_pl850_fc']
aux_dict['data_aux_surface_interpolated_00UTC.nc'] = ['cape_fc',
                                                      'sp_fc',
                                                      'tcc_fc']
train_set_aux, test_set_aux = get_train_test_sets(DATA_DIR, train_dates, test_dates,
                                                  aux_dict=aux_dict)
model = build_cat_model(train_set_aux.features.shape[1], [50], bin_edges.shape[0]-1)
model.summary()
model.compile(optimizer=Adam(0.0001), loss=approx_crps)
model.fit(train_set_aux.features, train_cat_targets_nearest, epochs=5, batch_size=1024,
          validation_data=[test_set_aux.features, test_cat_targets_nearest])
model = build_cat_emb_model(100, emb_size, max_id, train_set_aux.features.shape[1], 
                            bin_edges.shape[0]-1)
model.summary()
model.compile(optimizer=Adam(0.00001), loss=approx_crps)
model.fit([train_set_aux.features, train_set_aux.cont_ids], train_cat_targets_nearest, 
              epochs=10, batch_size=4096, 
              validation_data=[[test_set_aux.features, test_set_aux.cont_ids], 
                               test_cat_targets_nearest])
Explanation: Auxiliary data
Mean and std
End of explanation
train_set_aux_ens, test_set_aux_ens = get_train_test_sets(DATA_DIR, train_dates, test_dates,
                                                          aux_dict=aux_dict, 
                                                          full_ensemble_t=True)
train_set_aux_ens.features.shape
model = build_cat_emb_model(100, emb_size, max_id, train_set_aux_ens.features.shape[1], 
                            bin_edges.shape[0]-1)
model.compile(optimizer=Adam(0.0001), loss=approx_crps)
model.fit([train_set_aux_ens.features, train_set_aux_ens.cont_ids], train_cat_targets_nearest, 
              epochs=10, batch_size=4096, 
              validation_data=[[test_set_aux_ens.features, test_set_aux_ens.cont_ids], 
                               test_cat_targets_nearest])
preds = model.predict([test_set_aux_ens.features, test_set_aux_ens.cont_ids])
maybe_correct_cat_crps(preds, test_set_aux_ens.targets, bin_edges)
Explanation: Full ensemble
End of explanation
import xgboost as xgb
def convert_targets_nearest_idxs(targets, bin_edges):
    mat_bin_edges = np.repeat(np.atleast_2d(bin_edges), targets.shape[0], axis=0).T
    return np.argmin(np.abs(mat_bin_edges - targets),axis=0)
xgb_train_targets = convert_targets_nearest_idxs(train_conv_targets, conv_bin_edges)
xgb_test_targets = convert_targets_nearest_idxs(test_conv_targets, conv_bin_edges)
xgb_train_targets[:5]
dtrain = xgb.DMatrix(train_set.features, xgb_train_targets, 
                     feature_names=train_set.feature_names)
dtest = xgb.DMatrix(test_set.features, xgb_test_targets, 
                     feature_names=test_set.feature_names)
our_params={
            'eta':0.2,
            'subsample':0.8,
            'objective':'multi:softprob',
            'eval_metric': 'mlogloss',
            'num_class': conv_bin_edges.shape[0]-1
            }
watchlist  = [(dtest,'eval'), (dtrain,'train')]
bst=xgb.train(our_params, dtrain, 15, watchlist)
preds_xgb = bst.predict(dtest)
maybe_correct_cat_crps(preds_xgb, test_conv_targets, conv_bin_edges)
for i in np.random.randint(0, 182218, 10):
    print(i)
    print(test_set.targets[i], test_set.features[i, 0] * test_set.scale_factors[0])
    plt.bar(conv_bin_edges[:-1] + 0.25, preds_xgb[i], width=0.5, alpha=0.8, color='navy')
    plt.bar(conv_bin_edges[:-1] + 0.25, preds[i], width=0.5, alpha=0.8, color='lightblue')
    plt.bar(conv_bin_edges[:-1] + 0.25, conv_test_cat_targets_nearest[i], width=0.5, zorder=0.1)
    plt.axvline(test_conv_targets[i], c='r')
    plt.show()
Explanation: XGBoost
End of explanation
<END_TASK>