<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Overfitting Tutorial
Many deep learning models run the danger of overfitting on the training set. When this happens, the model fails to generalize its performance to unseen data, such as a separate validation set. Here we present a simple tutorial on how to recognize overfitting using our visualization tools, and how to apply Dropout layers to prevent overfitting.
We use a simple network of convolutional layers on the CIFAR-10 dataset, a dataset of images belonging to 10 categories. 
The code below will build the model and train on the CIFAR-10 dataset for 25 epochs (~2 minutes on Titan X GPUs), displaying both the training cost as well as the cost on the validation set.
Note
Step1:  Overfitting
You should notice that in the logs above, after around Epoch 15, the model begins to overfit. Even though the cost on the training set continues to decrease, the validation loss flattens (even increasing slightly). We can visualize these effects using the code below.
Note
Step2:  This situation illustrates the importance of plotting the validation loss (blue) in addition to the training cost (red). The training cost may mislead the user into thinking that model is continuing to perform well, but we can see from the validation loss that the model has begun to overfit.
Dropout layers
To correct overfitting, we introduce Dropout layers to the model, as shown below. Dropout layers randomly silence a subset of units for each minibatch, and are an effective means of preventing overfitting.
Step3:  We then plot the results of the training run below.
<ASSISTANT_TASK:>
Python Code:
from neon.initializers import Gaussian
from neon.optimizers import GradientDescentMomentum, Schedule
from neon.layers import Conv, Dropout, Activation, Pooling, GeneralizedCost
from neon.transforms import Rectlin, Softmax, CrossEntropyMulti, Misclassification
from neon.models import Model
from neon.data import CIFAR10
from neon.callbacks.callbacks import Callbacks
from neon.backends import gen_backend
be = gen_backend(batch_size=128, backend='gpu')
# hyperparameters
learning_rate = 0.05
weight_decay = 0.001
num_epochs = 25
print "Loading Data"
dataset = CIFAR10(path='data/', normalize=False, 
                  contrast_normalize=True, whiten=True, 
                  pad_classes=True)  # CIFAR10 has 10 classes, network has 16 outputs, so we pad some extra classes.
train_set = dataset.train_iter
valid_set = dataset.valid_iter
print "Building Model"
init_uni = Gaussian(scale=0.05)
opt_gdm = GradientDescentMomentum(learning_rate=float(learning_rate), momentum_coef=0.9,
                                  wdecay=float(weight_decay),
                                  schedule=Schedule(step_config=[200, 250, 300], change=0.1))
relu = Rectlin()
conv = dict(init=init_uni, batch_norm=False, activation=relu)
convp1 = dict(init=init_uni, batch_norm=False, activation=relu, padding=1)
convp1s2 = dict(init=init_uni, batch_norm=False, activation=relu, padding=1, strides=2)
layers = [
          Conv((3, 3, 64), **convp1),
          Conv((3, 3, 64), **convp1s2),
          Conv((3, 3, 128), **convp1),
          Conv((3, 3, 128), **convp1s2),
          Conv((3, 3, 128), **convp1),
          Conv((1, 1, 128), **conv),
          Conv((1, 1, 16), **conv),
          Pooling(8, op="avg"),
          Activation(Softmax())]
cost = GeneralizedCost(costfunc=CrossEntropyMulti())
mlp = Model(layers=layers)
# configure callbacks
callbacks = Callbacks(mlp, output_file='data.h5', eval_set=valid_set, eval_freq=1)
print "Training"
mlp.fit(train_set, optimizer=opt_gdm, num_epochs=num_epochs, cost=cost, callbacks=callbacks)
print('Misclassification error = %.1f%%' % (mlp.eval(valid_set, metric=Misclassification())*100))
Explanation: Overfitting Tutorial
Many deep learning models run the danger of overfitting on the training set. When this happens, the model fails to generalize its performance to unseen data, such as a separate validation set. Here we present a simple tutorial on how to recognize overfitting using our visualization tools, and how to apply Dropout layers to prevent overfitting.
We use a simple network of convolutional layers on the CIFAR-10 dataset, a dataset of images belonging to 10 categories. 
The code below will build the model and train on the CIFAR-10 dataset for 25 epochs (~2 minutes on Titan X GPUs), displaying both the training cost as well as the cost on the validation set.
Note: We highly recommend users run this model on Maxwell GPUs.
End of explanation
from neon.visualizations.figure import cost_fig, hist_fig, deconv_summary_page
from neon.visualizations.data import h5_cost_data, h5_hist_data, h5_deconv_data
from bokeh.plotting import output_notebook, show
cost_data = h5_cost_data('data.h5', False)
output_notebook()
show(cost_fig(cost_data, 400, 800, epoch_axis=False))
Explanation: Overfitting
You should notice that in the logs above, after around Epoch 15, the model begins to overfit. Even though the cost on the training set continues to decrease, the validation loss flattens (even increasing slightly). We can visualize these effects using the code below.
Note: The same plots can be created using our nvis command line utility (see: http://neon.nervanasys.com/docs/latest/tools.html)
End of explanation
layers = [
          Conv((3, 3, 64), **convp1),
          Conv((3, 3, 64), **convp1s2),
          Dropout(keep=.5),   # Added Dropout
          Conv((3, 3, 128), **convp1),
          Conv((3, 3, 128), **convp1s2),
          Dropout(keep=.5),   # Added Dropout
          Conv((3, 3, 128), **convp1),
          Conv((1, 1, 128), **conv),
          Conv((1, 1, 16), **conv),
          Pooling(8, op="avg"),
          Activation(Softmax())]
cost = GeneralizedCost(costfunc=CrossEntropyMulti())
mlp = Model(layers=layers)
# configure callbacks
callbacks = Callbacks(mlp, output_file='data.h5', eval_set=valid_set, eval_freq=1)
print "Training"
mlp.fit(train_set, optimizer=opt_gdm, num_epochs=num_epochs, cost=cost, callbacks=callbacks)
print('Misclassification error = %.1f%%' % (mlp.eval(valid_set, metric=Misclassification())*100))
Explanation: This situation illustrates the importance of plotting the validation loss (blue) in addition to the training cost (red). The training cost may mislead the user into thinking that model is continuing to perform well, but we can see from the validation loss that the model has begun to overfit.
Dropout layers
To correct overfitting, we introduce Dropout layers to the model, as shown below. Dropout layers randomly silence a subset of units for each minibatch, and are an effective means of preventing overfitting.
End of explanation
from neon.visualizations.figure import cost_fig, hist_fig, deconv_summary_page
from neon.visualizations.data import h5_cost_data, h5_hist_data, h5_deconv_data
from bokeh.plotting import output_notebook, show
cost_data = h5_cost_data('data.h5', False)
output_notebook()
show(cost_fig(cost_data, 400, 800, epoch_axis=False))
Explanation: We then plot the results of the training run below.
End of explanation
<END_TASK>