<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 This notebook investigates alexnet-based models with 8-augmentation, an extra convolutional layer with 192 output channels and various dropout values at each layer. 
When training the models, we saved both files from the best and most recent epoch.
Let's see what score we got with the best parameter values from the models. We first look at the model specified in  alexnet_extra_layer_dropouts.json model, which has 0.75 dropout on all but last convolutional layers, and 0.5 dropout on the last layer.
Step1:  Now check the model specified in alexnet_extra_layer_dropouts2.json model, which has 0.9 dropout on all but last convolutional layers, and 0.5 dropout on the last layer.
Step2:  Finally, check the model specified in alexnet_extra_layer_dropouts3.json model, which has 0.5 dropout on all convolutional layers.
Step3:  Looks like the alexnet_extra_layer_dropouts2.json model gave the best score. It also still continues to improve. Let's look at the evolution of nll
Step4:  The graph looks sort of stable for valid_y_nll. Perhaps there's not that much room for improvement? It could be useful to let the model run for around 85 epochs, which is roughly the time the current best model ran for. The #3 model was stopped, and #1 and #2 are let to run until ~epoch 85.
After ~85 epochs, the score is not better than the current best.
<ASSISTANT_TASK:>
Python Code:
import pylearn2.utils
import pylearn2.config
import theano
import neukrill_net.dense_dataset
import neukrill_net.utils
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
import holoviews as hl
%load_ext holoviews.ipython
import sklearn.metrics
cd ..
m = pylearn2.utils.serial.load("/disk/scratch/neuroglycerin/models/alexnet_extra_layer_dropouts2.pkl.recent")
%run check_test_score.py -v run_settings/alexnet_extra_layer_dropouts.json
Explanation: This notebook investigates alexnet-based models with 8-augmentation, an extra convolutional layer with 192 output channels and various dropout values at each layer. 
When training the models, we saved both files from the best and most recent epoch.
Let's see what score we got with the best parameter values from the models. We first look at the model specified in  alexnet_extra_layer_dropouts.json model, which has 0.75 dropout on all but last convolutional layers, and 0.5 dropout on the last layer.
End of explanation
%run check_test_score.py -v run_settings/alexnet_extra_layer_dropouts2.json
Explanation: Now check the model specified in alexnet_extra_layer_dropouts2.json model, which has 0.9 dropout on all but last convolutional layers, and 0.5 dropout on the last layer.
End of explanation
%run check_test_score.py -v run_settings/alexnet_extra_layer_dropouts3.json
Explanation: Finally, check the model specified in alexnet_extra_layer_dropouts3.json model, which has 0.5 dropout on all convolutional layers.
End of explanation
def plot_monitor(model,c = 'valid_y_nll'):
    channel = model.monitor.channels[c]
    plt.title(c)
    plt.grid(which="both")
    plt.plot(channel.example_record,channel.val_record)
    return None
plot_monitor(m)
plot_monitor(m,c="train_y_nll")
Explanation: Looks like the alexnet_extra_layer_dropouts2.json model gave the best score. It also still continues to improve. Let's look at the evolution of nll:
End of explanation
%run check_test_score.py -v run_settings/alexnet_extra_layer_dropouts2.json
Explanation: The graph looks sort of stable for valid_y_nll. Perhaps there's not that much room for improvement? It could be useful to let the model run for around 85 epochs, which is roughly the time the current best model ran for. The #3 model was stopped, and #1 and #2 are let to run until ~epoch 85.
After ~85 epochs, the score is not better than the current best.
End of explanation
<END_TASK>