<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Computer vision
Step1:  Load the MNIST dataset, flatten the images, convert the class labels, and scale the data.
Step2:  I. OverFeat adaptation of AlexNet (2012)
An adaptation of the 'fast' model from AlexNet applied to MNIST-10.
Step3:  As you can imagine, training this model (even on MNIST-10) is quite time consuming. I'll run just one Epoch with 10 samples to show how it works.
Step4:  The true power of this model really comes out when it is used on a larger corpus of images, such as ILSVRC and MS COCO, with images having a larger spatial size.
II. VGG-19 Model
Now, let's load the VGG-19 model using pre-trained weights. First, we'll create a keras model as normal
Step5:  We then load the weights of the model from a file (you can download this from the course website; it is not small, coming in at about half a gigabyte). We then have to compile the model, even though we have no intention of actually training it. This is because the compilation in part sets the forward propigation code, which we will need to do predictions.
Step6:  We will also load some metadata, that gives class labels to the output
Step7:  Now lets read in an image of a lion
Step8:  And now predict the class label from the VGG-19 model
Step9:  A relatively impressive result for an out of sample image!
III. GoogLeNet - Inception Module
An implementation of the Inception module, the basic building block of GoogLeNet (2014). As with OverFeat, I don't have enough compute power here to actually traing the model, but this does serve as a nice example of how to use the graph interface in keras.
Step10:  IV. Batch Normalization
Use the Batch Normalization of
Step11:  V. Residual block - as in ResNet (2015)
An example of the residual block used in the pre-print
<ASSISTANT_TASK:>
Python Code:
%pylab inline
import copy
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from keras.datasets import mnist, cifar10
from keras.models import Sequential, Graph
from keras.layers.core import Dense, Dropout, Activation, Flatten, Reshape
from keras.optimizers import SGD, RMSprop
from keras.utils import np_utils
from keras.regularizers import l2
from keras.layers.convolutional import Convolution2D, MaxPooling2D, ZeroPadding2D, AveragePooling2D
from keras.callbacks import EarlyStopping
from keras.preprocessing.image import ImageDataGenerator
from keras.layers.normalization import BatchNormalization
from PIL import Image
Explanation: Computer vision: AlexNet, VGG-19, GoogLeNet
Import various modules that we need for this notebook.
End of explanation
(X_train, y_train), (X_test, y_test) = mnist.load_data()
X_train = X_train.reshape(X_train.shape[0], 1, 28, 28).astype('float32') / 255
X_test = X_test.reshape(X_test.shape[0], 1, 28, 28).astype('float32') / 255
Y_train = np_utils.to_categorical(y_train, 10)
Y_test = np_utils.to_categorical(y_test, 10)
Explanation: Load the MNIST dataset, flatten the images, convert the class labels, and scale the data.
End of explanation
model = Sequential()
# Layer 1
model.add(Convolution2D(96, 11, 11, input_shape = (1,28,28), border_mode='same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
# Layer 2
model.add(Convolution2D(256, 5, 5, border_mode='same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
# Layer 3
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(512, 3, 3, border_mode='same'))
model.add(Activation('relu'))
# Layer 4
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(1024, 3, 3, border_mode='same'))
model.add(Activation('relu'))
# Layer 5
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(1024, 3, 3, border_mode='same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
# Layer 6
model.add(Flatten())
model.add(Dense(3072, init='glorot_normal'))
model.add(Activation('relu'))
model.add(Dropout(0.5))
# Layer 7
model.add(Dense(4096, init='glorot_normal'))
model.add(Activation('relu'))
model.add(Dropout(0.5))
# Layer 8
model.add(Dense(10, init='glorot_normal'))
model.add(Activation('softmax'))
Explanation: I. OverFeat adaptation of AlexNet (2012)
An adaptation of the 'fast' model from AlexNet applied to MNIST-10.
End of explanation
model.compile(loss='categorical_crossentropy', optimizer=RMSprop())
model.fit(X_train[:10], Y_train[:10], batch_size=1, nb_epoch=1,
          verbose=1, show_accuracy=True)
Explanation: As you can imagine, training this model (even on MNIST-10) is quite time consuming. I'll run just one Epoch with 10 samples to show how it works.
End of explanation
model = Sequential()
model.add(ZeroPadding2D((1,1),input_shape=(3,224,224)))
model.add(Convolution2D(64, 3, 3, activation='relu'))
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(64, 3, 3, activation='relu'))
model.add(MaxPooling2D((2,2), strides=(2,2)))
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(128, 3, 3, activation='relu'))
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(128, 3, 3, activation='relu'))
model.add(MaxPooling2D((2,2), strides=(2,2)))
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(256, 3, 3, activation='relu'))
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(256, 3, 3, activation='relu'))
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(256, 3, 3, activation='relu'))
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(256, 3, 3, activation='relu'))
model.add(MaxPooling2D((2,2), strides=(2,2)))
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(512, 3, 3, activation='relu'))
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(512, 3, 3, activation='relu'))
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(512, 3, 3, activation='relu'))
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(512, 3, 3, activation='relu'))
model.add(MaxPooling2D((2,2), strides=(2,2)))
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(512, 3, 3, activation='relu'))
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(512, 3, 3, activation='relu'))
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(512, 3, 3, activation='relu'))
model.add(ZeroPadding2D((1,1)))
model.add(Convolution2D(512, 3, 3, activation='relu'))
model.add(MaxPooling2D((2,2), strides=(2,2)))
model.add(Flatten())
model.add(Dense(4096, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(4096, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1000, activation='softmax'))
Explanation: The true power of this model really comes out when it is used on a larger corpus of images, such as ILSVRC and MS COCO, with images having a larger spatial size.
II. VGG-19 Model
Now, let's load the VGG-19 model using pre-trained weights. First, we'll create a keras model as normal:
End of explanation
model.load_weights("../../../class_data/keras/vgg19_weights.h5")
sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(optimizer=sgd, loss='categorical_crossentropy')
Explanation: We then load the weights of the model from a file (you can download this from the course website; it is not small, coming in at about half a gigabyte). We then have to compile the model, even though we have no intention of actually training it. This is because the compilation in part sets the forward propigation code, which we will need to do predictions.
End of explanation
synsets = []
with open("../../../class_data/keras/synset_words.txt", "r") as f:
    synsets += f.readlines()
synsets = [x.replace("\n","") for x in synsets]
Explanation: We will also load some metadata, that gives class labels to the output:
End of explanation
im = Image.open('img/lion.jpg').resize((224, 224), Image.ANTIALIAS)
plt.figure(figsize=(4, 4))
plt.axis("off")
plt.imshow(im)
im = np.array(im).astype(np.float32)
# scale the image, according to the format used in training
im[:,:,0] -= 103.939
im[:,:,1] -= 116.779
im[:,:,2] -= 123.68
im = im.transpose((2,0,1))
im = np.expand_dims(im, axis=0)
Explanation: Now lets read in an image of a lion:
End of explanation
out = model.predict(im)
for index in np.argsort(out)[0][::-1][:10]:
    print("%01.4f - %s" % (out[0][index], synsets[index].replace("\n","")))
Explanation: And now predict the class label from the VGG-19 model:
End of explanation
model = Graph()
model.add_input(name='n00', input_shape=(1,28,28))
# layer 1
model.add_node(Convolution2D(64,1,1, activation='relu'), name='n11', input='n00')
model.add_node(Flatten(), name='n11_f', input='n11')
model.add_node(Convolution2D(96,1,1, activation='relu'), name='n12', input='n00')
model.add_node(Convolution2D(16,1,1, activation='relu'), name='n13', input='n00')
model.add_node(MaxPooling2D((3,3),strides=(2,2)), name='n14', input='n00')
# layer 2
model.add_node(Convolution2D(128,3,3, activation='relu'), name='n22', input='n12')
model.add_node(Flatten(), name='n22_f', input='n22')
model.add_node(Convolution2D(32,5,5, activation='relu'), name='n23', input='n13')
model.add_node(Flatten(), name='n23_f', input='n23')
model.add_node(Convolution2D(32,1,1, activation='relu'), name='n24', input='n14')
model.add_node(Flatten(), name='n24_f', input='n24')
# output layer
model.add_node(Dense(1024, activation='relu'), name='layer4',
               inputs=['n11_f', 'n22_f', 'n23_f', 'n24_f'], merge_mode='concat')
model.add_node(Dense(10, activation='softmax'), name='layer5', input='layer4')
model.add_output(name='output1',input='layer5')
model.compile(loss={'output1':'categorical_crossentropy'}, optimizer=RMSprop())
model.fit({'n00':X_train[:100], 'output1':Y_train[:100]}, nb_epoch=1, verbose=1)
Explanation: A relatively impressive result for an out of sample image!
III. GoogLeNet - Inception Module
An implementation of the Inception module, the basic building block of GoogLeNet (2014). As with OverFeat, I don't have enough compute power here to actually traing the model, but this does serve as a nice example of how to use the graph interface in keras.
End of explanation
model = Sequential()
model.add(Convolution2D(6, 5, 5, border_mode='valid', input_shape = (1, 28, 28)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(BatchNormalization())
model.add(Activation("relu"))
model.add(Convolution2D(16, 5, 5, border_mode='valid'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(BatchNormalization())
model.add(Activation("relu"))
model.add(Dropout(0.5))
model.add(Convolution2D(120, 1, 1, border_mode='valid'))
model.add(Flatten())
model.add(Dense(84))
model.add(Activation("relu"))
model.add(Dense(10))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy', optimizer=RMSprop())
model.fit(X_train, Y_train, batch_size=32, nb_epoch=20,
          verbose=1, show_accuracy=True, validation_data=(X_test, Y_test))
Explanation: IV. Batch Normalization
Use the Batch Normalization of: Ioffe, Sergey, and Christian Szegedy. "Batch normalization: Accelerating deep network training by reducing internal covariate shift." arXiv preprint arXiv:1502.03167 (2015). We'll re-train LeNet-5, but use relu units.
End of explanation
model = Graph()
model.add_input(name='input0', input_shape=(1,28,28))
model.add_node(Flatten(), name='input1', input='input0')
model.add_node(Dense(50),   name='input2', input='input1')
model.add_node(Dense(50, activation='relu'), name='middle1', input='input2')
model.add_node(Dense(50, activation='relu'), name='middle2', input='middle1')
model.add_node(Dense(512, activation='relu'), name='top1',
               inputs=['input2', 'middle2'], merge_mode='sum')
model.add_node(Dense(10, activation='softmax'), name='top2', input='top1')
model.add_output(name='top3',input='top2')
model.compile(loss={'top3':'categorical_crossentropy'}, optimizer=RMSprop())
model.fit({'input0':X_train, 'top3':Y_train}, nb_epoch=25, verbose=1,
          validation_data={'input0':X_test, 'top3':Y_test})
Explanation: V. Residual block - as in ResNet (2015)
An example of the residual block used in the pre-print: "Deep Residual Learning for Image Recognition." (2015).
End of explanation
<END_TASK>