<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 CNN Intro
https
Step1:  Modell-Architektur
http
Step2:  Sigmoid
This is the classic
Continuous version of step function
Step3:  Relu
perfect for blacking out everyhing beyong threshold
this is just what everyone actually uses
Step4:  The classic VGG16 Architecture
Step5:  VGG starts with a number of convolutional blocks for feature extraction and ends with a fully connected classifier
Step6:  
Step7:  
Step8:  What does the CNN "see"?
Does it "see" the right thing?
Each filter output of a convolutional layer is called feature channel
with each input they should ideally either be
blank if they do not recognize any feature in the input or
encode what the feature channel "sees" in the input
feature channels directly before FC layers are often called bottleneck feature channels
Some activations from bottleneck features
Step9:  Visualizing feature channels using Quiver
Only works locally
Step10:  Modern Alternative
<ASSISTANT_TASK:>
Python Code:
import warnings
warnings.filterwarnings('ignore')
%matplotlib inline
%pylab inline
import matplotlib.pylab as plt
import numpy as np
from distutils.version import StrictVersion
import sklearn
print(sklearn.__version__)
assert StrictVersion(sklearn.__version__ ) >= StrictVersion('0.18.1')
import tensorflow as tf
tf.logging.set_verbosity(tf.logging.ERROR)
print(tf.__version__)
assert StrictVersion(tf.__version__) >= StrictVersion('1.1.0')
import keras
print(keras.__version__)
assert StrictVersion(keras.__version__) >= StrictVersion('2.0.0')
Explanation: CNN Intro
https://keras.io/applications/
http://www.image-net.org/
End of explanation
def centerAxis(uses_negative=False):
    # http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.plot
    ax = plt.gca()
    ax.spines['left'].set_position('center')
    if uses_negative:
        ax.spines['bottom'].set_position('center')
    ax.spines['right'].set_color('none')
    ax.spines['top'].set_color('none')
    ax.xaxis.set_ticks_position('bottom')
    ax.yaxis.set_ticks_position('left')
Explanation: Modell-Architektur
http://cs231n.github.io/neural-networks-1/#power
Layout of a typical CNN
http://cs231n.github.io/convolutional-networks/
Classic VGG like Architecture
we use a VGG like architecture
based on https://arxiv.org/abs/1409.1556
basic idea: sequential, deep, small convolutional filters, use dropouts to reduce overfitting
16/19 layers are typical
we choose less layers, because we have limited resources
Convolutional Blocks: Cascading many Convolutional Layers having down sampling in between
http://cs231n.github.io/convolutional-networks/#conv
Example of a Convolution
Original Image
Many convolutional filters applied over all channels
http://cs.stanford.edu/people/karpathy/convnetjs/demo/cifar10.html
Downlsampling Layer: Reduces data sizes and risk of overfitting
http://cs231n.github.io/convolutional-networks/#pool
Activation Functions
End of explanation
def np_sigmoid(X):
    return 1 / (1 + np.exp(X * -1))
x = np.arange(-10,10,0.01)
y = np_sigmoid(x)
centerAxis()
plt.plot(x,y,lw=3)
Explanation: Sigmoid
This is the classic
Continuous version of step function
End of explanation
def np_relu(x):
    return np.maximum(0, x)
x = np.arange(-10, 10, 0.01)
y = np_relu(x)
centerAxis()
plt.plot(x,y,lw=3)
Explanation: Relu
perfect for blacking out everyhing beyong threshold
this is just what everyone actually uses
End of explanation
def predict(model, img_path):
    img = image.load_img(img_path, target_size=(224, 224))
    x = image.img_to_array(img)
    x = np.expand_dims(x, axis=0)
    x = preprocess_input(x)
    preds = model.predict(x)
    # decode the results into a list of tuples (class, description, probability)
    # (one such list for each sample in the batch)
    print('Predicted:', decode_predictions(preds, top=3)[0])
from keras import applications
# applications.VGG16?
vgg16_model = applications.VGG16(weights='imagenet')
Explanation: The classic VGG16 Architecture
End of explanation
vgg16_model.summary()
!curl -O https://upload.wikimedia.org/wikipedia/commons/thumb/d/de/Beagle_Upsy.jpg/440px-Beagle_Upsy.jpg
Explanation: VGG starts with a number of convolutional blocks for feature extraction and ends with a fully connected classifier
End of explanation
predict(model = vgg16_model, img_path = '440px-Beagle_Upsy.jpg')
!curl -O https://djcordhose.github.io/ai/img/cat-bonkers.png
predict(model = vgg16_model, img_path = 'cat-bonkers.png')
!curl -O https://djcordhose.github.io/ai/img/squirrels/original/Michigan-MSU-raschka.jpg
!curl -O https://djcordhose.github.io/ai/img/squirrels/original/Black_New_York_stuy_town_squirrel_amanda_ernlund.jpeg
!curl -O https://djcordhose.github.io/ai/img/squirrels/original/london.jpg
Explanation: 
End of explanation
predict(model = vgg16_model, img_path = 'Michigan-MSU-raschka.jpg')
predict(model = vgg16_model, img_path = 'Black_New_York_stuy_town_squirrel_amanda_ernlund.jpeg')
predict(model = vgg16_model, img_path = 'london.jpg')
Explanation: 
End of explanation
# create a tmp dir in the local directory this notebook runs in, otherwise quiver will fail (and won't tell you why)
!rm -rf tmp
!mkdir tmp
Explanation: What does the CNN "see"?
Does it "see" the right thing?
Each filter output of a convolutional layer is called feature channel
with each input they should ideally either be
blank if they do not recognize any feature in the input or
encode what the feature channel "sees" in the input
feature channels directly before FC layers are often called bottleneck feature channels
Some activations from bottleneck features:
End of explanation
# https://github.com/keplr-io/quiver
# Alternative with more styles of visualization: https://github.com/raghakot/keras-vis
# https://github.com/keplr-io/quiver
from quiver_engine import server
server.launch(vgg16_model, input_folder='.', port=7000)
# open at http://localhost:7000/
# interrupt kernel to return control to notebook
Explanation: Visualizing feature channels using Quiver
Only works locally
End of explanation
from keras.applications.resnet50 import ResNet50
from keras.preprocessing import image
from keras.applications.resnet50 import preprocess_input, decode_predictions
import numpy as np
resnet_model = ResNet50(weights='imagenet')
resnet_model.summary()
predict(model = resnet_model, img_path = 'cat-bonkers.png')
predict(model = resnet_model, img_path = 'Michigan-MSU-raschka.jpg')
predict(model = resnet_model, img_path = 'Black_New_York_stuy_town_squirrel_amanda_ernlund.jpeg')
predict(model = resnet_model, img_path = 'london.jpg')
Explanation: Modern Alternative: Resnet
https://keras.io/applications/#resnet50
https://arxiv.org/abs/1512.03385
New Layer Type: https://keras.io/layers/normalization/
End of explanation
<END_TASK>