<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Practical introduction to convolutional neural networks (CNNs) in TensorFlow
Hi guys! My name Robert. I'm currently a student at OTH Regensburg. Due to a 2 months project work I had my first hands on experience with python, TensorFlow and convolutional neural networks. This notebook and the code that goes with it are the results of that work. This Notebook is supposed to help sutdents getting started with TensorFlow and give some insights into CNNs.
What are we going to do?
<> Build a CNN for traffic sign classification with ~ 99.1 % accuracy based on the GTSRB
<> Use TensorFlow and Python to do all the work
<> Get a feeling on how CNNs work
<> What to keep in mind when working with a CNN
Requirements
<> TensorFlow with GPU support (and a supported GPU!) 
<> 16 GB RAM (since we will do all the stuff quick and dirty ;)
<> skimage and OpenCV for python 3.5.x
1) Dataset
CNNs are a pretty good technology for image classification tasks. To have a more real life like feeling (looking at you MNIST) we are going to build a traffic sign classifier based on the German Traffic Sign Recognition Benchmark (GTSRB) dataset. You can get the raw - data here.
Lets have a look at what we are dealing with
Step1:  With this we have expanded our dataset by factor 8. Let's look at the distribution of datapoints over all classes
Step2:  All preprocessing steps can be found in [ xyz .py ].
Building the TensorFlow graph
At the beginnig I promised a final accuracy of ~99.1%. For this we are going to use a network with 10 Layers and a ResNet-like extension. But be !! WARNED !! training that one on a Nvidia GTX 1050Ti takes around 50 hours. The final network can be found in [ xyz .py ].
For now we will just go with the general structure
Step3:  To acutally see something you need to add some lines while running your session
<ASSISTANT_TASK:>
Python Code:
## Augumenting functions based on Naoki Shibuya's work! Thank you!
## https://github.com/naokishibuya/car-traffic-sign-classification
import cv2
import numpy as np
def resizeImage(image):
    return cv2.resize(img, (48,48))
def random_brightness(image, ratio):
    hsv = cv2.cvtColor(image, cv2.COLOR_RGB2HSV)
    brightness = np.float64(hsv[:, :, 2])
    brightness = brightness * (1.0 + np.random.uniform(-ratio, ratio))
    brightness[brightness>255] = 255
    brightness[brightness<0] = 0
    hsv[:, :, 2] = brightness
    return cv2.cvtColor(hsv, cv2.COLOR_HSV2RGB)
def random_rotation(image, angle):
    if angle == 0:
        return image
    angle = np.random.uniform(-angle, angle)
    rows, cols = image.shape[:2]
    size = cols, rows
    center = cols/2, rows/2
    scale = 1.0
    rotation = cv2.getRotationMatrix2D(center, angle, scale)
    return cv2.warpAffine(image, rotation, size)
def random_translation(image, translation):
    if translation == 0:
        return 0
    rows, cols = image.shape[:2]
    size = cols, rows
    x = np.random.uniform(-translation, translation)
    y = np.random.uniform(-translation, translation)
    trans = np.float32([[1,0,x],[0,1,y]])
    return cv2.warpAffine(image, trans, size)
def random_shear(image, shear):
    if shear == 0:
        return image
    rows, cols = image.shape[:2]
    size = cols, rows
    left, right, top, bottom = shear, cols - shear, shear, rows - shear
    dx = np.random.uniform(-shear, shear)
    dy = np.random.uniform(-shear, shear)
    p1 = np.float32([[left   , top],[right   , top   ],[left, bottom]])
    p2 = np.float32([[left+dx, top],[right+dx, top+dy],[left, bottom+dy]])
    move = cv2.getAffineTransform(p1,p2)
    return cv2.warpAffine(image, move, size)
def augment_image(image, brightness, angle, translation, shear):
    image = resizeImage(image)
    aug_images = []
    aug_images.append(image)
    aug_images.append(random_brightness(image, brightness))
    aug_images.append(random_rotation(image, angle))
    aug_images.append(random_translation(image, translation))
    aug_images.append(random_shear(image, shear))
    aug_images.append(random_rotation(random_brightness(image, brightness), angle))
    aug_images.append(random_translation(random_brightness(image, brightness), translation))
    aug_images.append(random_shear(random_brightness(image, brightness), shear))
    return aug_images
Explanation: Practical introduction to convolutional neural networks (CNNs) in TensorFlow
Hi guys! My name Robert. I'm currently a student at OTH Regensburg. Due to a 2 months project work I had my first hands on experience with python, TensorFlow and convolutional neural networks. This notebook and the code that goes with it are the results of that work. This Notebook is supposed to help sutdents getting started with TensorFlow and give some insights into CNNs.
What are we going to do?
<> Build a CNN for traffic sign classification with ~ 99.1 % accuracy based on the GTSRB
<> Use TensorFlow and Python to do all the work
<> Get a feeling on how CNNs work
<> What to keep in mind when working with a CNN
Requirements
<> TensorFlow with GPU support (and a supported GPU!) 
<> 16 GB RAM (since we will do all the stuff quick and dirty ;)
<> skimage and OpenCV for python 3.5.x
1) Dataset
CNNs are a pretty good technology for image classification tasks. To have a more real life like feeling (looking at you MNIST) we are going to build a traffic sign classifier based on the German Traffic Sign Recognition Benchmark (GTSRB) dataset. You can get the raw - data here.
Lets have a look at what we are dealing with:
<img src="files/Resources/Raw_images_every_class.png">
Images are in between 15x15 and 250x250 pixel with varying aspect ratios. Mean height and width are ~50.1 pixel. The trainig set has a moderate size with around 32.000 samples. It's not too bad but we should augument it a bit. We are also going to fix the aspect ratio problem first by resizing to 48x48 pixel.
End of explanation
import random
## our expanded dataset is stored in
enh_images, enh_labels = (...)
#since we need to know the start and end of the current class
lower_bound = 0
upper_bound = len(enh_images)
eq_images = []
eq_labels = []
#assumtion: class indizes are sorted and increasing
for classindex in range(43):
	for position in range(lower_bound, upper_bound):
		if enh_labels[position] != classindex:
			upper_bound	= position
			break
	sample_index= random.sample(range(lower_bound, upper_bound), 200)  
	for i in sample_index:
		eq_images.append(enh_images[i][:])
	for i in sample_index:
		eq_labels.append(enh_labels[i])
	cnt = 0
	for i in sample_index:
		enh_images.pop(i - cnt)
		enh_labels.pop(i - cnt)
		cnt +=1
        
	lower_bound = upper_bound
	upper_bound = len(enh_images)
Explanation: With this we have expanded our dataset by factor 8. Let's look at the distribution of datapoints over all classes:
<img src="files/Resources/Distribution_enh_skewed.png">
Working with a skewed dataset like this isn't a big problem. But since we want to be sure not to learn the distribution we are going to split our set in a skewed and an equalized set with 200 random samples per class. We will use the equalized set for a second training.
End of explanation
import skimage 
def variable_summaries(var, summary_name):
	with tf.name_scope("Summary_" + str(summary_name)):
		tf.summary.scalar("raw", var)
		tf.summary.histogram('histogram', var)
        
def convertToUINT8(x):
	return skimage.img_as_ubyte(x / np.amax(x))
## Based on panmari's post. Thank you!
## https://gist.github.com/panmari/4622b78ce21e44e2d69c
def VisualizeConvolutions(myTensor, sizeInfo,  name):
	V = tf.slice(myTensor, (0, 0, 0, 0), (1, -1, -1, -1), name='slice_' + name)
	V = tf.reshape(V, (sizeInfo[0], sizeInfo[1], sizeInfo[2]))
	V = tf.transpose(V, (2, 0, 1))
	V = tf.reshape(V, (-1, sizeInfo[0], sizeInfo[1], 1))
	tf.summary.image(name, V,  max_outputs=3)
    
## Add this in your graph definition to use TensorBoard's "IMAGE" Tab
## This will print the first 32 feature maps of each layer after the convolution
## as well as the original input image
with tf.name_scope('Visualiz_convolutions'):
    img_raw = tf.slice(x_image, (0, 0, 0, 0), (1, -1, -1, -1), name='slice_raw_image')
    tf.summary.image("raw_image", img_raw,  max_outputs=32)
    ## note: (width, height, output kernels)
    ## 48px = orig. image // 24px = feature map after maxpool 
    VisualizeConvolutions(h_conv1, [48, 48, 96], "Conv1")
    VisualizeConvolutions(h_conv2, [24, 24, 96], "Conv2")
    VisualizeConvolutions(h_conv3, [12, 12, 48], "Conv3")
Explanation: All preprocessing steps can be found in [ xyz .py ].
Building the TensorFlow graph
At the beginnig I promised a final accuracy of ~99.1%. For this we are going to use a network with 10 Layers and a ResNet-like extension. But be !! WARNED !! training that one on a Nvidia GTX 1050Ti takes around 50 hours. The final network can be found in [ xyz .py ].
For now we will just go with the general structure:
| In |                      RGB 48x48                     |
|----|:--------------------------------------------------:|
| 1  | Conv 5x5, 96 / ReLU -> Local Resp. Norm. -> Max Pool 2x2 |
| 2  | Conv 3x3, 96 / ReLU -> Local Resp. Norm. -> Max Pool 2x2 |
| 3  |                 Conv 3x3, 48 / ReLU                |
| 4  |                 Conv 3x3, 48 / ReLU                |
| 5  | Conv 3x3, 48 / ReLU  -> Local Resp. Norm. -> Max Pool 2x2 |
| 6  | Conv 2x2, 48 / ReLU -> Local Resp. Norm. -> Max Pool 2x2 |
| 7  |          FC Layer 3+6, 2048 / ReLU -> Dropout         |
| 8  |           FC Layer 6, 1024 / ReLU -> Dropout          |
| 9  |          FC Layer 7+8, 768 / ReLU -> Dropout          |
| 10 |                        FC 43                       |
Since it's kinda difficult to find code examples for working with the images-tab in TensorBoard I added a few nice visualizations
End of explanation
logs_path = r'C:/....'
cross_entropy = (...)
accuracy = (...)
variable_summaries(cross_entropy, 'crossentropy')
variable_summaries(accuracy, 'TrainingAcc')	
summary_op = tf.summary.merge_all()
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    summary_writer = tf.summary.FileWriter(logs_path)
    summary_writer.add_graph(sess.graph)
    
    #Start training and logging
    for i in range(100):
        (...)
        _, summary = sess.run([train_step, summary_op], feed_dict={(...)})
        summary_writer.add_summary(summary, i)	
    
Explanation: To acutally see something you need to add some lines while running your session:
End of explanation
<END_TASK>