<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Introduction
In this file we will try out open-source libraries for deep learning and CNN.
Data
Let's first prepare the data from CIFAR10 dataset.
Source
Step1:  Libraries available in python
Theano - the granddaddy. Have a few libraries built on top
Keras
Lasange
Tensorflow - the new hotshot from google 
Keras (can be used on top)
Caffe
Keras
Sits on top of either Theano or Tensorflow.
Sacrifice performance for ease-of-use.
Be very careful to have compatible .keras config for image_dim_ordering and backend. Else you will get an unclear error message "Flatten() - input_shape not fully defined"
{
    "image_dim_ordering"
Step2:  Theano
Installation Note
If g++ is not installed, theano will use the python implementation instead of optimized C++. 
On windows, in order to avoid all the weird errors, do try to install from the bleeding edge (dev) version.
Step3:  Tensorflow
The codes are mostly from the following tutorial. Note
<ASSISTANT_TASK:>
Python Code:
import pandas as pd
import numpy as np
from random import randint
from pandas import Series,DataFrame
import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib
import math
import time
%matplotlib inline
plt.rcParams['figure.figsize'] = (8, 6)
plt.rcParams['font.size'] = 14
def unpickle(file):
    import cPickle
    fo = open(file, 'rb')
    dict = cPickle.load(fo)
    fo.close()
    return dict
def array_to_image(arr):
    im = arr.reshape((32,32,3),order='F')/255.0
    im = np.transpose(im,axes=(1,0,2))
    return im
# source: http://www.cs.utoronto.ca/~kriz/cifar.html
# get the training data, batch 1 out of 6. do unzip the tar.gz first
from time import strftime
import os
if os.name == 'nt': # Windows
    path = 'cifar\\data_batch_1'
else:
    path = 'cifar/data_batch_1'
    
cifar = unpickle('cifar/data_batch_1')
# image data is uint8 array format 
# 3 RGB channels and 32x32 pixels means 32x32x3 = 1024x3 = 3072 numbers in one array
# 10,000 arrays in dataset
lucky_num = randint(0,10000)
sample_arr = cifar["data"][lucky_num].copy()
print 'array_size = {}'.format(len(sample_arr))
print sample_arr
print 'file_name = {}'.format(cifar["filenames"][lucky_num])
#show image
plt.rcParams['figure.figsize'] = (4, 4)
plt.imshow(array_to_image(sample_arr))
plt.show()
# X Format is [image_index, RGB, Height, Width]
X = cifar["data"].copy()
X = X.reshape((-1,3,32,32),order='C')/255.0 - 0.5 #standardize
#set size of input, features, hidden, target
instance_size = X.shape[0]
feature_size = X.shape[1]*X.shape[2]*X.shape[3]
target_size = 10
kernel_size = (12,3,3,3)
weight_size = (10,12,16,16) #softmax layer
#make a flat 10 output with all zeros
Y = np.zeros((instance_size,10))
for j in range(0,instance_size):
    Y[j][cifar['labels'][j]] = 1
#split train and test dataset
train_split = 0.8 #6000.0/instance_size
train_size = 200 #int(train_split*instance_size)
test_size = 40 #instance_size - train_size
index = np.random.permutation(instance_size)
train_ix, test_ix = index[:train_size], index[train_size:train_size+test_size]
Y_train , Y_test = Y[train_ix,:], Y[test_ix,:]
X_train , X_test = X[train_ix,:,:,:], X[test_ix,:,:,:]
Explanation: Introduction
In this file we will try out open-source libraries for deep learning and CNN.
Data
Let's first prepare the data from CIFAR10 dataset.
Source: http://www.cs.utoronto.ca/~kriz/cifar.html
The CIFAR-10 dataset consists of 60,000 32x32 color images in 10 classes, with 6,000 images per class. 
(we will be using batch_1 only out of 6 batches -- so 10,000 images in total)
The classes are ["airplane","automobile","bird","cat","deer","dog","frog","horse","ship","truck"]
| no. of Classes    | 10            |
|-------------------|---------------|
| Samples per class | ~1,000        |
| Samples total     | 10,000        |
| Dimensionality    | 32x32x3 = 3,072|
| Features          | int 0-255 each pixel     |
End of explanation
import os
os.environ["THEANO_FLAGS"] = "mode=FAST_RUN,device=cpu,floatX=float32"
import theano
import keras
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.layers import Convolution2D, MaxPooling2D
from keras.optimizers import SGD
model = Sequential()
# input: 32x32 images with 3 channels -> (3, 32, 32) tensors.
# this applies 32 convolution filters of size 3x3 each.
model.add(Convolution2D(nb_filter=12, nb_row=3, nb_col=3, border_mode='valid', \
                        batch_input_shape=(None, 3, 32, 32), bias=True))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
# Note: Keras does automatic shape inference.
model.add(Dense(output_dim=10, bias=True))
model.add(Activation('softmax'))
sgd = SGD(lr=0.02, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd)
model.fit(X_train, Y_train, batch_size=min(100,int(0.2*train_size)), nb_epoch=10)
model.save('keras_model.hdf5')
import pydot
from keras.utils.visualize_util import plot as kr_plot
from IPython.display import Image
kr_plot(model, to_file='model.png',show_shapes=True)
Image('model.png',width=380,height=700)
y_prob = model.predict(X_test, batch_size=10)
from sklearn.metrics import confusion_matrix
def show_confusion_matrix(cm_mat):
    accuracy = np.trace(cm_mat)*100.0/test_size
    print 'Test set Accuracy = {:.2f}%'.format(accuracy)
    df_cm = pd.DataFrame(cm_mat,index=label_textno[0:cm_mat.shape[0]],columns=label_textno[0:cm_mat.shape[1]])
    plt.figure(figsize = (8,6),dpi=300)
    sns.heatmap(df_cm, cbar=True ,annot=True, fmt=',.0f')
    plt.title('Confusion Matrix')
    plt.xlabel('Truth')
    plt.ylabel('Predicted')
#get the prediction to compare with target
label_text = ["plane","car","bird","cat","deer","dog","frog","horse","ship","truck"]
label_textno = label_text
for l in range(0,len(label_text)):
    label_textno[l] = str(l) + ' ' + label_text[l]
    
y_pred = [np.argmax(r) for r in y_prob]
y_truth = np.array(cifar['labels'])[test_ix]
cm_mat = confusion_matrix(y_truth,y_pred)
show_confusion_matrix(cm_mat)
Explanation: Libraries available in python
Theano - the granddaddy. Have a few libraries built on top
Keras
Lasange
Tensorflow - the new hotshot from google 
Keras (can be used on top)
Caffe
Keras
Sits on top of either Theano or Tensorflow.
Sacrifice performance for ease-of-use.
Be very careful to have compatible .keras config for image_dim_ordering and backend. Else you will get an unclear error message "Flatten() - input_shape not fully defined"
{
    "image_dim_ordering": "th", 
    "epsilon": 1e-07,
    "floatx": "float32",
    "backend": "theano"
}
End of explanation
# Note: 
# if g++ is not installed, theano will use the python implementation instead of optimized C++. 
# on windows, do try 
import theano
import theano.tensor as T
#Theano hello world - adding 2 matrices
x = T.dmatrix('x')
y = T.dmatrix('y')
z = x + y
f = theano.function([x, y], z)
print f([[0,1],[2,3]],[[0.5,2.5],[-1.5,-2.7]])
#Theano hello world - compute any symboling expression with multiple outputs
a, b = T.dmatrices('a', 'b')
diff = a - b
abs_diff = abs(diff)
diff_squared = diff**2
f = theano.function([a, b], [diff, abs_diff, diff_squared])
print f([[1,2],[3,4]],[[5,6],[7,8]])
Explanation: Theano
Installation Note
If g++ is not installed, theano will use the python implementation instead of optimized C++. 
On windows, in order to avoid all the weird errors, do try to install from the bleeding edge (dev) version.
End of explanation
import tensorflow as tf
Explanation: Tensorflow
The codes are mostly from the following tutorial. Note: as of 2016-09-28, Tensorflow works on OSX and Linux only. Still waiting for Windows version...
End of explanation
<END_TASK>