<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Introduction to Neural Networks
How manual coding works
Opposing basic Idea of Supervised Machine Learning
Hope
Step1:  One of the  Classics
Step2:  https
Step3:  This is the output of all 3 hidden neurons, but what we really want is a category for iris category
Softmax activation turns each output to a percantage between 0 and 1 all adding up to 1
interpretation is likelyhood of category
Step4:  <h1 style="color
Step5:  Evaluation
Step6:  Save Model in Keras Format
<ASSISTANT_TASK:>
Python Code:
import warnings
warnings.filterwarnings('ignore')
%matplotlib inline
%pylab inline
import matplotlib.pylab as plt
import numpy as np
from distutils.version import StrictVersion
import sklearn
print(sklearn.__version__)
assert StrictVersion(sklearn.__version__ ) >= StrictVersion('0.18.1')
import tensorflow as tf
tf.logging.set_verbosity(tf.logging.ERROR)
print(tf.__version__)
assert StrictVersion(tf.__version__) >= StrictVersion('1.1.0')
import keras
print(keras.__version__)
assert StrictVersion(keras.__version__) >= StrictVersion('2.0.0')
import pandas as pd
print(pd.__version__)
assert StrictVersion(pd.__version__) >= StrictVersion('0.20.0')
Explanation: Introduction to Neural Networks
How manual coding works
Opposing basic Idea of Supervised Machine Learning
Hope: System can generalize to previously unknown data and situations
Common Use Case: Classification
Types of Machine Learning
AI vs Machine Learning (ML)
NVIDIA Blog: What’s the Difference Between Artificial Intelligence, Machine Learning, and Deep Learning?
End of explanation
from sklearn.datasets import load_iris
iris = load_iris()
iris.data[0]
print(iris.DESCR)
import matplotlib.pyplot as plt
from matplotlib.colors import ListedColormap
iris_df = pd.DataFrame(iris.data, columns=iris.feature_names)
CMAP = ListedColormap(['#FF0000', '#00FF00', '#0000FF'])
pd.plotting.scatter_matrix(iris_df, c=iris.target, edgecolor='black', figsize=(15, 15), cmap=CMAP)
plt.show()
Explanation: One of the  Classics: Classify Iris Type by sizes of their flower
Solving Iris with Neural Networks
First we load the data set and get an impression
https://en.wikipedia.org/wiki/Iris_flower_data_set
End of explanation
# keras.layers.Input?
from keras.layers import Input
inputs = Input(shape=(4, ))
# keras.layers.Dense?
from keras.layers import Dense
# just linear activation (like no activation function at all)
fc = Dense(3)(inputs)
from keras.models import Model
model = Model(input=inputs, output=fc)
model.summary()
model.compile(optimizer='adam',
              loss='categorical_crossentropy',
              metrics=['accuracy'])
# this is just random stuff, no training has taken place so far
model.predict(np.array([[ 5.1,  3.5,  1.4,  0.2]]))
Explanation: https://en.wikipedia.org/wiki/File:Petal-sepal.jpg
The artificial Neuron
<h1 style="color: red">Question: What kind of equation is this? What is the graph of such a function?</h1>
The Classic: A fully connected network with a hidden layer
End of explanation
inputs = Input(shape=(4, ))
fc = Dense(10)(inputs)
predictions = Dense(3, activation='softmax')(fc)
model = Model(input=inputs, output=predictions)
model.summary()
model.compile(optimizer='adam',
              loss='categorical_crossentropy',
              metrics=['accuracy'])
model.predict(np.array([[ 5.1,  3.5,  1.4,  0.2]]))
Explanation: This is the output of all 3 hidden neurons, but what we really want is a category for iris category
Softmax activation turns each output to a percantage between 0 and 1 all adding up to 1
interpretation is likelyhood of category
End of explanation
X = np.array(iris.data)
y = np.array(iris.target)
X.shape, y.shape
y[100]
# tiny little pieces of feature engeneering
from keras.utils.np_utils import to_categorical
num_categories = 3
y = to_categorical(y, num_categories)
y[100]
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42, stratify=y)
X_train.shape, X_test.shape, y_train.shape, y_test.shape
!rm -r tf_log
tb_callback = keras.callbacks.TensorBoard(log_dir='./tf_log')
# https://keras.io/callbacks/#tensorboard
# To start tensorboard
# tensorboard --logdir=/mnt/c/Users/olive/Development/ml/tf_log
# open http://localhost:6006
%time model.fit(X_train, y_train, epochs=500, validation_split=0.2, callbacks=[tb_callback])
# %time model.fit(X_train, y_train, epochs=500, validation_split=0.2)
Explanation: <h1 style="color: red">Question: What is the minimum amount of hidden neurons to solve this categorization task?</h1>
Now we have likelyhoods for categories, but still our model is totally random
Training
training is performed using Backpropagation
each pair of ground truth input and output is passed through network
difference between expected output (ground truth) and actual result is summed up and forms loss function
loss function is to be minimized
optimizer defines strategy to minimize loss
Optimizers: Adam and RMSprop seem nice
http://cs231n.github.io/neural-networks-3/#ada
End of explanation
model.predict(np.array([[ 5.1,  3.5,  1.4,  0.2]]))
X[0], y[0]
train_loss, train_accuracy = model.evaluate(X_train, y_train)
train_loss, train_accuracy
test_loss, test_accuracy = model.evaluate(X_test, y_test)
test_loss, test_accuracy
Explanation: Evaluation
End of explanation
# Keras format
model.save('nn-iris.hdf5')
Explanation: Save Model in Keras Format
End of explanation
<END_TASK>