<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Building and training a single-layer neural network with keras
Keras is a high-level Python interface for building and training neural networks. 
It can use Tensorflow or Theano as backend, so as to make efficient use of the CPU and (if available) GPU.
Step1:  Defining the model with keras
Keras describes a neural network by a set of Dense layers (which contain the weights) and Activation layers (which contain the non-linear functions).
In the case of our simple single-layer network, the API for building the network is the following.
Step2:  Training the network
Again, we will load the training data.
Step3:  and define a function to look at the predictions of the model (which for the moment is untrained).
Step4:  Keras will then automatically adjust the weights by trying to minimize a given loss function.
When the network output is a probability, a good loss function is the binary cross-entropy
Step5:  What just happened?
Just like we did by hand, keras tried to find the best set of weights for the network.
In order to do so, Keras divided the dataset into batches of 16 examples (16 specimens of iris at a time). 
For each batch, it computed how the weights should be changed in order to make the loss function lower, and correspondingly ajusted them by a small amount (which is proportional to the learning rate lr).
The process of going through the whole dataset (batch by batch) is called an epoch. The network needs to "see" the dataset several times (i.e. it needs several epochs) in order to find the right weights.
A few remarks on training
The training depends on a number of parameters, that have to be "skillfully" chosen by the user
<ASSISTANT_TASK:>
Python Code:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
# Keras imports
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.optimizers import SGD
Explanation: Building and training a single-layer neural network with keras
Keras is a high-level Python interface for building and training neural networks. 
It can use Tensorflow or Theano as backend, so as to make efficient use of the CPU and (if available) GPU.
End of explanation
# Build the model with keras
model = Sequential()
model.add( Dense( output_dim=1, input_dim=2 ) )
model.add( Activation( 'sigmoid' ) )
# Print the summary
model.summary()
Explanation: Defining the model with keras
Keras describes a neural network by a set of Dense layers (which contain the weights) and Activation layers (which contain the non-linear functions).
In the case of our simple single-layer network, the API for building the network is the following.
End of explanation
# Load data
df = pd.read_csv('./data/setosa/train.csv')
X = df[['petal length (cm)', 'petal width (cm)']].values
y = df['setosa'].values
Explanation: Training the network
Again, we will load the training data.
End of explanation
def plot_keras_model():
    "Plot the results of the model, along with the data points"
    # Calculate the probability on a mesh
    petal_width_mesh, petal_length_mesh = \
        np.meshgrid( np.linspace(0,3,100), np.linspace(0,8,100) )
    petal_width_mesh = petal_width_mesh.flatten()
    petal_length_mesh = petal_length_mesh.flatten()
    p = model.predict( np.stack( (petal_length_mesh, petal_width_mesh), axis=1 ) )
    p = p.reshape((100,100))
    # Plot the probability on the mesh
    plt.clf()
    plt.imshow( p.T, extent=[0,8,0,3], origin='lower', 
               vmin=0, vmax=1, cmap='RdBu', aspect='auto', alpha=0.7 )
    # Plot the data points
    plt.scatter( df['petal length (cm)'], df['petal width (cm)'], c=df['setosa'], cmap='RdBu')
    plt.xlabel('petal length (cm)')
    plt.ylabel('petal width (cm)')
    cb = plt.colorbar()
    cb.set_label('setosa')
plot_keras_model()
Explanation: and define a function to look at the predictions of the model (which for the moment is untrained).
End of explanation
# Prepare the model for training
model.compile(loss='binary_crossentropy', optimizer=SGD(lr=0.1), metrics=['accuracy'])
# Train the network
model.fit( X, y, batch_size=16, nb_epoch=20, verbose=1 )
plot_keras_model()
Explanation: Keras will then automatically adjust the weights by trying to minimize a given loss function.
When the network output is a probability, a good loss function is the binary cross-entropy:
$ L(p_{setosa}) = - y\log(p_{setosa}) - (1-y) \log(1-p_{setosa}) $
End of explanation
df_test = pd.read_csv('./data/setosa/test.csv')
df_test.head(10)
model.predict( np.array([[4.2, 1.5]]) )
df_test['probability_setosa_predicted'] = model.predict( df_test[['petal length (cm)', 'petal width (cm)']].values )
df_test
Explanation: What just happened?
Just like we did by hand, keras tried to find the best set of weights for the network.
In order to do so, Keras divided the dataset into batches of 16 examples (16 specimens of iris at a time). 
For each batch, it computed how the weights should be changed in order to make the loss function lower, and correspondingly ajusted them by a small amount (which is proportional to the learning rate lr).
The process of going through the whole dataset (batch by batch) is called an epoch. The network needs to "see" the dataset several times (i.e. it needs several epochs) in order to find the right weights.
A few remarks on training
The training depends on a number of parameters, that have to be "skillfully" chosen by the user:
- loss function
- optimizer and learning rate
- batch size
- number of epochs
A "bad" choice may result in the network training more slowly, or not properly training at all.
The training is stochastic.
This is because:
- The initial values of the weights are chosen randomly.
- The way the data is divided into batch is random.
Performing predictions on the test sets
The optimal weights are stored in the model object. As before, we can thus use them to make new predictions on the test dataset.
End of explanation
<END_TASK>