<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Quick and Dirty Introduction to Neural Networks
Fabio A. González, Universidad Nacional de Colombia
Artificial Neuron
<img src="http
Step1:  AND Neural Network
<img width=500 src="2in-neuron.jpg" align="middle">
Step2:  Question
Step3:  Plotting the NN Output
Step4:  <br/>
<br/>
Answer
Step5:  XOR NN Output Plot
Step6:  The Little Red Riding Hood Neural Network
<img src="http
Step7:  Backpropagation
Step8:  Prediction
<ASSISTANT_TASK:>
Python Code:
from pybrain.tools.shortcuts import buildNetwork
net = buildNetwork(2, 1, outclass=pybrain.SigmoidLayer)
print net.params
def print_pred2(dataset, network):
    df = pd.DataFrame(dataset.data['sample'][:dataset.getLength()],columns=['X', 'Y'])
    prediction = np.round(network.activateOnDataset(dataset),3)
    df['output'] = pd.DataFrame(prediction)
    return df
from pybrain.datasets import UnsupervisedDataSet, SupervisedDataSet
D = UnsupervisedDataSet(2) # define a dataset in pybrain
D.addSample([0,0])
D.addSample([0,1])
D.addSample([1,0])
D.addSample([1,1])
print_pred2(D, net)
Explanation: Quick and Dirty Introduction to Neural Networks
Fabio A. González, Universidad Nacional de Colombia
Artificial Neuron
<img src="http://upload.wikimedia.org/wikipedia/commons/thumb/6/60/ArtificialNeuronModel_english.png/600px-ArtificialNeuronModel_english.png" >
$$o_j^{(n)} = \varphi\left(\sum_{i\; in\; layer (n-1)}w_{ij}o_i^{(n-1)} \right)$$
Step activation function
<img src="https://c.mql5.com/2/4/act1.png" align="middle">
Logistic activation function
$$\varphi(x) = \frac{1}{1 - e^{-(x-b)}}$$
<img width= 300 src="http://upload.wikimedia.org/wikipedia/commons/thumb/b/b5/SigmoidFunction.png/400px-SigmoidFunction.png" align="middle">
Question: How to program an artificial neuron to calculate the and function?
<br>
<table>
<tr>
<th>$X$</th>
<th>$Y$</th>
<th>$X$ and $Y$</th>
</tr>
<tr>
<td>0</td>
<td>0</td>
<td style="text-align:center">0</td>
</tr>
<tr>
<td>0</td>
<td>1</td>
<td style="text-align:center">0</td>
</tr>
<tr>
<td>1</td>
<td>0</td>
<td style="text-align:center">0</td>
</tr>
<tr>
<td>1</td>
<td>1</td>
<td style="text-align:center">1</td>
</tr>
</table>
AND Neural Network
<img width=500 src="2in-neuron.jpg" align="middle">
End of explanation
net.params[:] = [0, 0,  0]
print_pred2(D, net)
Explanation: AND Neural Network
<img width=500 src="2in-neuron.jpg" align="middle">
End of explanation
def plot_nn_prediction(N): 
    # a function to plot the binary output of a network on the [0,1]x[0,1] space
    x_list = np.arange(0.0,1.0,0.025)
    y_list = np.arange(1.0,0.0,-0.025)    
    z = [0.0 if N.activate([x,y])[0] <0.5 else 1.0  for y in y_list for x in x_list]
    z = np.array(z)
    grid = z.reshape((len(x_list), len(y_list)))
    plt.imshow(grid, extent=(x_list.min(), x_list.max(), y_list.min(), y_list.max()),cmap=plt.get_cmap('Greys_r'))
    plt.show()
Explanation: Question: How to program an artificial neuron to calculate the xor function?
<br/>
<table>
<tr>
<th>$X$</th>
<th>$Y$</th>
<th>$X$ xor $Y$</th>
</tr>
<tr>
<td>0</td>
<td>0</td>
<td style="text-align:center">0</td>
</tr>
<tr>
<td>0</td>
<td>1</td>
<td style="text-align:center">1</td>
</tr>
<tr>
<td>1</td>
<td>0</td>
<td style="text-align:center">1</td>
</tr>
<tr>
<td>1</td>
<td>1</td>
<td style="text-align:center">0</td>
</tr>
</table>
Plotting the NN Output
End of explanation
net.params[:] = [-30, 20, 20]
plot_nn_prediction(net)
Explanation: Plotting the NN Output
End of explanation
Dtrain = SupervisedDataSet(2,1) # define a dataset in pybrain
Dtrain.addSample([0,0],[0])
Dtrain.addSample([0,1],[1])
Dtrain.addSample([1,0],[1])
Dtrain.addSample([1,1],[0])
from pybrain.supervised.trainers import BackpropTrainer
net = buildNetwork(2, 2, 1, hiddenclass=pybrain.SigmoidLayer, outclass=pybrain.SigmoidLayer)
T = BackpropTrainer(net, learningrate=0.1, momentum=0.9)
T.trainOnDataset(Dtrain, 1000)
print_pred2(D, net)
Explanation: <br/>
<br/>
Answer: It is impossible with only one neuron!
<br/>
<br/>   
We need to use more than one neuron....
Multilayer Neural Network
<img src="http://www.cs.nott.ac.uk/~gxk/courses/g5aiai/006neuralnetworks/images/ffnet.jpg">
Learning an XOR NN
End of explanation
plot_nn_prediction(net)
Explanation: XOR NN Output Plot
End of explanation
from pybrain.tools.validation import Validator
validator =  Validator()
Dlrrh = SupervisedDataSet(4,4) 
Dlrrh.addSample([1,1,0,0],[1,0,0,0])
Dlrrh.addSample([0,1,1,0],[0,0,1,1])
Dlrrh.addSample([0,0,0,1],[0,1,1,0])
df = pd.DataFrame(Dlrrh['input'],columns=['Big Ears', 'Big Teeth', 'Handsome', 'Wrinkled'])
print df.join(pd.DataFrame(Dlrrh['target'],columns=['Scream', 'Hug', 'Food', 'Kiss']))
net = buildNetwork(4, 3, 4, hiddenclass=pybrain.SigmoidLayer, outclass=pybrain.SigmoidLayer)
Explanation: The Little Red Riding Hood Neural Network
<img src="http://themaleharem.com/wp-content/uploads/2014/06/Walter-crane-little-red-riding-hood-meets-the-wolf-in-the-woods.jpg">
LRRH Network Architecture
<img src="lrrh net.jpg">
Training
End of explanation
T = BackpropTrainer(net, learningrate=0.01, momentum=0.99)
scores = []
for i in xrange(1000):
    T.trainOnDataset(Dlrrh, 1)
    prediction = net.activateOnDataset(Dlrrh)
    scores.append(validator.MSE(prediction, Dlrrh.getField('target')))
plt.ylabel('Mean Square Error')
plt.xlabel('Iteration')
plt.plot(scores)
Explanation: Backpropagation
End of explanation
def lrrh_input(vals):
    return pd.DataFrame(vals,index=['big ears', 'big teeth', 'handsome', 'wrinkled'], columns=['input'])
def lrrh_output(vals):
    return pd.DataFrame(vals,index=['scream', 'hug', 'offer food', 'kiss cheek'], columns=['output'])
in_vals = [1, 1, 0, 0]
lrrh_input(in_vals)
lrrh_output(net.activate(in_vals))
Explanation: Prediction
End of explanation
<END_TASK>