<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Initial Setup
Step1:  Example Data (centered Quadratic)
Step2:  Simple Feed-Foward 1-Layer Neural Networks
Ok, try having a simple 3-node 1-layer neural network
Step3:  Let's visualize the inputs to the final layer
Step4:  hmm, maybe not enough hidden nodes to get the nonlinearity, try 10
Step5:  Ok, it learned better and faster, let's see fewer training epochs per line
Step6:  WTF? ... oh, right .... we are using a local optimizer so we get local optima 
<ASSISTANT_TASK:>
Python Code:
## Setup the path for our codebase
import sys
sys.path.append( '../code/' )
%matplotlib inline
import matplotlib.pyplot as plt
Explanation: Initial Setup
End of explanation
import neural_network.simple as simple
data = simple.generate_hill_data(100)
xs = map(lambda z: z[0], data)
ys = map(lambda z: z[1], data)
plt.plot( xs, ys )
Explanation: Example Data (centered Quadratic)
End of explanation
nn = simple.SimpleScalarF_1Layer(hidden_layer_size=3)
simple.plot_fits( nn, data, train_epochs=200 )
Explanation: Simple Feed-Foward 1-Layer Neural Networks
Ok, try having a simple 3-node 1-layer neural network
End of explanation
nn.visualize_inputs_to_final_layer(data)
Explanation: Let's visualize the inputs to the final layer
End of explanation
nn = simple.SimpleScalarF_1Layer(hidden_layer_size=10)
simple.plot_fits( nn, data, train_epochs=200 )
Explanation: hmm, maybe not enough hidden nodes to get the nonlinearity, try 10
End of explanation
nn = simple.SimpleScalarF_1Layer(hidden_layer_size=10,learning_rate=0.001)
simple.plot_fits( nn, data, train_epochs=20 )
Explanation: Ok, it learned better and faster, let's see fewer training epochs per line
End of explanation
nn = simple.SimpleScalarF_1Layer(hidden_layer_size=10,learning_rate=0.001)
simple.plot_fits( nn, data, train_epochs=20 )
nn = simple.SimpleScalarF_1Layer(hidden_layer_size=10,learning_rate=0.001)
simple.plot_fits( nn, data, train_epochs=2000 )
nn.visualize_inputs_to_final_layer(data)
Explanation: WTF? ... oh, right .... we are using a local optimizer so we get local optima :-)
End of explanation
<END_TASK>