<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Base Code
Here the neural net code is created and implemented. For the following notebooks, the neural network code will be saved to disk and imported. All weights are also saved to disk and upload from a previous training run. The code to train the weights has been commented out but remains in the notebook to display how the weights are called.
Step1:  Visualizing our Data
Step4:  Useful Functions
Step5:  Neural Networks 1D Input
Neural networks require a 1D input per output. Our data initially is 2D.
Step6:  Here, the input is now 1D. Because it has 64 elements per array, we'll need 64 input neurons.
Step8:  Create training and testing inputs
The training input for maximum accuracy is 1000 random samples. 
The testing input is the last 797 random samples. 
All inputs are converted to decimals
Step12:  Create training solution
Sigmoid functions can only produce values 0-1
We want output values 0-9
This means we need an output layer of 10 neurons
Whichever number neuron is "firing" (closest to 1) is the number answer
Example
Step13:  Creating the Training Class
This is the class used to train the neural network. The "train" function iterates through the input arrays and solution arrays and returns the weights.
Step14:  Our input array is "training_input" our solution array is "training_solution". I chose 40 hidden nodes and 90 iterations with a learning rate of 0.7.
Step15:  To generate the weights, use the commented out line below. Because it is a slow process, I will be loading weights from a previous training.
Step17:  Testing Class
Whent the "get_ans" function is called, the weights and input arrays will be used to calculate the solution.
Step18:  Calculate and Visualize Accuray
<ASSISTANT_TASK:>
Python Code:
import numpy as np
from sklearn.datasets import load_digits 
digits = load_digits()
from IPython.html.widgets import interact
%matplotlib inline
import matplotlib.pyplot as plt
import NNpix as npx
Explanation: Base Code
Here the neural net code is created and implemented. For the following notebooks, the neural network code will be saved to disk and imported. All weights are also saved to disk and upload from a previous training run. The code to train the weights has been commented out but remains in the notebook to display how the weights are called.
End of explanation
def interact_fun(i):
    plt.matshow(digits.images[i])
    plt.show()
    print("True Number: %d" % digits.target[i])
interact(interact_fun, i=(0,1796));
Explanation: Visualizing our Data:
End of explanation
 The activation function. 
def sigmoid(x):
    return 1/(1+np.exp(-x))
assert sigmoid(np.log(2)) == 2/3
 The derivative of the activation function 
def sigmoid_prime(x):
    return sigmoid(x)*(1-sigmoid(x))
Explanation: Useful Functions:
In order to change the weights, we'll need the sigmoid and sigmoid prime functions.
$\sigma(x) = \frac {1}{1+e^{-x}} \
 \sigma\prime(x) = \sigma(x) \times (1-\sigma(x)) $
End of explanation
print(digits.images[0])
Explanation: Neural Networks 1D Input
Neural networks require a 1D input per output. Our data initially is 2D.
End of explanation
print(digits.images[0].flatten())
Explanation: Here, the input is now 1D. Because it has 64 elements per array, we'll need 64 input neurons.
End of explanation
perm = np.random.permutation(1797)
 Divide by 100 to get inputs into decimals 
training_input = np.array([digits.images[perm[i]].flatten() for i in range(1000)])/100
test_input = np.array([digits.images[perm[i]].flatten() for i in range(1000,1797)])/100
assert len(training_input[0]) == 64
assert len(test_input[0]) == 64
Explanation: Create training and testing inputs
The training input for maximum accuracy is 1000 random samples. 
The testing input is the last 797 random samples. 
All inputs are converted to decimals
End of explanation
def create_soln(training_numbers):
     Creates 2D array for training solutions 
    a = np.repeat(0,10,None)
    a = np.repeat([a], len(training_numbers), 0)
    for i in range(len(training_numbers)):
        a[i][training_numbers[i]] = 1
    return a
 True number solutions used to calculate accuracy
number_solution = np.array([digits.target[perm[i]] for i in range(1000,1797)])
 Creates the array of solutions to be entered into the neural network
training_solution = create_soln([digits.target[perm[i]] for i in range(1000)])
assert len(training_solution[0]) == 10
Explanation: Create training solution
Sigmoid functions can only produce values 0-1
We want output values 0-9
This means we need an output layer of 10 neurons
Whichever number neuron is "firing" (closest to 1) is the number answer
Example:
Output Value: 4
Neural Network Output: [0,0,0,0,1,0,0,0,0,0]
End of explanation
class NN_training(object):
    
    def __init__(self, input_array, soln, hidnum, iters, lr):
        self.input_array = input_array
        self.soln = soln
        #Number of hidden nodes
        self.hidnum = hidnum
        #Number of iterations through the training set
        self.iters = iters
        #Initalize WIJ weights (input to hidden)
        self.wij = np.random.uniform(-.5,0.5,(hidnum,65))
        #Initalize WJK weights (hidden to output)
        self.wjk = np.random.uniform(-0.5,0.5,(10,hidnum+1))
        #Set a learning rate
        self.lr = lr
    def train(self):
        iters = self.iters   
        for n in range(iters): 
            for i in range(len(self.input_array)):
                soln = self.soln[i]
                hidnum = self.hidnum
                input_array = np.append(self.input_array[i],[1])
                #Find sum of weights x input array values for each hidden
                self.hidden_sums = (sum((input_array * self.wij).T)).T
                #Find outputs of hidden neurons; include bias
                self.hidden_out = np.append(sigmoid(self.hidden_sums),[1])
                #Find sums of weights x hidden outs for each neuron in output layer
                self.output_sums = (sum((self.hidden_out * self.wjk).T)).T
                #Find output of the outputs
                self.output_out = sigmoid(self.output_sums)
                self.E = self.output_out - soln
                 #Find delta values for each output
                self.output_deltas = self.E * sigmoid_prime(self.output_sums)
                 #Find delta values for each hidden
                self.hidden_deltas = sigmoid_prime(np.delete(self.hidden_out,[hidnum],None)) * sum((self.output_deltas * (np.delete(self.wjk, [hidnum], 1)).T).T)
                #Change weights
                self.wij = -self.lr * (self.hidden_deltas*(np.repeat([input_array],hidnum,0)).T).T + self.wij
                self.wjk = -self.lr * (self.output_deltas*(np.repeat([self.hidden_out],10,0)).T).T + self.wjk
        return (self.wij, self.wjk)
Explanation: Creating the Training Class
This is the class used to train the neural network. The "train" function iterates through the input arrays and solution arrays and returns the weights.
End of explanation
my_net = NN_training(training_input, training_solution, 40, 90, 0.7)
Explanation: Our input array is "training_input" our solution array is "training_solution". I chose 40 hidden nodes and 90 iterations with a learning rate of 0.7.
End of explanation
# x, y = my_net.train()
f = np.load("NNweights.npz")
list(f)
x = f['arr_0']
y = f['arr_1']
assert len(x) == 40
assert len(y) == 10
Explanation: To generate the weights, use the commented out line below. Because it is a slow process, I will be loading weights from a previous training.
End of explanation
class NN_ask (object):
     Feed forward using final weights from training backpropagation 
    def __init__(self, input_array, wij, wjk):
        self.input_array = input_array
        self.wij = wij
        self.wjk = wjk
    def get_ans(self):
        wij = self.wij
        wjk = self.wjk
        soln = []
        for i in range(len(self.input_array)):
            input_array = np.append(self.input_array[i],[1])
            self.hidden_sums = (sum((input_array * wij).T)).T
            self.hidden_out = np.append(sigmoid(self.hidden_sums),[1]) 
            self.output_sums = (sum((self.hidden_out * wjk).T)).T
            self.output_out = sigmoid(self.output_sums)
            for i in range(len(self.output_out)):
                if self.output_out[i] == max(self.output_out):
                    a = i
                    soln.append(a)
        return soln
test_net = NN_ask(test_input, x, y)
comp_vals = test_net.get_ans() 
Explanation: Testing Class
Whent the "get_ans" function is called, the weights and input arrays will be used to calculate the solution.
End of explanation
print(((sum((comp_vals - number_solution == 0).astype(int)) / (1797-1000)) * 100), "%")
def interacting(i):
    plt.matshow(digits.images[perm[i+1000]])
    plt.show()
    print("Neural Network's Value: %d" %comp_vals[i])
    print("True Known Solution %d" %number_solution[i])
interact(interacting, i=(0,796));
Explanation: Calculate and Visualize Accuray
End of explanation
<END_TASK>