<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 3.1. Randomizing our prediction
The first approach we're going to try is to randomize our character prediction. However, with an important detail
Step1:  Now, we'll use our model to generate a single prediction
Step2:  As we can see, the array contains numbers between 0 and 1; each one represents the probability of the character in that position of being the correct character to output. However, this doesn't seem to be a very clear representation. Let's see it graphically.
Step3:  As we can see, our model seems quite convinced that the correct character to output is 'i' (and, watching the seed, I can tell you that it's probably correct
Step4:  As you can see the final value is 1, because it is sure that the output will be chosen from one of the characters in the array. 
In order to use that as the PDF, what we'll do is to generate a random number between 0 and 1 and choose the first element of the array to be greater than it. As you can imagine, the char that will appear the most is the 'i', because a lot of numbers between 0 and 1 are lower than prob_cum['i'] but greater than prob_cum['h']. However, we won't always output 'i', so our output will have some random component while keeping a plausible distribution. 
3.1.1. Testing the improvement
Now it's time to see if our improvement is really an improvement or not
<ASSISTANT_TASK:>
Python Code:
import numpy as np
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Dropout
from keras.layers import LSTM
from keras.callbacks import ModelCheckpoint
from keras.utils import np_utils
-
# Load the network weights
filename = "weights-improvement-03-1.4625.hdf5"
model.load_weights(filename)
model.compile(loss='categorical_crossentropy', optimizer='adam')
# Pick a random seed
start = np.random.randint(0, len(dataX)-1)
pattern = dataX[start]
seed = ''.join([int_to_char[value] for value in pattern])
print "Seed:"
print "\"", seed, "\""
result_str = ""
Explanation: 3.1. Randomizing our prediction
The first approach we're going to try is to randomize our character prediction. However, with an important detail: the random distribution we'll use will be given by the output of our RNN.
Our RNN outputs an array of floating point numbers compressed between 0 and 1, each one representing the probability of the character in that position of being the most suitable option for the prediction. However, currently we search for the most suitable one and select it as our result. What we're going to try now, instead, is to randomize our prediction while taking into account this probabilities. 
We'll achieve this by using the output array of our RNN as the Probability Density Function [1] for our randomized prediction. As this array contains the probability of each character, the sum of all its elements will always be 1 (because the probability of our prediction being a character, no matter which one, is 1). Let's see it with an example
We're going to load our last model and try to predict a random character, in order to see the output we get (the code is the same we've already used on previous notebooks, so don't worry about it)
End of explanation
# Generate characters
x = np.reshape(pattern, (1, len(pattern), 1))
x = x / float(n_vocab)
prediction = model.predict(x, verbose=0)
print(prediction)
Explanation: Now, we'll use our model to generate a single prediction: we're not even going to learn a character for it, we simply want to check the output array
End of explanation
# Creating the indices array
indices = np.arange(len(prediction[0]))
# Creating the plots
fig, ax = plt.subplots()
preds = ax.bar(indices, prediction[0])
# Adding some text for labels and titles
ax.set_xlabel('Character')
ax.set_ylabel('Probability')
ax.set_title('Probabilty of each character of being the correct output (according to our model)')
ax.set_xticks(indices)
ax.set_xticklabels((c for c in chars if ord(c) < 0x81))  # We can't use non ascii chars as label
ax.set_ylim(top=1.0)
plt.show()
Explanation: As we can see, the array contains numbers between 0 and 1; each one represents the probability of the character in that position of being the correct character to output. However, this doesn't seem to be a very clear representation. Let's see it graphically.
End of explanation
prob_cum = np.cumsum(prediction[0])
# Creating the indices array
indices = np.arange(len(prob_cum))
# Creating the plots
fig, ax = plt.subplots()
preds = ax.bar(indices, prob_cum)
# Adding some text for labels and titles
ax.set_xlabel('Character')
ax.set_ylabel('Probability')
ax.set_title('Cummulative output probability')
ax.set_xticks(indices)
ax.set_xticklabels((c for c in chars if ord(c) < 0x81))  # We can't use non ascii chars as label
plt.show()
Explanation: As we can see, our model seems quite convinced that the correct character to output is 'i' (and, watching the seed, I can tell you that it's probably correct: "arremetió" is a real word in Spanish, and it would make sense in that context). However, we cannot simply discard the other characters: although they may seem improbable, there may be cases where an improbable option is the correct one (or, what happens more often, that the decision is not so clear). Therefore, as we've said, we're going to choose the output randomly by using this distribution as the PDF. 
We'll generate a new array with the same length as the output one that contains the cummulative probability of the characters, i.e. each element will contain the probability of the correct output character to be at his position or a previous one. The result will be similar to the following one:
End of explanation
# Generate characters
for i in range(500):
	x = np.reshape(pattern, (1, len(pattern), 1))
	x = x / float(n_vocab)
    
	prediction = model.predict(x, verbose=0)
    
    # Choosing the character randomly
	prob_cum = np.cumsum(prediction[0])
	rand_ind = np.random.rand()
	for i in range(len(prob_cum)):
	    if (rand_ind <= prob_cum[i]) and ((i == 0) or (rand_ind > prob_cum[i-1])):
	        index = i
	        break
            
	result = int_to_char[index]
	seq_in = [int_to_char[value] for value in pattern]
	result_str += result
	pattern.append(index)
	pattern = pattern[1:len(pattern)]
print "\nDone."
print seed + result_str
Explanation: As you can see the final value is 1, because it is sure that the output will be chosen from one of the characters in the array. 
In order to use that as the PDF, what we'll do is to generate a random number between 0 and 1 and choose the first element of the array to be greater than it. As you can imagine, the char that will appear the most is the 'i', because a lot of numbers between 0 and 1 are lower than prob_cum['i'] but greater than prob_cum['h']. However, we won't always output 'i', so our output will have some random component while keeping a plausible distribution. 
3.1.1. Testing the improvement
Now it's time to see if our improvement is really an improvement or not: we'll implement it in our predictions.
As the model and the libraries are already loaded we don't need to repeat that step, so we'll start directly by loading the weights and picking a random seed.
End of explanation
<END_TASK>