<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 LSTM Poem Generator
I'm trying to generate poem
Method 1 - Character based sequence generation with LSTM
Method 2 - Word based sequence generation with LSTM
Download the sonnets text from 
Step1:  Method 1 - Character Based Poem Generation
Step2:  Observation...
I guess those readable words came from the original poem, they served as the testing data.
Method 2 - Word Based Poem Generation
Simply map words to index, without tokenizing
<ASSISTANT_TASK:>
Python Code:
import numpy as np
import pandas as pd
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Dropout
from keras.layers import LSTM
from keras.layers import RNN
from keras.utils import np_utils
sample_poem = open('sample_sonnets.txt').read().lower()
sample_poem[77:99]
Explanation: LSTM Poem Generator
I'm trying to generate poem
Method 1 - Character based sequence generation with LSTM
Method 2 - Word based sequence generation with LSTM
Download the sonnets text from : https://github.com/pranjal52/text_generators/blob/master/sonnets.txt
End of explanation
characters = sorted(list(set(sample_poem)))
n_to_char = {n:char for n, char in enumerate(characters)}  # store characters and their index
char_to_n = {char:n for n, char in enumerate(characters)}
print(n_to_char[7])
print(n_to_char[9])
X = []
y = []
total_len = len(sample_poem)
seq_len = 100  # each time we choose 100 character as a sequence and predict the next character after the sequence
for i in range(total_len - seq_len):
    seq = sample_poem[i:i+seq_len]
    label = sample_poem[i+seq_len]
    
    X.append([char_to_n[char] for char in seq])
    y.append(char_to_n[label])
# LSTM acceptable format, (number of sequneces(batch size), sequnece length (timesteps), number of features)
X_modified = np.reshape(X, (len(X), seq_len, 1))  
X_modified = X_modified / float(len(characters))  # normalize the value
y_modified = np_utils.to_categorical(y)  # convert to one-hot format, there are 36 distinct characters in total
print(X_modified.shape)
print(y_modified[4:10])
model = Sequential()
model.add(LSTM(700, input_shape=(X_modified.shape[1], X_modified.shape[2]), return_sequences=True))
model.add(Dropout(0.2))  # dropout is used for regularization
model.add(LSTM(700, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(700))
model.add(Dropout(0.2))
model.add(Dense(y_modified.shape[1], activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam')
model.fit(X_modified, y_modified, epochs=10, batch_size=100)
model.save_weights('poem_generator_gigantic.h5')  # save weights, so that later we can use without re-running the model
model.load_weights('poem_generator_gigantic.h5')
new_poem_lst = []
for j in range(77, 99):  # randomly choose some records and predict the sequence (generate the poem)
    string_mapped = X[j]  
    full_string = [n_to_char[value] for value in string_mapped]
    for i in range(10):  # predict the next 10 character
        x = np.reshape(string_mapped,(1,len(string_mapped), 1))
        x = x / float(len(characters))
        # predict the next character
        pred_index = np.argmax(model.predict(x, verbose=0))  
        seq = [n_to_char[value] for value in string_mapped]
        full_string.append(n_to_char[pred_index])
        # predicted character will be added to support the next prediction
        string_mapped.append(pred_index)
        string_mapped = string_mapped[1:len(string_mapped)]
        
    new_poem_lst.extend(full_string)
generated_poem = ''.join(new_poem_lst)
print(generated_poem)
Explanation: Method 1 - Character Based Poem Generation
End of explanation
words = sorted(list(set(sample_poem.split())))
n_to_word = {n:word for n, word in enumerate(words)}  # store characters and their index
word_to_n = {word:n for n, word in enumerate(words)}
print(n_to_word[7])
print(n_to_word[9])
X = []
y = []
all_words = sample_poem.split()
total_len = len(all_words)
seq_len = 100  # each time we choose 100 character as a sequence and predict the next character after the sequence
for i in range(total_len - seq_len):
    seq = all_words[i:i+seq_len]
    label = all_words[i+seq_len]
    
    X.append([word_to_n[word] for word in seq])
    y.append(word_to_n[label])
# LSTM acceptable format, (number of sequneces(batch size), sequnece length (timesteps), number of features)
X_modified = np.reshape(X, (len(X), seq_len, 1))  
X_modified = X_modified / float(len(words))  # normalize the value
y_modified = np_utils.to_categorical(y)  # convert to one-hot format, there are 36 distinct characters in total
print(X_modified.shape)
print(y_modified[4:10])
model = Sequential()
model.add(LSTM(700, input_shape=(X_modified.shape[1], X_modified.shape[2]), return_sequences=True))
model.add(Dropout(0.2))  # dropout is used for regularization
model.add(LSTM(700, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(700))
model.add(Dropout(0.2))
model.add(Dense(y_modified.shape[1], activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam')
model.fit(X_modified, y_modified, epochs=10, batch_size=100)
model.save_weights('poem_generator_gigantic_word.h5')
model.load_weights('poem_generator_gigantic_word.h5')
new_poem_lst = []
for j in range(77, 99):  # randomly choose some records and predict the sequence (generate the poem)
    string_mapped = X[j]  
    full_string = []  # different from character based, here not recording the original sequence
    for i in range(10):  # predict the next 10 character
        x = np.reshape(string_mapped,(1,len(string_mapped), 1))
        x = x / float(len(words))
        # predict the next character
        pred_index = np.argmax(model.predict(x, verbose=0))  
        seq = [n_to_word[value] for value in string_mapped]
        full_string.append(n_to_word[pred_index])
        # predicted character will be added to support the next prediction
        string_mapped.append(pred_index)
        string_mapped = string_mapped[1:len(string_mapped)]
        
    new_poem_lst.extend(full_string)
generated_poem = ' '.join(new_poem_lst)
print(generated_poem)
Explanation: Observation...
I guess those readable words came from the original poem, they served as the testing data.
Method 2 - Word Based Poem Generation
Simply map words to index, without tokenizing
End of explanation
<END_TASK>