<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 I followed along Aneesh Joshi's blog post on word2vec in this notebook.
Step1:  step 1. read in the data, create word dictionary, created one-hot vectors for each word
Step2:  step 2. create tensorflow word2vec model
<ASSISTANT_TASK:>
Python Code:
import numpy as np
import tensorflow as tf
Explanation: I followed along Aneesh Joshi's blog post on word2vec in this notebook.
End of explanation
# load data
with open('darksouls_training.txt', 'r') as fh:
    training = [sent.replace('.','').replace('\n', '').lower() for sent in fh.readlines()]
# with open('darksouls_test.txt', 'r') as fh:
#     test = [sent.replace('.','').replace('\n', '').lower() for sent in fh.readlines()]
# create vocabulary
word_list = []
for sent in training:
    for word in sent.split(' '):
        word_list.append(word)
# for sent in test:
#     for word in sent.split(' '):
#         word_list.append(word)
voc = set(word_list)
# create one-hot vector for each word
word2int = {}
int2word = {}
for ind, word in enumerate(voc):
    word2int[word] = ind
    int2word[ind] = word
# split the sentences
sent_train = []
for sent in training:
    sent_train.append(sent.split(' '))
# sent_test = []
# for sent in test:
#     sent_test.append(sent.split(' '))
# create word pairs
data_train = []
WINDOW_SIZE = 5
for sentence in sent_train:
    for ind, word in enumerate(sentence):
        for nb_word in sentence[max(ind - WINDOW_SIZE, 0) : min(ind + WINDOW_SIZE, len(sentence)) + 1] :
            if nb_word != word:
                data_train.append([word, nb_word])
# convert to one-hot
def to_one_hot(data_point_index, vocab_size):
    temp = np.zeros(vocab_size)
    temp[data_point_index] = 1
    return temp
data_train[0]
x_train = []
y_train = []
for word_pair in data_train:
    x_train.append(to_one_hot(word2int[word_pair[0]], len(voc)))
    y_train.append(to_one_hot(word2int[word_pair[1]], len(voc)))
x_train = np.asarray(x_train)
y_train = np.asarray(y_train)
Explanation: step 1. read in the data, create word dictionary, created one-hot vectors for each word
End of explanation
x = tf.placeholder(dtype=tf.float32, shape=(None, len(voc)))
y_label = tf.placeholder(dtype=tf.float32, shape=(None, len(voc)))
# hidden layer
EMBEDDING_DIM = 5
W1 = tf.Variable(tf.random_normal([len(voc), EMBEDDING_DIM]))
b1 = tf.Variable(tf.random_normal([EMBEDDING_DIM]))
hidden_rep = tf.add(tf.matmul(x, W1), b1)
W2 = tf.Variable(tf.random_normal([EMBEDDING_DIM, len(voc), ]))
b2 = tf.Variable(tf.random_normal([len(voc)]))
pred = tf.nn.softmax(tf.add(tf.matmul(hidden_rep, W2), b2))
# run the model
sess = tf.Session()
init = tf.global_variables_initializer()
sess.run(init)
#loss function
cross_entropy_loss = tf.reduce_mean(-tf.reduce_sum(y_label * tf.log(pred),
                                                   reduction_indices=1))
# training step
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy_loss)
# epoch number
n_epoch = 10000
for epoch in xrange(n_epoch):
    sess.run(train_step,
             feed_dict={x: x_train, y_label:y_train})
    if epoch % 100 == 0:
        print('epoch {}: loss is '.format(epoch), sess.run(cross_entropy_loss,
                                                           feed_dict={x: x_train, y_label: y_train}))
saver = tf.train.Saver()
saver.save(sess, 'dark_souls_word2vec_model.ckpt')
Explanation: step 2. create tensorflow word2vec model
End of explanation
<END_TASK>