Creating sequence vector from text in Python
<p>I am now trying to prepare the input data for LSTM-based NN.
I have some big number of text documents and what i want is to make sequence vectors for each document so i am able to feed them as train data to LSTM RNN.</p>

<p>My poor approach:</p>

<pre><code>import re
import numpy as np
#raw data
train_docs = ['this is text number one', 'another text that i have']

#put all docs together
train_data = ''
for val in train_docs:
    train_data += ' ' + val

tokens = np.unique(re.findall('[a-zа-я0-9]+', train_data.lower()))
voc = {v: k for k, v in dict(enumerate(tokens)).items()}
</code></pre>

<p>and then brutforce replace each doc with a "voc" dict.</p>

<p>Is there any libs which can help with this task?</p>