Neural Network LSTM input shape from dataframe
<p>I am trying to implement an <a href="https://keras.io/layers/recurrent/#lstm" rel="noreferrer">LSTM with Keras</a>.</p>

<p>I know that LSTM's in Keras require a 3D tensor with shape <code>(nb_samples, timesteps, input_dim)</code> as an input. However, I am not entirely sure how the input should look like in my case, as I have just one sample of <code>T</code> observations for each input, not multiple samples, i.e. <code>(nb_samples=1, timesteps=T, input_dim=N)</code>. Is it better to split each of my inputs into samples of length <code>T/M</code>? <code>T</code> is around a few million observations for me, so how long should each sample in that case be, i.e., how would I choose <code>M</code>?</p>

<p>Also, am I right in that this tensor should look something like:</p>

<pre><code>[[[a_11, a_12, ..., a_1M], [a_21, a_22, ..., a_2M], ..., [a_N1, a_N2, ..., a_NM]], 
 [[b_11, b_12, ..., b_1M], [b_21, b_22, ..., b_2M], ..., [b_N1, b_N2, ..., b_NM]], 
 ..., 
 [[x_11, x_12, ..., a_1M], [x_21, x_22, ..., x_2M], ..., [x_N1, x_N2, ..., x_NM]]]
</code></pre>

<p>where M and N defined as before and x corresponds to the last sample that I would have obtained from splitting as discussed above? </p>

<p>Finally, given a pandas dataframe with <code>T</code> observations in each column, and <code>N</code> columns, one for each input, how can I create such an input to feed to Keras?</p>