LSTM Autoencoder for music - Keras [Sequence to sequence]
<p>So, I'm trying to learn fixed vector representations for segments of about 200 songs (~ 3-5 minutes per song) and wanted to use an LSTM-based Sequence-to-sequence Autoencoder for it.</p>

<p>I'm preprocessing the audio (using librosa) as follows:</p>

<ol>
<li>I'm first just getting a raw audio signal time series of shape around (1500000,) - (2500000,) per song.</li>
<li>I'm then slicing each raw time series into segments and getting a lower-level mel spectrogram matrix of shape (512, 3000) - (512, 6000) per song. Each of these (512,) vectors can be referred to as 'mini-songs' as they represent parts of the song.</li>
<li>I vertically stack all these mini-songs of all the songs together to create the training data (let's call this <strong>X</strong>). X turns out to be (512, 600000) in size, where the first dimension (512) is the window size and the second dimension (600000) is the total number of 'mini-songs' in the dataset.</li>
<li>Which is to say, there are about 600000 mini-songs in X - each column in X represents a mini-song of length (512,).</li>
<li>Each of these (512,) mini-song vectors should be encoded into a (50,) vector per mini-song i.e. we will have 600000 (50,) vectors at the end of the process.</li>
<li>In more standard terminology, I have 600000 training samples each of length 512. [Think of this as being similar to an image dataset - 600000 images, each of length 784, where the images are of resolution 32x32. Except in my case I want to treat the 512-length samples as sequences that have temporal properties.]</li>
</ol>

<p>I read the example <a href="https://blog.keras.io/building-autoencoders-in-keras.html" rel="nofollow noreferrer">here</a> and was looking to extend that for my use case. I was wondering what the <code>timesteps</code> and <code>input_dim</code> parameters to the <code>Input</code> layer should be set to. </p>

<p>I'm setting <code>timesteps = X.shape[0]</code> (i.e. 512 in this case) and <code>input_dim = X.shape[1]</code> (i.e 600000). Is this the correct way to go about it?</p>

<p><strong>Edit:</strong> Added clarifications above. </p>