input dimensions to a one dimensional convolutional network in keras
<p>really finding it hard to understand the input dimensions to the convolutional 1d <a href="http://keras.io/layers/convolutional/#convolution1d" rel="noreferrer">layer</a> in keras:</p>

<p>Input shape</p>

<p>3D tensor with shape: (samples, steps, input_dim).</p>

<p>Output shape</p>

<p>3D tensor with shape: (samples, new_steps, nb_filter). steps value might have changed due to padding.</p>

<p>I want my network to take in a time series of prices (101, in order) and output 4 probabilities. My current non-convolutional network which does this fairly well (with a training set of 28000) looks like this:</p>

<pre><code>standardModel = Sequential()
standardModel.add(Dense(input_dim=101, output_dim=100, W_regularizer=l2(0.5), activation='sigmoid'))
standardModel.add(Dense(4, W_regularizer=l2(0.7), activation='softmax'))
</code></pre>

<p>To improve this, I want to make a feature map from the input layer which has a local receptive field of length 10. (and therefore has 10 shared weights and 1 shared bias). I then want to use max pooling and feed this in to a hidden layer of 40 or so neurons and then output this with 4 neurons with softmax in the outer layer. </p>

<p><a href="http://i.stack.imgur.com/Kx8yT.png" rel="noreferrer">picture (it's quite awful sorry!)</a></p>

<p>So ideally, the convolutional layer would take a 2d tensor of dimensions:</p>

<p>(minibatch_size, 101)</p>

<p>and output a 3d tensor of dimensions</p>

<p>(minibatch_size, 91, no_of_featuremaps)</p>

<p>However, the keras layer seems to require a dimension in the input called step. I've tried understanding this and still don't quite get it. In my case, should step be 1 as each step in the vector is an increase in the time by 1? Also, what is new_step? </p>

<p>In addition, how do you turn the output of the pooling layers (a 3d tensor) into input suitable for the standard hidden layer (i.e a Dense keras layer) in the form of a 2d tensor?</p>

<p>Update: After the very helpful suggestions given, I tried making a convolutional network like so:</p>

<pre><code>conv = Sequential()
conv.add(Convolution1D(64, 10, input_shape=(1,101)))
conv.add(Activation('relu'))
conv.add(MaxPooling1D(2))
conv.add(Flatten())
conv.add(Dense(10))
conv.add(Activation('tanh'))
conv.add(Dense(4))
conv.add(Activation('softmax'))
</code></pre>

<p>The line conv.Add(Flatten()) throws a range exceeds valid bounds error. Interestingly, this error is <strong>not</strong> thrown for just this code:</p>

<pre><code>conv = Sequential()
conv.add(Convolution1D(64, 10, input_shape=(1,101)))
conv.add(Activation('relu'))
conv.add(MaxPooling1D(2))
conv.add(Flatten())
</code></pre>

<p>doing </p>

<pre><code>print conv.input_shape
print conv.output_shape
</code></pre>

<p>results in </p>

<pre><code>(None, 1, 101
(None, -256)
</code></pre>

<p>being returned</p>

<p>Update 2:</p>

<p>Changed </p>

<pre><code>conv.add(Convolution1D(64, 10, input_shape=(1,101)))
</code></pre>

<p>to</p>

<pre><code>conv.add(Convolution1D(10, 10, input_shape=(101,1))
</code></pre>

<p>and it started working. However, is there any important different between 
inputting (None, 101, 1) to a 1d conv layer or (None, 1, 101) that I should be aware of? Why does (None, 1, 101) not work?</p>