Basic 1d convolution in tensorflow
<p>OK, I'd like to do a 1-dimensional convolution of time series data in Tensorflow. This is apparently supported using <code>tf.nn.conv2d</code>, according to <a href="https://github.com/tensorflow/tensorflow/issues/2165" rel="noreferrer">these</a> <a href="https://github.com/tensorflow/tensorflow/issues/1136" rel="noreferrer">tickets</a>, and <a href="https://www.tensorflow.org/versions/r0.9/api_docs/python/nn.html#convolution" rel="noreferrer">the manual</a>. the only requirement is to set <code>strides=[1,1,1,1]</code>. Sounds simple!</p>

<p>However, I cannot work out how to do this in even a very minimal test case. What am I doing wrong?</p>

<p>Let's set this up.</p>

<pre><code>import tensorflow as tf
import numpy as np
print(tf.__version__)
&gt;&gt;&gt; 0.9.0
</code></pre>

<p>OK, now generate a basic convolution test on two small arrays. I will make it easy by using a batch size of 1, and since time series are 1-dimensional, I will have an "image height" of 1. And since it's a univariate time series, clearly the number of "channels" is also 1, so this will be simple, right?</p>

<pre><code>g = tf.Graph()
with g.as_default():
    # data shape is "[batch, in_height, in_width, in_channels]",
    x = tf.Variable(np.array([0.0, 0.0, 0.0, 0.0, 1.0]).reshape(1,1,-1,1), name="x")
    # filter shape is "[filter_height, filter_width, in_channels, out_channels]"
    phi = tf.Variable(np.array([0.0, 0.5, 1.0]).reshape(1,-1,1,1), name="phi")
    conv = tf.nn.conv2d(
        phi,
        x,
        strides=[1, 1, 1, 1],
        padding="SAME",
        name="conv")
</code></pre>

<p>BOOM. Error.</p>

<pre><code>ValueError: Dimensions 1 and 5 are not compatible
</code></pre>

<p>OK, For a start, I don't understand how this should happen with <em>any</em> dimension, since I've specified that I'm padding the arguments in the convolution OP. </p>

<p>but fine, maybe there are limits to that. I must have got the documentation confused and set up this convolution on the wrong axes of the tensor. I'll try all possible permutations:</p>

<pre><code>for i in range(4):
    for j in range(4):
        shape1 = [1,1,1,1]
        shape1[i] = -1
        shape2 = [1,1,1,1]
        shape2[j] = -1
        x_array = np.array([0.0, 0.0, 0.0, 0.0, 1.0]).reshape(*shape1)
        phi_array = np.array([0.0, 0.5, 1.0]).reshape(*shape2)
        try:
            g = tf.Graph()
            with g.as_default():
                x = tf.Variable(x_array, name="x")
                phi = tf.Variable(phi_array, name="phi")
                conv = tf.nn.conv2d(
                    x,
                    phi,
                    strides=[1, 1, 1, 1],
                    padding="SAME",
                    name="conv")
                init_op = tf.initialize_all_variables()
            sess = tf.Session(graph=g)
            sess.run(init_op)
            print("SUCCEEDED!", x_array.shape, phi_array.shape, conv.eval(session=sess))
            sess.close()
        except Exception as e:
            print("FAILED!", x_array.shape, phi_array.shape, type(e), e.args or e._message)
</code></pre>

<p>Result:</p>

<pre><code>FAILED! (5, 1, 1, 1) (3, 1, 1, 1) &lt;class 'ValueError'&gt; ('Filter must not be larger than the input: Filter: (3, 1) Input: (1, 1)',)
FAILED! (5, 1, 1, 1) (1, 3, 1, 1) &lt;class 'ValueError'&gt; ('Filter must not be larger than the input: Filter: (1, 3) Input: (1, 1)',)
FAILED! (5, 1, 1, 1) (1, 1, 3, 1) &lt;class 'ValueError'&gt; ('Dimensions 1 and 3 are not compatible',)
FAILED! (5, 1, 1, 1) (1, 1, 1, 3) &lt;class 'tensorflow.python.framework.errors.InvalidArgumentError'&gt; No OpKernel was registered to support Op 'Conv2D' with these attrs
     [[Node: conv = Conv2D[T=DT_DOUBLE, data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](x/read, phi/read)]]
FAILED! (1, 5, 1, 1) (3, 1, 1, 1) &lt;class 'tensorflow.python.framework.errors.InvalidArgumentError'&gt; No OpKernel was registered to support Op 'Conv2D' with these attrs
     [[Node: conv = Conv2D[T=DT_DOUBLE, data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](x/read, phi/read)]]
FAILED! (1, 5, 1, 1) (1, 3, 1, 1) &lt;class 'ValueError'&gt; ('Filter must not be larger than the input: Filter: (1, 3) Input: (5, 1)',)
FAILED! (1, 5, 1, 1) (1, 1, 3, 1) &lt;class 'ValueError'&gt; ('Dimensions 1 and 3 are not compatible',)
FAILED! (1, 5, 1, 1) (1, 1, 1, 3) &lt;class 'tensorflow.python.framework.errors.InvalidArgumentError'&gt; No OpKernel was registered to support Op 'Conv2D' with these attrs
     [[Node: conv = Conv2D[T=DT_DOUBLE, data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](x/read, phi/read)]]
FAILED! (1, 1, 5, 1) (3, 1, 1, 1) &lt;class 'ValueError'&gt; ('Filter must not be larger than the input: Filter: (3, 1) Input: (1, 5)',)
FAILED! (1, 1, 5, 1) (1, 3, 1, 1) &lt;class 'tensorflow.python.framework.errors.InvalidArgumentError'&gt; No OpKernel was registered to support Op 'Conv2D' with these attrs
     [[Node: conv = Conv2D[T=DT_DOUBLE, data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](x/read, phi/read)]]
FAILED! (1, 1, 5, 1) (1, 1, 3, 1) &lt;class 'ValueError'&gt; ('Dimensions 1 and 3 are not compatible',)
FAILED! (1, 1, 5, 1) (1, 1, 1, 3) &lt;class 'tensorflow.python.framework.errors.InvalidArgumentError'&gt; No OpKernel was registered to support Op 'Conv2D' with these attrs
     [[Node: conv = Conv2D[T=DT_DOUBLE, data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](x/read, phi/read)]]
FAILED! (1, 1, 1, 5) (3, 1, 1, 1) &lt;class 'ValueError'&gt; ('Dimensions 5 and 1 are not compatible',)
FAILED! (1, 1, 1, 5) (1, 3, 1, 1) &lt;class 'ValueError'&gt; ('Dimensions 5 and 1 are not compatible',)
FAILED! (1, 1, 1, 5) (1, 1, 3, 1) &lt;class 'ValueError'&gt; ('Dimensions 5 and 3 are not compatible',)
FAILED! (1, 1, 1, 5) (1, 1, 1, 3) &lt;class 'ValueError'&gt; ('Dimensions 5 and 1 are not compatible',)
</code></pre>

<p>Hmm. OK, it looks like there are two problems now. Firstly, the <code>ValueError</code> is about applying the filter along the wrong axis, I guess, although there are two forms.</p>

<p>But then the axes along which I can apply the filter are confusing too - notice that it actually constructs the graph with input shape (5, 1, 1, 1)  and filter shape (1, 1, 1, 3). AFAICT from the documentation, this should be a filter that looks at on example from the batch, one "pixel" and one "channel" and outputs 3 "channels". Why does that one work, then, when others do not?</p>

<p>Anyway, sometimes it does not fail while constructing the graph.
Sometime it constructs the graph; then we get the <code>tensorflow.python.framework.errors.InvalidArgumentError</code>. From some <a href="https://github.com/tensorflow/tensorflow/issues/524" rel="noreferrer">confusing github tickets</a> I gather this is probably due to <del>the fact that I'm running on CPU instead of GPU, or vice versa</del> the fact that the convolution Op is only defined for 32 bit floats, not 64 bit floats. If anyone could throw some light on <em>which</em> axes I should be aligning <em>what</em> on, in order to convolve a time series with a kernel, I'd be very grateful.</p>