Tensorflow convert array of tensors into single tensor
<p>I'm building a LSTM RNN with Tensorflow that performs pixel-wise classification (or, maybe a better way to put it is, pixel-wise prediction?) </p>

<p>Bear with me as I explain the title.</p>

<p>The network looks like the following drawing...</p>

<p><a href="https://i.stack.imgur.com/djRCX.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/djRCX.jpg" alt="enter image description here"></a></p>

<p>The idea goes like this... an input image of size (200,200) is the input into a LSTM RNN of size (200,200,200). Each sequence output from the LSTM tensor vector (the pink boxes in the LSTM RNN) is fed into a MLP, and then the MLP makes a single output prediction -- ergo <em>pixel-wise prediction</em> (you can see how one input pixel generates one output "pixel"</p>

<p>The code looks like this (not all of the code, just parts that are needed):</p>

<pre><code>...
n_input_x = 200
n_input_y = 200

x = tf.placeholder("float", [None, n_input_x, n_input_y])
y = tf.placeholder("float", [None, n_input_x, n_input_y])

def RNN(x):
    x = tf.transpose(x, [1, 0, 2])
    x = tf.reshape(x, [-1, n_input_x])
    x = tf.split(0, n_steps, x)

    lstm_cell = rnn_cell.BasicLSTMCell(n_hidden, forget_bias=1.0, state_is_tuple=True)
    outputs, states = rnn.rnn(lstm_cell, x, dtype=tf.float32)

    output_matrix = []
    for i in xrange(200):
        temp_vector = []
        for j in xrange(200):
            lstm_vector = outputs[j]
            pixel_pred = multilayer_perceptron(lstm_vector, mlp_weights, mlp_biases)
            temp_vector.append(pixel_pred)
        output_matrix.append(temp_vector)
        print i

    return output_matrix

temp = RNN(x)
pred = tf.placeholder(temp, [None, n_input_x, n_input_y])
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
...
</code></pre>

<p>I have confirmed that the output of <code>RNN</code> -- that is, what is stored in <code>temp</code> is a 200x200 array of <code>&lt;tf.Tensor 'Softmax_39999:0' shape=(?, 1) dtype=float32&gt;</code></p>

<p>As you can see, I place <code>temp</code> in a <code>tf.placeholder</code> of the same shape (<code>None</code> for the batch size ... or do I need this?)... and the program just exits as if it completed running. Ideally what I want to see when I debug and print <code>pred</code> is something like <code>&lt;tf.Tensor shape=(200,200)&gt;</code> </p>

<p>When I debug, the first time I execute <code>pred = tf.placeholder(temp, [None, n_input_x, n_input_y])</code> I get <code>TypeError: TypeErro...32&gt;]].",)</code> and then it <code>returns</code> and I try again, and it says <code>Exception AttributeError: "'NoneType' object has no attribute 'path'" in &lt;function _remove at 0x7f1ab77c26e0&gt; ignored</code></p>

<p><strong>EDIT</strong> I also now realize that I need to place the lines </p>

<pre><code>lstm_cell = rnn_cell.BasicLSTMCell(n_hidden, forget_bias=1.0, state_is_tuple=True)
outputs, states = rnn.rnn(lstm_cell, x, dtype=tf.float32)
</code></pre>

<p>Inside the first loop so that new 2D LSTM RNN are generated, however I'm getting an error about variable reusing <code>ValueError: Variable RNN/BasicLSTMCell/Linear/Matrix does not exist, disallowed. Did you mean to set reuse=None in VarScope?</code> </p>

<p>So in other words, is it isn't auto incrementing the RNN tensor name?</p>