Fully-connected layer weight dimensions in TensorFlow ConvNet
<p>I've been coding along <a href="https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/3%20-%20Neural%20Networks/convolutional_network.ipynb" rel="noreferrer">this example</a> of a convolution net in TensorFlow and I'm mystified by this allocation of weights:</p>

<pre><code>weights = {

# 5x5 conv, 1 input, 32 outputs
'wc1': tf.Variable(tf.random_normal([5, 5, 1, 32])),

# 5x5 conv, 32 inputs, 64 outputs
'wc2': tf.Variable(tf.random_normal([5, 5, 32, 64])), 

# fully connected, 7*7*64 inputs, 1024 outputs
'wd1': tf.Variable(tf.random_normal([7*7*64, 1024])), 

# 1024 inputs, 10 outputs (class prediction)
'out': tf.Variable(tf.random_normal([1024, n_classes])) 

}
</code></pre>

<p><strong>How do we know the 'wd1' weight matrix should have 7 x 7 x 64 rows?</strong></p>

<p>It's later used to reshape the output of the second convolution layer:</p>

<pre><code># Fully connected layer
# Reshape conv2 output to fit dense layer input
dense1 = tf.reshape(conv2, [-1, _weights['wd1'].get_shape().as_list()[0]]) 

# Relu activation
dense1 = tf.nn.relu(tf.add(tf.matmul(dense1, _weights['wd1']), _biases['bd1']))
</code></pre>

<p>By my math, pooling layer 2 (conv2 output) has 4 x 4 x 64 neurons.</p>

<p><strong>Why are we reshaping to [-1, 7*7*64]?</strong></p>