Why do we have normally more than one fully connected layers in the late steps of the CNNs?
<p>As I noticed, in many popular architectures of the convolutional neural networks (e.g. AlexNet), people use more than one fully connected layers with almost the same dimension to gather the responses to previously detected features in the early layers. </p>

<p>Why do not we use just one FC for that? Why this <strong>hierarchical arrangement</strong> of the fully connected layers is possibly more useful?</p>

<p><a href="https://i.stack.imgur.com/L5rfa.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/L5rfa.jpg" alt="enter image description here"></a></p>