Is there any sense to use autoencoder for network with batch normalization?
<p>As known, main problem in DNN is long time of learning. </p>

<p>But there are some ways to accelerate learning:</p>

<ol>
<li><strong>Batch Normalization</strong> <code>=(x-AVG)/Variance</code>: <a href="https://arxiv.org/abs/1502.03167" rel="nofollow noreferrer">https://arxiv.org/abs/1502.03167</a></li>
</ol>

<blockquote>
  <p>Batch Normalization achieves the same accuracy with <strong>14 times fewer
  training steps</strong></p>
</blockquote>

<ol start="2">
<li><strong>ReLU</strong> <a href="https://en.wikipedia.org/wiki/Rectifier_(neural_networks)" rel="nofollow noreferrer"><code>=max(x, 0)</code></a> - rectified linear unit (ReLU,LReLU,PReLU,RReLU): <a href="https://arxiv.org/abs/1505.00853" rel="nofollow noreferrer">https://arxiv.org/abs/1505.00853</a></li>
</ol>

<blockquote>
  <p>The advantage of using non-saturated activation function lies in two
  aspects: The first is to solve the so called “exploding/vanishing
  gradient”. The second is to <strong>accelerate the convergence speed</strong>.</p>
</blockquote>

<p>Or any one: (maxout, ReLU-family, tanh)</p>

<ol start="3">
<li><strong>Fast weight initialization</strong> (with avoiding vanishing or exploding gradients): <a href="https://arxiv.org/abs/1511.06856" rel="nofollow noreferrer">https://arxiv.org/abs/1511.06856</a></li>
</ol>

<blockquote>
  <p>Our <strong>initialization</strong> matches the current state-of-the-art unsupervised
  or self-supervised pre-training methods on standard computer vision
  tasks, such as image classification and object detection, while being
  roughly <strong>three orders of magnitude faster</strong>.</p>
</blockquote>

<p>Or LSUV-initialization (Layer-sequential unit-variance): <a href="https://arxiv.org/abs/1511.06422" rel="nofollow noreferrer">https://arxiv.org/abs/1511.06422</a></p>

<p>But if we use all steps: (1) Batch Normalization, (2) ReLU, (3) Fast weight initialization or LSUV - then is there any sense to use <a href="https://en.wikipedia.org/wiki/Autoencoder" rel="nofollow noreferrer">autoencoder/autoassociator</a> in any steps of training deep neural network?</p>