Why input is scaled in tf.nn.dropout in tensorflow?
<p>I can't understand why dropout works like this in tensorflow. The blog of <a href="http://cs231n.github.io/neural-networks-2/">CS231n</a> says that, <code>"dropout is implemented by only keeping a neuron active with some probability p (a hyperparameter), or setting it to zero otherwise."</code> Also you can see this from picture(Taken from the same site)
<a href="https://i.stack.imgur.com/SbXq1.jpg"><img src="https://i.stack.imgur.com/SbXq1.jpg" alt="enter image description here"></a></p>

<p>From tensorflow site, <code>With probability keep_prob, outputs the input element scaled up by 1 / keep_prob, otherwise outputs 0.</code></p>

<p>Now, why the input element is scaled up by <code>1/keep_prob</code>? Why not keep the input element as it is with probability and not scale it with <code>1/keep_prob</code>? </p>