What's the best way to overfit a neural network?
<p>I've been playing with this online demo of a in-browser <a href="https://cs.stanford.edu/people/karpathy/convnetjs/demo/image_regression.html" rel="nofollow noreferrer">https://cs.stanford.edu/people/karpathy/convnetjs/demo/image_regression.html</a> and I've been trying to make the image with text (the one with "What does the fox say..." caption) to appear more clearer (with readable text) to no avail. </p>

<p>This neural network receives two inputs - X and Y positions of a pixel and three outputs - RGB channels for said pixel. The network is trained by a selected picture and then tries to "draw" it.</p>

<p>I would like to train this network in a way that would make each pixel match as close as possible to the original image, i.e. make it overfit. How can this be achieved? What combination or parameters should be used? Maybe the structure of the networks needs to be different (more layers, more neurons in layers, different type of layers, etc.)?</p>

<p>I've tried playing with the learning rate without changing the structure from the original, but even after 400k iterations most of the text is indistinguishable: </p>

<p><a href="https://i.stack.imgur.com/v8jjH.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/v8jjH.png" alt="screenshot after 400k iterations"></a></p>

<p>Is it just a matter of time and maybe after a few millions iterations the desired result will be achieved? Can it be sped up somehow (without increasing the processing power and other obvious stuff)? </p>

<p>It seems that making it return the same value for the same inputs every time should be a really easy task - the network in being trained on the exact same data hundreds of thousand of times, so why does it still return so many gray pixels?</p>

<p>I'm new to neural networks, so maybe it's a stupid question and neural networks can't be used for that at all.</p>