Inputs and Outputs for Neural Turing Machine Copy Task
<p>I'm trying to understand the copy task from the NTM <a href="https://arxiv.org/pdf/1410.5401.pdf" rel="nofollow noreferrer">paper</a> by Graves et al.</p>

<p>I have experience in language modeling with LSTMs and there the network is usually fed a sequence of words, one word at a time, and the output for each timestep is the predicted next word.</p>

<p>For the copy task of NTMs however, the output seems to be delayed (which is the whole point I guess):</p>

<p><a href="https://i.stack.imgur.com/v13pz.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/v13pz.png" alt="enter image description here"></a><br>
Source: <a href="https://blog.wtf.sg/2014/11/11/neural-turing-machines-copy-task/" rel="nofollow noreferrer">https://blog.wtf.sg/2014/11/11/neural-turing-machines-copy-task/</a></p>

<p>So exactly how does this work in code during training? Are the true output vectors for the first half and input vectors during the second half of the sequence just zero vectors, with the network being expected to output zero vectors for the firs half and then the correct sequence during the second half?</p>

<p>That part has me confused.</p>