How can I train neural network with more data than can fit the memory?
<p>Trying to train a classifier with a recurrent layer using a lot of data. As a result, all data can not fit into the memory. It provides me with the following error:</p>

<pre><code>Error using zeros
Requested 1x2114046976 (15.8GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long
time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.
Error in nnMex.perfsGrad (line 3)
TEMP = zeros(1,ceil(hints.tempSizeBG/8)*8);
Error in nnCalcLib/perfsGrad (line 294)
                lib.calcMode.perfsGrad(calcNet,lib.calcData,lib.calcHints);
Error in trainscg&gt;initializeTraining (line 153)
[worker.perf,worker.vperf,worker.tperf,worker.gWB,worker.gradient] = calcLib.perfsGrad(calcNet);
Error in nnet.train.trainNetwork&gt;trainNetworkInMainThread (line 28)
worker = localFcns.initializeTraining(archNet,calcLib,calcNet,tr);
Error in nnet.train.trainNetwork (line 16)
    [archNet,tr] = trainNetworkInMainThread(archNet,rawData,calcLib,calcNet,tr,feedback,localFcns);
Error in trainscg&gt;train_network (line 147)
[archNet,tr] = nnet.train.trainNetwork(archNet,rawData,calcLib,calcNet,tr,localfunctions);
Error in trainscg (line 59)
            [out1,out2] = train_network(varargin{2:end});
Error in network/train (line 369)
    [net,tr] = feval(trainFcn,'apply',net,data,calcLib,calcNet,tr);
</code></pre>

<p>It should be noted that currently my training input is 11x52266 and the network has ~3k weight elements due to the recurrent layer. I would like, however, to provide 15 times as much data for training.</p>

<p>How can I cope? Are there any techniques to map the local variable it's trying to initialize on my SSD instead of memory?</p>

<p>There is the "reduction" option for training, but it does not seem to do any difference on this matter. The same error occurs regardless.</p>