How to deal with GPU memory leakage issues in Torch?
<p>My machine's GPU has 2 GBs of memory. When I run the following code for the first time, I get no errors. However, the second time I run the code I get memory error. As a short-time remedy, the only thing I can do is to cast the data into float32 using <code>torch.Tensor.float()</code>. However, the problem still persists and the occupied memory is not released after the process is done, or the process is terminated while running. This is the case for the machine RAM as well. How should one prevent memory leakage in Torch or release the memory?</p>

<pre><code>require 'nn'
require 'image'
require 'cunn'
require 'paths'



collectgarbage(); collectgarbage()
if (not paths.filep("cifar10torchsmall.zip")) then
    os.execute('wget -c https://s3.amazonaws.com/torch7/data/cifar10torchsmall.zip')
    os.execute('unzip cifar10torchsmall.zip')
end
trainset = torch.load('cifar10-train.t7')
testset = torch.load('cifar10-test.t7')
classes = {'airplane', 'automobile', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse', 'ship', 'truck'}

setmetatable(trainset, 
    {__index = function(t, i) 
                    return {t.data[i], t.label[i]} 
                end}
);
trainset.data = trainset.data:double() -- convert the data from a ByteTensor to a DoubleTensor.

function trainset:size() 
    return self.data:size(1) 
end

mean = {} -- store the mean, to normalize the test set in the future
stdv  = {} -- store the standard-deviation for the future
for i=1,3 do -- over each image channel
    mean[i] = trainset.data[{ {}, {i}, {}, {}  }]:mean() -- mean estimation
    print('Channel ' .. i .. ', Mean: ' .. mean[i])
    trainset.data[{ {}, {i}, {}, {}  }]:add(-mean[i]) -- mean subtraction

    stdv[i] = trainset.data[{ {}, {i}, {}, {}  }]:std() -- std estimation
    print('Channel ' .. i .. ', Standard Deviation: ' .. stdv[i])
    trainset.data[{ {}, {i}, {}, {}  }]:div(stdv[i]) -- std scaling
end


testset.data = testset.data:double()   -- convert from Byte tensor to Double tensor
for i=1,3 do -- over each image channel
    testset.data[{ {}, {i}, {}, {}  }]:add(-mean[i]) -- mean subtraction    
    testset.data[{ {}, {i}, {}, {}  }]:div(stdv[i]) -- std scaling
end

trainset.data = trainset.data:cuda()
testset.data = testset.data:cuda()

net = nn.Sequential()
net:add(nn.SpatialConvolution(3, 6, 5, 5)) -- 3 input image channels, 6 output channels, 5x5 convolution kernel
net:add(nn.ReLU())                       -- non-linearity 
net:add(nn.SpatialMaxPooling(2,2,2,2))     -- A max-pooling operation that looks at 2x2 windows and finds the max.
net:add(nn.SpatialConvolution(6, 16, 5, 5))
net:add(nn.ReLU())                       -- non-linearity 
net:add(nn.SpatialMaxPooling(2,2,2,2))
net:add(nn.View(16*5*5))                    -- reshapes from a 3D tensor of 16x5x5 into 1D tensor of 16*5*5
net:add(nn.Linear(16*5*5, 120))             -- fully connected layer (matrix multiplication between input and weights)
net:add(nn.ReLU())                       -- non-linearity 
net:add(nn.Linear(120, 84))
net:add(nn.ReLU())                       -- non-linearity 
net:add(nn.Linear(84, 10))                   -- 10 is the number of outputs of the network (in this case, 10 digits)
net:add(nn.LogSoftMax())  
net = net:cuda()

criterion = nn.ClassNLLCriterion()
criterion = criterion:cuda()



pred = net:forward(trainset.data)
outputEr = criterion:forward(pred, trainset.label:cuda())
net:zeroGradParameters()
outputGrad = criterion:backward(pred, trainset.label:cuda())
collectgarbage()
inputGrad = net:backward(trainset.data, outputGrad)
</code></pre>

<p>Side question: Why Torch initializes the network parameters as double although GPUs are pretty slow at computing double-precision operations and there is actually no need of having 64-bit parameter values for almost all neural network applications? How can I initialize a model with float (32-bit) parameter vectors?</p>

<p>I found the answer to the side question. You can easily make torch's default data type as float using the following at the beginning of your code:</p>

<pre><code>torch.setdefaulttensortype('torch.FloatTensor')
</code></pre>