Session.close() doesn't free resources on GPU using tensorflow.
<p>I would like to perform pretraining of neural network using autoencoders implemented in TensorFlow. </p>

<ol>
<li>I am able to run whole network. (Using TF or Keras). the whole graph fits into GPU memory so that's fine. </li>
<li>Problem occurs when I create more graphs (autoencoders). GPU run out of memory very quickly. Right now I have example where building second level autoencoder causes GPU out of mem. exception. </li>
</ol>

<p>So what is happening: </p>

<p>I have implementation of autoencoders which has session as it's attribute, so : </p>

<pre><code>self.session = tf.Session() 
</code></pre>

<p>and implements method </p>

<pre><code>destroy()
</code></pre>

<p>where </p>

<pre><code>self.session.close()
</code></pre>

<p>is called. </p>

<p>When stacking autoencoders, some instances of Session are needed and that's when I got the problem. </p>

<p>What am I missing? isn't .close() enough?</p>

<p>Thanks  </p>