R data.frame memory usage -- especially after subsetting
<p>A colleague of mine is interested in how much overhead (in terms of memory) is in an R <code>data.frame</code>. He uses the following example:</p>

<pre><code>n = 1e6
df = data.frame(v1=rnorm(n), v2 = rnorm(n))
object.size(df)-sum(sapply(df, object.size)) # some overhead in the df

#&gt;696 bytes
</code></pre>

<p>All well and good. Now, let's take a random subset of this <code>data.frame</code></p>

<pre><code>idx = as.logical(rbinom(n, size = 1, prob = 0.99))
df0 = df[idx,]
</code></pre>

<p>So - the overhead in <code>df0</code> should be the same as <code>df</code> right?</p>

<pre><code>object.size(df0) - sum(sapply(df0, object.size))
#&gt; 3961136 bytes
</code></pre>

<p>To quote the Grinch "Wrongo!" It seems to me that information about the subsetting variable is being stored here, because if I change this to:</p>

<pre><code>object.size(df0) - sum(sapply(df0, object.size)) - object.size(idx[idx])
</code></pre>

<p>Then I get <code>648 bytes</code> which is almost right. However, I cannot see where this information is being stored. </p>