Size of numpy strided array/broadcast array in memory?
<p>I'm trying to create efficient broadcast arrays in numpy, e.g. a set of <code>shape=[1000,1000,1000]</code> arrays that have only 1000 elements, but repeated 1e6 times.  This can be achieved both through <code>np.lib.stride_tricks.as_strided</code> and <code>np.broadcast_arrays</code>.</p>

<p>However, I am having trouble verifying that there is no duplication in memory, and this is critical since tests that actually duplicate the arrays in memory tend to crash my machine leaving no traceback.</p>

<p>I've tried examining the size of the arrays using <code>.nbytes</code>, but that doesn't seem to correspond to the actual memory usage:</p>

<pre><code>&gt;&gt;&gt; import numpy as np
&gt;&gt;&gt; import resource
&gt;&gt;&gt; initial_memuse = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
&gt;&gt;&gt; pagesize = resource.getpagesize()
&gt;&gt;&gt;
&gt;&gt;&gt; x = np.arange(1000)
&gt;&gt;&gt; memuse_x = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
&gt;&gt;&gt; print("Size of x = {0} MB".format(x.nbytes/1e6))
Size of x = 0.008 MB
&gt;&gt;&gt; print("Memory used = {0} MB".format((memuse_x-initial_memuse)*resource.getpagesize()/1e6))
Memory used = 150.994944 MB
&gt;&gt;&gt;
&gt;&gt;&gt; y = np.lib.stride_tricks.as_strided(x, [1000,10,10], strides=x.strides + (0, 0))
&gt;&gt;&gt; memuse_y = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
&gt;&gt;&gt; print("Size of y = {0} MB".format(y.nbytes/1e6))
Size of y = 0.8 MB
&gt;&gt;&gt; print("Memory used = {0} MB".format((memuse_y-memuse_x)*resource.getpagesize()/1e6))
Memory used = 201.326592 MB
&gt;&gt;&gt;
&gt;&gt;&gt; z = np.lib.stride_tricks.as_strided(x, [1000,100,100], strides=x.strides + (0, 0))
&gt;&gt;&gt; memuse_z = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
&gt;&gt;&gt; print("Size of z = {0} MB".format(z.nbytes/1e6))
Size of z = 80.0 MB
&gt;&gt;&gt; print("Memory used = {0} MB".format((memuse_z-memuse_y)*resource.getpagesize()/1e6))
Memory used = 0.0 MB
</code></pre>

<p>So <code>.nbytes</code> reports the "theoretical" size of the array, but apparently not the actual size.  The <code>resource</code> checking is a little awkward, as it looks like there are some things being loaded &amp; cached (perhaps?) that result in the first striding taking up some amount of memory, but future strides take none.</p>

<p>tl;dr: How do you determine the actual size of a numpy array or array view in memory?</p>