How to estimate dataframe real size in pyspark?
<p>How to determine a dataframe size?</p>

<p>Right now I estimate the real size of a dataframe as follows:</p>

<pre><code>headers_size = key for key in df.first().asDict()
rows_size = df.map(lambda row: len(value for key, value in row.asDict()).sum()
total_size = headers_size + rows_size
</code></pre>

<p>It is too slow and I'm looking for a better way.</p>