Memory error on large Shapefile in Python
<pre><code>import shapefile
data = shapefile.Reader("data_file.shp")
shapes = data.shapes()
</code></pre>

<p>My problem is that getting the shapes from the Shapefile reader gives me an exception <code>MemoryError</code> when using <a href="https://pypi.python.org/pypi/pyshp" rel="nofollow">Pyshp</a>.</p>

<p>The <code>.shp</code> file is quite large, at 1.2 gB. But I am using ony 3% of my machine's 32gB, so I don't understand it.</p>

<p>Is there any other approach that I can take? Can process the file in chunks in Python? Or use some tool to spilt the file into chinks, then process each of them individually?</p>