BeautifulSoup `find_all` generator
<p>Is there any way to turn <code>find_all</code> into a more memory efficient generator? For example:  </p>

<p>Given:</p>

<pre><code>soup = BeautifulSoup(content, "html.parser")
return soup.find_all('item')
</code></pre>

<p>I would like to instead use:  </p>

<pre><code>soup = BeautifulSoup(content, "html.parser")
while True:
    yield soup.next_item_generator()
</code></pre>

<p>(assume proper handing of the final <code>StopIteration</code> exception)</p>

<p>There are some generators built in, but not to yield the next result in a find. <code>find</code> returns just the first item. With thousands of items, <code>find_all</code> sucks up a <em>lot</em> of memory. For 5792 items, I'm seeing a spike of just over 1GB of RAM.</p>

<p>I am well aware that there are more efficient parsers, such as lxml, that can accomplish this. Let's assume that there are other business constraints preventing me from using anything else.</p>

<p>How can I turn <code>find_all</code> into a generator to iterate through in a more <em>memory efficient</em> way.</p>