unable to complete promises due to out of memory
<p>I have a script to scrape ~1000 webpages. I'm using Promise.all to fire them together, and it returns when all pages are done:</p>

<pre><code>Promise.all(urls.map(url =&gt; scrap(url)))
    .then(results =&gt; console.log('all done!', results));
</code></pre>

<p>This is sweet and correct, except for one thing - machine goes <strong>out of memory</strong> because of the concurrent requests. I'm use <code>jsdom</code> for scrapping, it quickly takes up a few GB of mem, which is understandable considering it instantiates hundreds of <code>window</code>.</p>

<p>I have an idea to fix but I don't like it. That is, change control flow to not use Promise.all, but chain my promises:</p>

<pre><code>let results = {};
urls.reduce((prev, cur) =&gt;
    prev
        .then(() =&gt; scrap(cur))
        .then(result =&gt; results[cur] = result)
        // ^ not so nice. 
, Promise.resolve())
    .then(() =&gt; console.log('all done!', results));
</code></pre>

<p>This is not as good as Promise.all... Not performant as it's chained, and returned values have to be stored for later processing.</p>

<p>Any suggestions? Should I improve the control flow or should I improve mem usage in scrap(), or is there a way to let node throttle mem allocation?</p>