Joining strings. Generator or list comprehension?
<p>Consider the problem of extracting alphabets from a huge string. </p>

<p>One way to do is</p>

<pre><code>''.join([c for c in hugestring if c.isalpha()])
</code></pre>

<p>The mechanism is clear: The list comprehension generates a list of characters. The join method knows how many characters it needs to join by accessing the length of the list.</p>

<p>Other way to do is</p>

<pre><code>''.join(c for c in hugestring if c.isalpha())
</code></pre>

<p>Here the generator comprehension results in a generator. The join method does not know how many characters it is going to join because the generator does not possess <strong>len</strong> attribute. So this way of joining should be slower than the list comprehension method.</p>

<p>But testing in python shows that it is not slower. Why is this so?
Can anyone explain how join works on a generator.</p>

<p>To be clear: </p>

<pre><code>sum(j for j in range(100))
</code></pre>

<p>doesn't need to have any knowledge of 100 because it can keep track of the cumulative sum. It can access the next element using the next method on the generator and then add to the cumulative sum.
However, since strings are immutable, joining strings cumulatively would create a new string in each iteration. So this would take lot of time.</p>