Why is writing to a file faster than appending a string?
<p>Given that RAM is much faster than a hard drive, I was surprised by the code below.</p>

<p>I was trying to split a CSV files based on the value of one column, and write each line with different values in that cell to different files.</p>

<p>I was trying:</p>

<pre><code>List&lt;string&gt; protocolTypes = new List&lt;string&gt;();
List&lt;string&gt; splitByProtocol = new List&lt;string&gt;();
foreach (string s in lineSplit)
{
    string protocol = getProtocol();
    index = protocolTypes.IndexOf(protocol);
    splitByProtocol[index] = splitByProtocol[index] + s + "\n";
}
</code></pre>

<p>Which took ages, but changing it to a stream writer was much faster:</p>

<pre><code>List&lt;string&gt; protocolTypes = new List&lt;string&gt;();
List&lt;StreamWriter&gt; splitByProtocol = new List&lt;StreamWriter&gt;();
foreach (string s in lineSplit)
{
    string protocol = getProtocol();
    index = protocolTypes.IndexOf(protocol);
    splitByProtocol[index].WriteLine(s);
}
</code></pre>

<p>Why is writing to disk so much faster than appending strings together in memory? I know adding to a string requires copying the whole string to a new memory location, but appending a string was orders of magnitude slower than writing to disk which seems counter intuitive.</p>