Failed to write large amount of data to stream
<p>When I'm trying to write very large amount of data (list with 300 000 rows and more) to memory stream using <a href="https://github.com/JoshClose/CsvHelper" rel="noreferrer">CsvHelper</a>, it throws the exception <strong>"System.IO.IOException: Stream was too long."</strong>. </p>

<p>Data class is rather big and has ~30 properties, consequently each record in the file would have ~30 columns. </p>

<p>This is the actual writing code where exception throws (by the way this code is based on <a href="https://stackoverflow.com/a/22997765/3006185">that</a> answer of CsvHelper lib's author):</p>

<pre><code>using (var memoryStream = new MemoryStream())
{
    using (var streamWriter = new StreamWriter(memoryStream, encoding ?? Encoding.ASCII))
    {
        var csvWriter = new CsvWriter(streamWriter, GetConfiguration(delimiter, mappingClassType, mappingActions));
        csvWriter.WriteRecords(data); //data is IEnumerable&lt;T&gt; and has more than 300k records

        streamWriter.Flush();
        return memoryStream.ToArray();
    }
}
</code></pre>

<p>Then I save the resulted bytes array into the file.</p>

<pre><code>File.WriteAllBytes(filePath, resultedBytesArray); 
</code></pre>

<p>Please note, that <strong>the same code works well when I write 100 000 records</strong> to the file (in that case the file has size about 1GB). By the way, my goal is to write more then 600 000 data records. </p>

<p>This is the relevant part of the stack trace related to this issue. </p>

<pre><code>Stream was too long.|System.IO.IOException: Stream was too long.
at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count) 
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder) 
at System.IO.StreamWriter.Write(Char[] buffer, Int32 index, Int32 count) 
at CsvHelper.CsvWriter.NextRecord() in C:\Users\Josh\Projects\CsvHelper\src\CsvHelper\CsvWriter.cs:line 290 
at CsvHelper.CsvWriter.WriteRecords(IEnumerable records) in C:\Users\Josh\Projects\CsvHelper\src\CsvHelper\CsvWriter.cs:line 490 
at FileExport.Csv.CsvDocument.Create[T](IEnumerable`1 data, String delimiter, Encoding encoding, Type mappingClassType, IDictionary`2 mappingActions) in d:\Dev\DrugDevExport\FileExport\Csv\CsvDocument.cs:line 33 
</code></pre>

<p>As far as I'm concerned the basic way to achieve my goal and avoid that issue is to split my list of written data up on few parts and concatenate them together then, but may be is there any pretty obvious and easy solution without a significant code refactoring (like increase the default stream/buffer size, etc..)? </p>

<p>Also keep in mind, that I've also applied two possible solutions in order to prevent "Out Of Memory" objects exception. </p>

<ul>
<li>got rid of 2GB limitation for objects (from here <a href="https://stackoverflow.com/a/20912869">https://stackoverflow.com/a/20912869</a>) Yes, I'm running on x64 OS with 32GB RAM. </li>
<li>set up x64 "Platform target" in the build settings section (from here <a href="https://stackoverflow.com/a/22592876">https://stackoverflow.com/a/22592876</a>) </li>
</ul>

<p>Thanks in advance. </p>