CsvHelper - Reading Stream Asynchronously
<p>I have a service that takes an input Stream containing CSV data that needs to be bulk-inserted into a database, and my application is using async/await wherever possible.</p>
<p>The process is: Parse stream using CsvHelper's CsvParser, add each row to DataTable, use SqlBulkCopy to copy the DataTable to the database.</p>
<p>The data could be any size so I'd like to avoid reading the whole thing into memory at one time - obviously I'll have all that data in the DataTable by the end anyway so would essentially have 2 copies in memory.</p>
<p>I would like to do all of this as asynchronously as possible, but CsvHelper doesn't have any async methods so I've come up with the following workaround:</p>
<pre><code>using (var inputStreamReader = new StreamReader(inputStream))
{
    while (!inputStreamReader.EndOfStream)
    {
        // Read line from the input stream
        string line = await inputStreamReader.ReadLineAsync();

        using (var memoryStream = new MemoryStream())
        using (var streamWriter = new StreamWriter(memoryStream))
        using (var memoryStreamReader = new StreamReader(memoryStream))
        using (var csvParser = new CsvParser(memoryStreamReader))
        {
            await streamWriter.WriteLineAsync(line);
            await streamWriter.FlushAsync();

            memoryStream.Position = 0;

            // Loop through all the rows (should only be one as we only read a single line...)
            while (true)
            {
                var row = csvParser.Read();

                // No more rows to process
                if (row == null)
                {
                    break;
                }

                // Add row to DataTable
            }
        }
    }
}
</code></pre>
<p>Are there any issues with this solution? Is it even necessary? I've seen that the CsvHelper devs specifically did not add async functionality (<a href="https://github.com/JoshClose/CsvHelper/issues/202" rel="nofollow noreferrer">https://github.com/JoshClose/CsvHelper/issues/202</a>) but I don't really follow the reasoning behind not doing so.</p>
<p>EDIT: I've just realised that this solution isn't going to work for instances where a column contains a line break anyway :( Guess I'll just have to copy the whole input Stream to a MemoryStream or something</p>
<p>EDIT2: Some more information.</p>
<p>This is in an async method in a library where I am trying to do async all the way down. It'll likely be consumed by an MVC controller (if I just wanted to offload it from a UI thread I would just Task.Run() it). Mostly the method will be waiting on external sources such as a database / DFS, and I would like for the thread to be freed while it is.</p>
<p>CsvParser.Read() is going to block even if what's blocking is reading the Stream (e.g. if the data I'm attempting to read resides on a server on the other side of the world), whereas if CsvHelper were to implement an async method that uses TextReader.ReadAsync(), then I wouldn't be blocked waiting for my data to arrive from Dubai. As far as I can tell, I'm not asking for an async wrapper around a synchronous method.</p>
<p>EDIT3: Update from far in the future! Async functionality was actually added to CsvHelper back in 2017. I hope someone at the company I was working for has upgraded to the newer version since then!</p>