Efficient way to load csv subset in R
<p>I have 350 csv files with between 10,000 and 700,000 rows in each. I would like to read a subset of each file into R. My method so far is:</p>

<pre><code>library(dplyr)

to_load &lt;- Sys.glob("data/*.csv")

data &lt;- data_frame(file = to_load) %&gt;%
           rowwise() %&gt;%
           do(read.csv(.$file) %&gt;% filter(condition == "a"))
</code></pre>

<p>When I try this out with just the first 6 files, the estimated completion time from <code>do()</code> is 3 minutes, which will average out to about 3 hours in total. My question is whether there is a more efficient approach to this. I'm open to trying just about anything.</p>