Does Spark do UnionAll in parallel?
<p>I got 10 <code>DataFrame</code>s with the same schema which I'd like to combine into one <code>DataFrame</code>. Each <code>DataFrame</code> is constructed using a <code>sqlContext.sql("select ... from ...").cahce</code>, which means that technically, the <code>DataFrame</code>s are not really calculated until it's time to use them.</p>

<p>So, if I run:</p>

<pre><code>val df_final = df1.unionAll(df2).unionAll(df3).unionAll(df4) ...
</code></pre>

<p>will Spark calculate all these <code>DataFrame</code>s in parallel or one by one (due to the dot operator)?</p>

<p>And also, while we're here - is there a more elegant way to preform a <code>unionAll</code> on several <code>DataFrame</code>s than the one I listed above?</p>