Spark write to parquet on hdfs
<p>I have 3 nodes hadoop and spark installed. I would like to take data from rdbms into data frame and write this data into parquet on HDFS. "dfs.replication" value is 1 . </p>

<p>When i try this with following command i have seen all HDFS blocks are located on node which i executed spark-shell.</p>

<pre><code>scala&gt; xfact.write.parquet("hdfs://sparknode01.localdomain:9000/xfact")
</code></pre>

<p>Is this the intended behaviour or should all blocks be distributed across the cluster?</p>

<p>Thanks</p>