Apache Spark accessing the data in hdfs through cross cluster
<p>I am running Spark on Amazon EMR whose Public DNS is, lets say, <code>23.21.40.15</code>.</p>

<p>Now I am executing my Spark Jar on this cluster &amp; I want to write the output of my Spark Job to other Amazon EMR HDFS whose Public DNS is <code>29.45.56.72</code>.</p>

<p>I am able to access my own cluster HDFS i.e. <code>23.21.40.15</code> but I am not able to write to the cluster <code>29.45.56.72</code>.</p>

<ul>
<li>What do I need to do so that my spark job can access cross cluster
HDFS??</li>
<li>If possible, can anyone share a sample code for this??</li>
</ul>