Yarn keeps on killing Spark Application master on EMR
<p>I'm running a spark application on  EMR emr-4.3.0 with 1 Master and 4 Nodes</p>

<p>Each one of them has 5GB memory and 2 cores.</p>

<p>The following are my spark-submit options</p>

<blockquote>
  <p>--class com.mobi.vserv.driver.Query5kPids1<br>
  --num-executors 4<br>
  --executor-memory 4g<br>
  --executor-cores 2<br>
  --driver-memory 4g</p>
</blockquote>

<p>But I keep getting the following error </p>

<blockquote>
  <p>ERROR executor.CoarseGrainedExecutorBackend: Driver
  10.225.19.144:56334 disassociated! Shutting down.</p>
</blockquote>

<p>And finally Yarn Kills the Application master</p>

<blockquote>
  <p>ERROR ApplicationMaster: RECEIVED SIGNAL 15: SIGTERM</p>
</blockquote>

<p>1) Can I further Improve my spark-submit options for num-executors and executor-cores.</p>

<p>2) I have seen in the Perfomance Graph only 2 nodes CPU's utilization is above 50 % , while the other 2 has less than 5 %.</p>

<p>3) Interestingly , I have ran the same application on 2 nodes and 1 master with the same configurations given in Spark-submit and the application ran successfully. So whats the reason of this behaviour with 4 nodes </p>