Spark error: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
<p>I have a virtual machine in which a spark-2.0.0-bin-hadoop2.7 in standalone mode is installed.</p>

<p>I ran <code>./sbin/start-all.sh</code> to run the master and the slave.</p>

<p>When I do <code>./bin/spark-shell --master spark://192.168.43.27:7077 --driver-memory 600m --executor-memory 600m --executor-cores 1</code> in the machine itself the task's status is <code>RUNNING</code> and I am able to compute code in spark shell. </p>

<p><a href="https://i.stack.imgur.com/LbMSw.png" rel="noreferrer"><img src="https://i.stack.imgur.com/LbMSw.png" alt="ran spark shell in virtual machine"></a></p>

<p>When I do exactly the same command from another machine in the network, the status is "RUNNING" again, but the spark-shell throws <code>WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources</code>. I guess the problem is not directly related to resources because the same command works in the virtual machine itself, but not when it comes from other machines.</p>

<p><a href="https://i.stack.imgur.com/S5KlP.png" rel="noreferrer"><img src="https://i.stack.imgur.com/S5KlP.png" alt="ran spark shell through another machine in the network"></a></p>

<p>I checked most of the topics related to this error and none of them solved my problem. I even disabled firewall with <code>sudo ufw disable</code> just to make sure but no success (based on <a href="https://stackoverflow.com/questions/29469462/taskschedulerimpl-initial-job-has-not-accepted-any-resources">this link</a>) which suggests:</p>

<blockquote>
  <p>Disable Firewall on the client : This was the solution that worked for me. Since I was working on a prototype in-house code, I disabled the firewall on the client node. For some reason the worker nodes, were not able to talk back to the client for me. For production purposes, you would want to open-up certain number of ports required.</p>
</blockquote>