{"action":"opened","issue":{"url":"https://api.github.com/repos/nchammas/flintrock/issues/43","labels_url":"https://api.github.com/repos/nchammas/flintrock/issues/43/labels{/name}","comments_url":"https://api.github.com/repos/nchammas/flintrock/issues/43/comments","events_url":"https://api.github.com/repos/nchammas/flintrock/issues/43/events","html_url":"https://github.com/nchammas/flintrock/issues/43","id":115231301,"number":43,"title":"Mystery: Why doesn't SPARK_MASTER_IP accept actual IP addresses?","user":{"login":"nchammas","id":1039369,"avatar_url":"https://avatars.githubusercontent.com/u/1039369?v=3","gravatar_id":"","url":"https://api.github.com/users/nchammas","html_url":"https://github.com/nchammas","followers_url":"https://api.github.com/users/nchammas/followers","following_url":"https://api.github.com/users/nchammas/following{/other_user}","gists_url":"https://api.github.com/users/nchammas/gists{/gist_id}","starred_url":"https://api.github.com/users/nchammas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nchammas/subscriptions","organizations_url":"https://api.github.com/users/nchammas/orgs","repos_url":"https://api.github.com/users/nchammas/repos","events_url":"https://api.github.com/users/nchammas/events{/privacy}","received_events_url":"https://api.github.com/users/nchammas/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-11-05T08:21:08Z","updated_at":"2015-11-05T08:21:08Z","closed_at":null,"body":"This is a mystery that someone can take on for fun or for glory.\r\n\r\nIf I change [these](https://github.com/nchammas/flintrock/blob/f662022024603199e25eb4c6f708c176170af1f6/flintrock.py#L601-L602) [two](https://github.com/nchammas/flintrock/blob/f662022024603199e25eb4c6f708c176170af1f6/flintrock.py#L1165-L1166) blocks of code from this:\r\n\r\n```python\r\nmaster_host=master_instance.public_dns_name,\r\nslave_hosts=[i.public_dns_name for i in slave_instances],\r\n```\r\n\r\nto this:\r\n\r\n```python\r\nmaster_host=master_instance.ip_address,\r\nslave_hosts=[i.ip_address for i in slave_instances],\r\n```\r\n\r\nthen Spark fails to launch. `master_host`, in particular, gets plugged into `SPARK_MASTER_IP` in [this template](https://github.com/nchammas/flintrock/blob/f662022024603199e25eb4c6f708c176170af1f6/templates/spark/conf/spark-env.sh#L11), which seems to set off the problem.\r\n\r\nFor whatever reason, DNS names work but IP addresses don't. I'm not sure why. Spark's documentation suggests that IP addresses [should](https://github.com/apache/spark/blob/188ea348fdcf877d86f3c433cd15f6468fe3b42a/conf/spark-env.sh.template#L49) [work](https://github.com/apache/spark/blob/f85aa06464a10f5d1563302fd76465dded475a12/docs/spark-standalone.md#cluster-launch-scripts).\r\n\r\nI've probably misunderstood something about how to configure Spark. Another possibility is that there is a documentation or code bug in Spark itself that needs to be fixed.\r\n\r\nOne clue I've come across but not tested out is the fact that [`SPARK_MASTER_HOST` is checked here](https://github.com/apache/spark/blob/f85aa06464a10f5d1563302fd76465dded475a12/core/src/main/scala/org/apache/spark/deploy/master/MasterArguments.scala#L33-L35), even though it is not mentioned anywhere else in the Spark codebase. I have a suspicion that `SPARK_MASTER_HOST` should instead be `SPARK_MASTER_IP`.\r\n\r\nWhat I can say for certain is that this file is where some master configurations get set, and I have traced code there from `start-master.sh`. So it's probably a good place to start digging."}}