{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/8349","labels_url":"https://api.github.com/repos/apache/spark/issues/8349/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/8349/comments","events_url":"https://api.github.com/repos/apache/spark/issues/8349/events","html_url":"https://github.com/apache/spark/pull/8349","id":102305343,"number":8349,"title":"[SPARK-9669][MESOS] Support PySpark on Mesos cluster mode.","user":{"login":"tnachen","id":67244,"avatar_url":"https://avatars.githubusercontent.com/u/67244?v=3","gravatar_id":"","url":"https://api.github.com/users/tnachen","html_url":"https://github.com/tnachen","followers_url":"https://api.github.com/users/tnachen/followers","following_url":"https://api.github.com/users/tnachen/following{/other_user}","gists_url":"https://api.github.com/users/tnachen/gists{/gist_id}","starred_url":"https://api.github.com/users/tnachen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tnachen/subscriptions","organizations_url":"https://api.github.com/users/tnachen/orgs","repos_url":"https://api.github.com/users/tnachen/repos","events_url":"https://api.github.com/users/tnachen/events{/privacy}","received_events_url":"https://api.github.com/users/tnachen/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":11,"created_at":"2015-08-21T05:50:37Z","updated_at":"2015-09-15T08:07:24Z","closed_at":"2015-09-04T22:22:56Z","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/8349","html_url":"https://github.com/apache/spark/pull/8349","diff_url":"https://github.com/apache/spark/pull/8349.diff","patch_url":"https://github.com/apache/spark/pull/8349.patch"},"body":"Support running pyspark with cluster mode on Mesos!\r\nThis doesn't upload any scripts, so if running in a remote Mesos requires the user to specify the script from a available URI."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/140315329","html_url":"https://github.com/apache/spark/pull/8349#issuecomment-140315329","issue_url":"https://api.github.com/repos/apache/spark/issues/8349","id":140315329,"user":{"login":"viesti","id":57011,"avatar_url":"https://avatars.githubusercontent.com/u/57011?v=3","gravatar_id":"","url":"https://api.github.com/users/viesti","html_url":"https://github.com/viesti","followers_url":"https://api.github.com/users/viesti/followers","following_url":"https://api.github.com/users/viesti/following{/other_user}","gists_url":"https://api.github.com/users/viesti/gists{/gist_id}","starred_url":"https://api.github.com/users/viesti/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/viesti/subscriptions","organizations_url":"https://api.github.com/users/viesti/orgs","repos_url":"https://api.github.com/users/viesti/repos","events_url":"https://api.github.com/users/viesti/events{/privacy}","received_events_url":"https://api.github.com/users/viesti/received_events","type":"User","site_admin":false},"created_at":"2015-09-15T08:07:23Z","updated_at":"2015-09-15T08:07:23Z","body":"Trying this out by compiling current master (at the time of writing `a2249359d5b0368318a714b292bb1d0dc70c0e27`) with mesos 0.23.0 and 0.24.0 and I got the following error:\r\n\r\n```\r\n0% ./bin/spark-submit --master mesos://localhost:5050 --deploy-mode cluster test.py\r\nUsing Spark's default log4j profile: org/apache/spark/log4j-defaults.properties\r\n15/09/15 10:54:31 INFO RestSubmissionClient: Submitting a request to launch an application in mesos://localhost:5050.\r\nException in thread \"main\" org.apache.spark.deploy.rest.SubmitRestProtocolException: Malformed response received from server\r\n\tat org.apache.spark.deploy.rest.RestSubmissionClient.readResponse(RestSubmissionClient.scala:258)\r\n\tat org.apache.spark.deploy.rest.RestSubmissionClient.org$apache$spark$deploy$rest$RestSubmissionClient$$postJson(RestSubmissionClient.scala:219)\r\n\tat org.apache.spark.deploy.rest.RestSubmissionClient$$anonfun$createSubmission$3.apply(RestSubmissionClient.scala:84)\r\n\tat org.apache.spark.deploy.rest.RestSubmissionClient$$anonfun$createSubmission$3.apply(RestSubmissionClient.scala:80)\r\n\tat scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772)\r\n\tat scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)\r\n\tat scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)\r\n\tat scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771)\r\n\tat org.apache.spark.deploy.rest.RestSubmissionClient.createSubmission(RestSubmissionClient.scala:80)\r\n\tat org.apache.spark.deploy.rest.RestSubmissionClient$.run(RestSubmissionClient.scala:403)\r\n\tat org.apache.spark.deploy.rest.RestSubmissionClient$.main(RestSubmissionClient.scala:416)\r\n\tat org.apache.spark.deploy.rest.RestSubmissionClient.main(RestSubmissionClient.scala)\r\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\r\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\r\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\r\n\tat java.lang.reflect.Method.invoke(Method.java:483)\r\n\tat org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:680)\r\n\tat org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)\r\n\tat org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)\r\n\tat org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120)\r\n\tat org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)\r\nCaused by: org.apache.spark.deploy.rest.SubmitRestProtocolException: Server returned empty body\r\n\tat org.apache.spark.deploy.rest.RestSubmissionClient.readResponse(RestSubmissionClient.scala:237)\r\n\t... 20 more\r\n```\r\n\r\nCapturing traffic to my local mesos show that spark-submit tries to `POST /v1/submissions/create` to which mesos responds with `HTTP/1.1 404`. So I'm probably doing something categorically wrong here :) Any pointers to help would make me and my colleague glad :)"}}