{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/5478","labels_url":"https://api.github.com/repos/apache/spark/issues/5478/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/5478/comments","events_url":"https://api.github.com/repos/apache/spark/issues/5478/events","html_url":"https://github.com/apache/spark/pull/5478","id":67885038,"number":5478,"title":"[SPARK-6869][PySpark] Add pyspark archives path to PYTHONPATH","user":{"login":"Sephiroth-Lin","id":10866770,"avatar_url":"https://avatars.githubusercontent.com/u/10866770?v=3","gravatar_id":"","url":"https://api.github.com/users/Sephiroth-Lin","html_url":"https://github.com/Sephiroth-Lin","followers_url":"https://api.github.com/users/Sephiroth-Lin/followers","following_url":"https://api.github.com/users/Sephiroth-Lin/following{/other_user}","gists_url":"https://api.github.com/users/Sephiroth-Lin/gists{/gist_id}","starred_url":"https://api.github.com/users/Sephiroth-Lin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Sephiroth-Lin/subscriptions","organizations_url":"https://api.github.com/users/Sephiroth-Lin/orgs","repos_url":"https://api.github.com/users/Sephiroth-Lin/repos","events_url":"https://api.github.com/users/Sephiroth-Lin/events{/privacy}","received_events_url":"https://api.github.com/users/Sephiroth-Lin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":20,"created_at":"2015-04-12T08:05:32Z","updated_at":"2015-04-16T12:43:16Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/5478","html_url":"https://github.com/apache/spark/pull/5478","diff_url":"https://github.com/apache/spark/pull/5478.diff","patch_url":"https://github.com/apache/spark/pull/5478.patch"},"body":"From SPARK-1920 and SPARK-1520 we know PySpark on Yarn can not work when the assembly jar are package by JDK 1.7+, so ship pyspark archives to executors by Yarn with --py-files. The pyspark  archives name must contains \"spark-pyspark\"."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/93724717","html_url":"https://github.com/apache/spark/pull/5478#issuecomment-93724717","issue_url":"https://api.github.com/repos/apache/spark/issues/5478","id":93724717,"user":{"login":"Sephiroth-Lin","id":10866770,"avatar_url":"https://avatars.githubusercontent.com/u/10866770?v=3","gravatar_id":"","url":"https://api.github.com/users/Sephiroth-Lin","html_url":"https://github.com/Sephiroth-Lin","followers_url":"https://api.github.com/users/Sephiroth-Lin/followers","following_url":"https://api.github.com/users/Sephiroth-Lin/following{/other_user}","gists_url":"https://api.github.com/users/Sephiroth-Lin/gists{/gist_id}","starred_url":"https://api.github.com/users/Sephiroth-Lin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Sephiroth-Lin/subscriptions","organizations_url":"https://api.github.com/users/Sephiroth-Lin/orgs","repos_url":"https://api.github.com/users/Sephiroth-Lin/repos","events_url":"https://api.github.com/users/Sephiroth-Lin/events{/privacy}","received_events_url":"https://api.github.com/users/Sephiroth-Lin/received_events","type":"User","site_admin":false},"created_at":"2015-04-16T12:43:16Z","updated_at":"2015-04-16T12:43:16Z","body":"@andrewor14 @sryza @WangTaoTheTonic As I have test again, if we install Spark on each node, then we can set spark.executorEnv.PYTHONPATH=${SPARK_HOME}/python:${SPARK_HOME}/python/lib/py4j-0.8.2.1-src.zip to pass PYTHONPATH to executor. So this PR is another solution to run PySpark on yan if we don't install Spark on each node."}}