{"action":"created","issue":{"url":"https://api.github.com/repos/apache/incubator-zeppelin/issues/264","labels_url":"https://api.github.com/repos/apache/incubator-zeppelin/issues/264/labels{/name}","comments_url":"https://api.github.com/repos/apache/incubator-zeppelin/issues/264/comments","events_url":"https://api.github.com/repos/apache/incubator-zeppelin/issues/264/events","html_url":"https://github.com/apache/incubator-zeppelin/pull/264","id":103774036,"number":264,"title":"ZEPPELIN-270 added env varibles for pyspark's correct function in yarn.","user":{"login":"riverajo","id":2325467,"avatar_url":"https://avatars.githubusercontent.com/u/2325467?v=3","gravatar_id":"","url":"https://api.github.com/users/riverajo","html_url":"https://github.com/riverajo","followers_url":"https://api.github.com/users/riverajo/followers","following_url":"https://api.github.com/users/riverajo/following{/other_user}","gists_url":"https://api.github.com/users/riverajo/gists{/gist_id}","starred_url":"https://api.github.com/users/riverajo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/riverajo/subscriptions","organizations_url":"https://api.github.com/users/riverajo/orgs","repos_url":"https://api.github.com/users/riverajo/repos","events_url":"https://api.github.com/users/riverajo/events{/privacy}","received_events_url":"https://api.github.com/users/riverajo/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-08-28T18:41:45Z","updated_at":"2015-08-29T02:28:59Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/incubator-zeppelin/pulls/264","html_url":"https://github.com/apache/incubator-zeppelin/pull/264","diff_url":"https://github.com/apache/incubator-zeppelin/pull/264.diff","patch_url":"https://github.com/apache/incubator-zeppelin/pull/264.patch"},"body":"Simple doc fix for pyspark in yarn."},"comment":{"url":"https://api.github.com/repos/apache/incubator-zeppelin/issues/comments/135931941","html_url":"https://github.com/apache/incubator-zeppelin/pull/264#issuecomment-135931941","issue_url":"https://api.github.com/repos/apache/incubator-zeppelin/issues/264","id":135931941,"user":{"login":"felixcheung","id":8969467,"avatar_url":"https://avatars.githubusercontent.com/u/8969467?v=3","gravatar_id":"","url":"https://api.github.com/users/felixcheung","html_url":"https://github.com/felixcheung","followers_url":"https://api.github.com/users/felixcheung/followers","following_url":"https://api.github.com/users/felixcheung/following{/other_user}","gists_url":"https://api.github.com/users/felixcheung/gists{/gist_id}","starred_url":"https://api.github.com/users/felixcheung/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/felixcheung/subscriptions","organizations_url":"https://api.github.com/users/felixcheung/orgs","repos_url":"https://api.github.com/users/felixcheung/repos","events_url":"https://api.github.com/users/felixcheung/events{/privacy}","received_events_url":"https://api.github.com/users/felixcheung/received_events","type":"User","site_admin":false},"created_at":"2015-08-29T02:28:59Z","updated_at":"2015-08-29T02:28:59Z","body":"Hmm, should we use SparkConf instead then:\r\n\r\nhttps://github.com/apache/spark/blob/69c9c177160e32a2fbc9b36ecc52156077fca6fc/yarn/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala\r\n```\r\n    // Keep this for backwards compatibility but users should move to the config\r\n    sys.env.get(\"SPARK_YARN_USER_ENV\").foreach { userEnvs =>\r\n      YarnSparkHadoopUtil.setEnvFromInputString(env, userEnvs)\r\n    }\r\n```\r\n\r\nI believe all spark.* in the interpreter settings are passed to SparkConf.\r\n\r\nhttps://github.com/apache/incubator-zeppelin/blob/master/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java\r\n\r\nIs this a bug:\r\n```\r\n      if (!key.startsWith(\"spark.\") || !val.trim().isEmpty()) {\r\n        logger.debug(String.format(\"SparkConf: key = [%s], value = [%s]\", key, val));\r\n        conf.set(key, val);\r\n      }\r\n```\r\n\r\nShouldn't it say `if (key.startsWith(\"spark.\")`?"}}