{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/6166","labels_url":"https://api.github.com/repos/apache/spark/issues/6166/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/6166/comments","events_url":"https://api.github.com/repos/apache/spark/issues/6166/events","html_url":"https://github.com/apache/spark/pull/6166","id":76563451,"number":6166,"title":"[SPARK-7657][YARN] Add driver logs links in application UI, in cluster mode.","user":{"login":"harishreedharan","id":1481194,"avatar_url":"https://avatars.githubusercontent.com/u/1481194?v=3","gravatar_id":"","url":"https://api.github.com/users/harishreedharan","html_url":"https://github.com/harishreedharan","followers_url":"https://api.github.com/users/harishreedharan/followers","following_url":"https://api.github.com/users/harishreedharan/following{/other_user}","gists_url":"https://api.github.com/users/harishreedharan/gists{/gist_id}","starred_url":"https://api.github.com/users/harishreedharan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/harishreedharan/subscriptions","organizations_url":"https://api.github.com/users/harishreedharan/orgs","repos_url":"https://api.github.com/users/harishreedharan/repos","events_url":"https://api.github.com/users/harishreedharan/events{/privacy}","received_events_url":"https://api.github.com/users/harishreedharan/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":42,"created_at":"2015-05-15T02:32:37Z","updated_at":"2015-05-16T00:28:30Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/6166","html_url":"https://github.com/apache/spark/pull/6166","diff_url":"https://github.com/apache/spark/pull/6166.diff","patch_url":"https://github.com/apache/spark/pull/6166.patch"},"body":"The idea is to get the links to the logs in AM and then when the SparkContext starts up - send out a fake `SparkListenerExecutorAdded` event with the executorId as `driver` and the actual links to the logs. This ends up showing the actual links.\r\n\r\nThis PR adds new environment variables that can be set by any cluster manager before starting the driver. These environment variables must begin with `spark.driver.log.` followed by the name of the log type, with its value being the URL that will be show in the UI/history server. For example, YARN AM's stderr would be reported as: `spark.driver.log.stderr` etc. In YARN mode, this and `spark.driver.log.stdout` are added.\r\n\r\n![amlinks](https://cloud.githubusercontent.com/assets/1481194/7645832/ecd36804-fa6f-11e4-9994-bccc423179fd.png)\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/102553683","html_url":"https://github.com/apache/spark/pull/6166#issuecomment-102553683","issue_url":"https://api.github.com/repos/apache/spark/issues/6166","id":102553683,"user":{"login":"vanzin","id":1694083,"avatar_url":"https://avatars.githubusercontent.com/u/1694083?v=3","gravatar_id":"","url":"https://api.github.com/users/vanzin","html_url":"https://github.com/vanzin","followers_url":"https://api.github.com/users/vanzin/followers","following_url":"https://api.github.com/users/vanzin/following{/other_user}","gists_url":"https://api.github.com/users/vanzin/gists{/gist_id}","starred_url":"https://api.github.com/users/vanzin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vanzin/subscriptions","organizations_url":"https://api.github.com/users/vanzin/orgs","repos_url":"https://api.github.com/users/vanzin/repos","events_url":"https://api.github.com/users/vanzin/events{/privacy}","received_events_url":"https://api.github.com/users/vanzin/received_events","type":"User","site_admin":false},"created_at":"2015-05-16T00:28:30Z","updated_at":"2015-05-16T00:28:30Z","body":"LGTM aside from a small issue."}}