{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/5423","labels_url":"https://api.github.com/repos/apache/spark/issues/5423/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/5423/comments","events_url":"https://api.github.com/repos/apache/spark/issues/5423/events","html_url":"https://github.com/apache/spark/pull/5423","id":67135385,"number":5423,"title":"[SPARK-1537] [YARN] [WiP] Add history provider for YARN Application Timeline Server","user":{"login":"steveloughran","id":162090,"avatar_url":"https://avatars.githubusercontent.com/u/162090?v=3","gravatar_id":"","url":"https://api.github.com/users/steveloughran","html_url":"https://github.com/steveloughran","followers_url":"https://api.github.com/users/steveloughran/followers","following_url":"https://api.github.com/users/steveloughran/following{/other_user}","gists_url":"https://api.github.com/users/steveloughran/gists{/gist_id}","starred_url":"https://api.github.com/users/steveloughran/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/steveloughran/subscriptions","organizations_url":"https://api.github.com/users/steveloughran/orgs","repos_url":"https://api.github.com/users/steveloughran/repos","events_url":"https://api.github.com/users/steveloughran/events{/privacy}","received_events_url":"https://api.github.com/users/steveloughran/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":88,"created_at":"2015-04-08T13:52:10Z","updated_at":"2015-10-28T12:12:17Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/5423","html_url":"https://github.com/apache/spark/pull/5423","diff_url":"https://github.com/apache/spark/pull/5423.diff","patch_url":"https://github.com/apache/spark/pull/5423.patch"},"body":"This pull request incorporates the work of SPARK-11314 and SPARK-11315, adding in the history server side of the system: a subclass of `ApplicationHistoryProvider` which can enumerate application histories listed in the YARN timeline server, and retrieve them on demand.\r\n\r\nIt includes \r\n\r\n1. The history provider itself\r\n1. A REST client for the timeline service, one which supports kerberos authentication against the remote timeline service. (Keytab based; not delegation token: as with the `FsHistoryProvider`, the History Server needs to be started with a keytab).\r\n1. A set of integration tests which do end-to-end testing of the history, from application-side event publishing to web UI retrieval and validation.\r\n1. Tests to verify the robustness of the history provider against failures. The history enumeration is asynchronous, so transient failures will result in out of date listings, rather than stack traces. Attempts to explicitly retrieve an application will fail when there are connectivity problems of any kind."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/151825740","html_url":"https://github.com/apache/spark/pull/5423#issuecomment-151825740","issue_url":"https://api.github.com/repos/apache/spark/issues/5423","id":151825740,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2015-10-28T12:12:17Z","updated_at":"2015-10-28T12:12:17Z","body":" Build triggered."}}