{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/6827","labels_url":"https://api.github.com/repos/apache/spark/issues/6827/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/6827/comments","events_url":"https://api.github.com/repos/apache/spark/issues/6827/events","html_url":"https://github.com/apache/spark/pull/6827","id":88358810,"number":6827,"title":"[SPARK-8372] History server shows incorrect information for application not started","user":{"login":"carsonwang","id":9278199,"avatar_url":"https://avatars.githubusercontent.com/u/9278199?v=3","gravatar_id":"","url":"https://api.github.com/users/carsonwang","html_url":"https://github.com/carsonwang","followers_url":"https://api.github.com/users/carsonwang/followers","following_url":"https://api.github.com/users/carsonwang/following{/other_user}","gists_url":"https://api.github.com/users/carsonwang/gists{/gist_id}","starred_url":"https://api.github.com/users/carsonwang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/carsonwang/subscriptions","organizations_url":"https://api.github.com/users/carsonwang/orgs","repos_url":"https://api.github.com/users/carsonwang/repos","events_url":"https://api.github.com/users/carsonwang/events{/privacy}","received_events_url":"https://api.github.com/users/carsonwang/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":25,"created_at":"2015-06-15T08:40:50Z","updated_at":"2015-06-25T16:33:32Z","closed_at":"2015-06-17T20:43:51Z","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/6827","html_url":"https://github.com/apache/spark/pull/6827","diff_url":"https://github.com/apache/spark/pull/6827.diff","patch_url":"https://github.com/apache/spark/pull/6827.patch"},"body":"The history server may show an incorrect App ID for an incomplete application like <App ID>.inprogress. This app info will never disappear even after the app is completed.\r\n![incorrectappinfo](https://cloud.githubusercontent.com/assets/9278199/8156147/2a10fdbe-137d-11e5-9620-c5b61d93e3c1.png)\r\n\r\nThe cause of the issue is that a log path name is used as the app id when app id cannot be got during replay."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/115313702","html_url":"https://github.com/apache/spark/pull/6827#issuecomment-115313702","issue_url":"https://api.github.com/repos/apache/spark/issues/6827","id":115313702,"user":{"login":"vanzin","id":1694083,"avatar_url":"https://avatars.githubusercontent.com/u/1694083?v=3","gravatar_id":"","url":"https://api.github.com/users/vanzin","html_url":"https://github.com/vanzin","followers_url":"https://api.github.com/users/vanzin/followers","following_url":"https://api.github.com/users/vanzin/following{/other_user}","gists_url":"https://api.github.com/users/vanzin/gists{/gist_id}","starred_url":"https://api.github.com/users/vanzin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vanzin/subscriptions","organizations_url":"https://api.github.com/users/vanzin/orgs","repos_url":"https://api.github.com/users/vanzin/repos","events_url":"https://api.github.com/users/vanzin/events{/privacy}","received_events_url":"https://api.github.com/users/vanzin/received_events","type":"User","site_admin":false},"created_at":"2015-06-25T16:33:32Z","updated_at":"2015-06-25T16:33:32Z","body":"So, one thing that I noticed after I said \"LGTM\" is that this change breaks old logs (those generated by versions of Spark that do not record the app id). Those will never show up anymore (I think that should only be Spark 1.0?).\r\n\r\nIf we care about that use case, this version should probably be reverted. The proper fix could be something as simple as this:\r\n\r\n    appListener.appId.getOrElse(logPath.getName().stripSuffix(EventLoggingListener.IN_PROGRESS)),"}}