{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/7753","labels_url":"https://api.github.com/repos/apache/spark/issues/7753/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/7753/comments","events_url":"https://api.github.com/repos/apache/spark/issues/7753/events","html_url":"https://github.com/apache/spark/pull/7753","id":97991340,"number":7753,"title":"[SPARK-9104][SPARK-9105][SPARK-9106][SPARK-9107][CORE][WIP] Netty network layer memory usage on webUI","user":{"login":"liyezhang556520","id":4716022,"avatar_url":"https://avatars.githubusercontent.com/u/4716022?v=3","gravatar_id":"","url":"https://api.github.com/users/liyezhang556520","html_url":"https://github.com/liyezhang556520","followers_url":"https://api.github.com/users/liyezhang556520/followers","following_url":"https://api.github.com/users/liyezhang556520/following{/other_user}","gists_url":"https://api.github.com/users/liyezhang556520/gists{/gist_id}","starred_url":"https://api.github.com/users/liyezhang556520/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/liyezhang556520/subscriptions","organizations_url":"https://api.github.com/users/liyezhang556520/orgs","repos_url":"https://api.github.com/users/liyezhang556520/repos","events_url":"https://api.github.com/users/liyezhang556520/events{/privacy}","received_events_url":"https://api.github.com/users/liyezhang556520/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":28,"created_at":"2015-07-29T17:49:24Z","updated_at":"2015-11-17T16:18:47Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/7753","html_url":"https://github.com/apache/spark/pull/7753","diff_url":"https://github.com/apache/spark/pull/7753.diff","patch_url":"https://github.com/apache/spark/pull/7753.patch"},"body":"This is a sub-task of [SPARK-9103](https://issues.apache.org/jira/browse/SPARK-9103), we'd like to expose the memory usage for spark running time, this is the first step to expose the netty buffer used both with on-heap and off-heap memory. Also the metrics are showed on WebUI. In this PR, a new web Tab name `Memory` is added. Which is used to show the memory usage of each executors (can be in more details in future). the screenshot is like the following:\r\n\r\n![image](https://cloud.githubusercontent.com/assets/4716022/9379782/46e8d4a6-475e-11e5-8d39-f0556d569797.png)\r\n\r\nThis is for each stages memory info:\r\n![image](https://cloud.githubusercontent.com/assets/4716022/9379831/9a9a0250-475e-11e5-9d52-97f868605d11.png)\r\n\r\nThis is History View:\r\n![image](https://cloud.githubusercontent.com/assets/4716022/9379991/bd64f0e6-475f-11e5-8b12-15ea476195a7.png)\r\n\r\n\r\nThis is WIP because with unit tests left.\r\n  "},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/157418522","html_url":"https://github.com/apache/spark/pull/7753#issuecomment-157418522","issue_url":"https://api.github.com/repos/apache/spark/issues/7753","id":157418522,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2015-11-17T16:18:47Z","updated_at":"2015-11-17T16:18:47Z","body":"**[Test build #46091 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46091/consoleFull)** for PR 7753 at commit [`75e63c3`](https://github.com/apache/spark/commit/75e63c366fcee3fed5cfe5c1066ab9727fd1f574).\n * This patch **fails Scala style tests**.\n * This patch merges cleanly.\n * This patch adds the following public classes _(experimental)_:\\n  * `class ExecutorMetrics extends Serializable `\\n  * `class TransportMetrics (`\\n  * `class MemoryListener extends SparkListener `\\n  * `class MemoryUIInfo `\\n  * `class TransportMemSize `\\n  * `case class MemTime(memorySize: Long = System.currentTimeMillis, timeStamp: Long = 0L)`\\n"}}