{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/4006","labels_url":"https://api.github.com/repos/apache/spark/issues/4006/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/4006/comments","events_url":"https://api.github.com/repos/apache/spark/issues/4006/events","html_url":"https://github.com/apache/spark/pull/4006","id":54046849,"number":4006,"title":"[SPARK-4859][Core][Streaming] Refactor LiveListenerBus and StreamingListenerBus ","user":{"login":"zsxwing","id":1000778,"avatar_url":"https://avatars.githubusercontent.com/u/1000778?v=3","gravatar_id":"","url":"https://api.github.com/users/zsxwing","html_url":"https://github.com/zsxwing","followers_url":"https://api.github.com/users/zsxwing/followers","following_url":"https://api.github.com/users/zsxwing/following{/other_user}","gists_url":"https://api.github.com/users/zsxwing/gists{/gist_id}","starred_url":"https://api.github.com/users/zsxwing/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zsxwing/subscriptions","organizations_url":"https://api.github.com/users/zsxwing/orgs","repos_url":"https://api.github.com/users/zsxwing/repos","events_url":"https://api.github.com/users/zsxwing/events{/privacy}","received_events_url":"https://api.github.com/users/zsxwing/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":27,"created_at":"2015-01-12T12:11:42Z","updated_at":"2015-01-26T03:46:46Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/4006","html_url":"https://github.com/apache/spark/pull/4006","diff_url":"https://github.com/apache/spark/pull/4006.diff","patch_url":"https://github.com/apache/spark/pull/4006.patch"},"body":"This PR refactors LiveListenerBus and StreamingListenerBus and extracts the common codes to a parent class `ListenerBus`. \r\n\r\nIt also includes bug fixes in #3710:\r\n1. Fix the race condition of queueFullErrorMessageLogged in LiveListenerBus and StreamingListenerBus to avoid outputing `queue-full-error` logs multiple times.\r\n2. Make sure the SHUTDOWN message will be delivered to listenerThread, so that we can make sure listenerThread will always be able to exit.\r\n3. Log the error from listener rather than crashing listenerThread in StreamingListenerBus.\r\n\r\nDuring fixing the above bugs, we find it's better to make LiveListenerBus and StreamingListenerBus have the same bahaviors. Then there will be many duplicated codes in LiveListenerBus and StreamingListenerBus. \r\n\r\nTherefore, I extracted their common codes to `ListenerBus` as a parent class: LiveListenerBus and StreamingListenerBus only need to extend `ListenerBus` and implement `onPostEvent` (how to process an event) and `onDropEvent` (do something when droppping an event)."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/71411703","html_url":"https://github.com/apache/spark/pull/4006#issuecomment-71411703","issue_url":"https://api.github.com/repos/apache/spark/issues/4006","id":71411703,"user":{"login":"andrewor14","id":2133137,"avatar_url":"https://avatars.githubusercontent.com/u/2133137?v=3","gravatar_id":"","url":"https://api.github.com/users/andrewor14","html_url":"https://github.com/andrewor14","followers_url":"https://api.github.com/users/andrewor14/followers","following_url":"https://api.github.com/users/andrewor14/following{/other_user}","gists_url":"https://api.github.com/users/andrewor14/gists{/gist_id}","starred_url":"https://api.github.com/users/andrewor14/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andrewor14/subscriptions","organizations_url":"https://api.github.com/users/andrewor14/orgs","repos_url":"https://api.github.com/users/andrewor14/repos","events_url":"https://api.github.com/users/andrewor14/events{/privacy}","received_events_url":"https://api.github.com/users/andrewor14/received_events","type":"User","site_admin":false},"created_at":"2015-01-26T03:46:46Z","updated_at":"2015-01-26T03:46:46Z","body":"@zsxwing rather than making a `ListenerHelper` take in two type parameters, can we do the following two things: (1) make `StreamingListenerEvent` extend `SparkListenerEvent`, and (2) make a new trait `Listener` or `ListensToSparkEvents`  and have both `SparkListener` and `StreamingListener` extend it. Would that simplify things or not?"}}