{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/9888","labels_url":"https://api.github.com/repos/apache/spark/issues/9888/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/9888/comments","events_url":"https://api.github.com/repos/apache/spark/issues/9888/events","html_url":"https://github.com/apache/spark/pull/9888","id":118227240,"number":9888,"title":"[SPARK-11904] [PySpark] reduceByKeyAndWindow does not require checkpointing when invFunc is None","user":{"login":"dtolpin","id":591245,"avatar_url":"https://avatars.githubusercontent.com/u/591245?v=3","gravatar_id":"","url":"https://api.github.com/users/dtolpin","html_url":"https://github.com/dtolpin","followers_url":"https://api.github.com/users/dtolpin/followers","following_url":"https://api.github.com/users/dtolpin/following{/other_user}","gists_url":"https://api.github.com/users/dtolpin/gists{/gist_id}","starred_url":"https://api.github.com/users/dtolpin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dtolpin/subscriptions","organizations_url":"https://api.github.com/users/dtolpin/orgs","repos_url":"https://api.github.com/users/dtolpin/repos","events_url":"https://api.github.com/users/dtolpin/events{/privacy}","received_events_url":"https://api.github.com/users/dtolpin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-11-21T23:25:28Z","updated_at":"2015-11-21T23:27:09Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/9888","html_url":"https://github.com/apache/spark/pull/9888","diff_url":"https://github.com/apache/spark/pull/9888.diff","patch_url":"https://github.com/apache/spark/pull/9888.patch"},"body":"when invFunc is None, `reduceByKeyAndWindow(func, None, winsize, slidesize)` is equivalent to\r\n\r\n     reduceByKey(func).window(winsize, slidesize).reduceByKey(winsize, slidesize)\r\n\r\nand no checkpoint is necessary. The corresponding Scala code does exactly that, but Python code always creates a windowed stream with obligatory checkpointing. The patch fixes this. \r\n\r\nI do not know how to unit-test this."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/158690246","html_url":"https://github.com/apache/spark/pull/9888#issuecomment-158690246","issue_url":"https://api.github.com/repos/apache/spark/issues/9888","id":158690246,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2015-11-21T23:27:09Z","updated_at":"2015-11-21T23:27:09Z","body":"Can one of the admins verify this patch?"}}