{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/9480","labels_url":"https://api.github.com/repos/apache/spark/issues/9480/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/9480/comments","events_url":"https://api.github.com/repos/apache/spark/issues/9480/events","html_url":"https://github.com/apache/spark/pull/9480","id":115188660,"number":9480,"title":"[SPARK-10371] [SQL] Implement subexpr elimination for UnsafeProjections","user":{"login":"nongli","id":242468,"avatar_url":"https://avatars.githubusercontent.com/u/242468?v=3","gravatar_id":"","url":"https://api.github.com/users/nongli","html_url":"https://github.com/nongli","followers_url":"https://api.github.com/users/nongli/followers","following_url":"https://api.github.com/users/nongli/following{/other_user}","gists_url":"https://api.github.com/users/nongli/gists{/gist_id}","starred_url":"https://api.github.com/users/nongli/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nongli/subscriptions","organizations_url":"https://api.github.com/users/nongli/orgs","repos_url":"https://api.github.com/users/nongli/repos","events_url":"https://api.github.com/users/nongli/events{/privacy}","received_events_url":"https://api.github.com/users/nongli/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-11-05T01:22:29Z","updated_at":"2015-11-05T01:27:21Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/9480","html_url":"https://github.com/apache/spark/pull/9480","diff_url":"https://github.com/apache/spark/pull/9480.diff","patch_url":"https://github.com/apache/spark/pull/9480.patch"},"body":"This patch adds the building blocks for codegening subexpr elimination and implements\r\nit end to end for UnsafeProjection. The building blocks can be used to do the same thing\r\nfor other operators.\r\n\r\nIt introduces some utilities to compute common sub expressions. Expressions can be added to\r\nthis data structure. The expr and its children will be recursively matched against existing\r\nexpressions (ones previously added) and grouped into common groups. This is built using\r\nthe existing `semanticEquals`. It does not understand things like commutative or associative\r\nexpressions. This can be done as future work.\r\n\r\nAfter building this data structure, the codegen process takes advantage of it by:\r\n  1. Generating a helper function in the generated class that computes the common\r\n     subexpression. This is done for all common subexpressions that have at least\r\n     two occurrences and the expression tree is sufficiently complex.\r\n  2. When generating the apply() function, if the helper function exists, call that\r\n     instead of regenerating the expression tree. Repeated calls to the helper function\r\n     shortcircuit the evaluation logic."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/153924543","html_url":"https://github.com/apache/spark/pull/9480#issuecomment-153924543","issue_url":"https://api.github.com/repos/apache/spark/issues/9480","id":153924543,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2015-11-05T01:27:21Z","updated_at":"2015-11-05T01:27:21Z","body":"Merged build started."}}