{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/8907","labels_url":"https://api.github.com/repos/apache/spark/issues/8907/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/8907/comments","events_url":"https://api.github.com/repos/apache/spark/issues/8907/events","html_url":"https://github.com/apache/spark/pull/8907","id":108211100,"number":8907,"title":"[SPARK-10811] [SQL] Eliminates unnecessary byte array copying","user":{"login":"liancheng","id":230655,"avatar_url":"https://avatars.githubusercontent.com/u/230655?v=3","gravatar_id":"","url":"https://api.github.com/users/liancheng","html_url":"https://github.com/liancheng","followers_url":"https://api.github.com/users/liancheng/followers","following_url":"https://api.github.com/users/liancheng/following{/other_user}","gists_url":"https://api.github.com/users/liancheng/gists{/gist_id}","starred_url":"https://api.github.com/users/liancheng/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/liancheng/subscriptions","organizations_url":"https://api.github.com/users/liancheng/orgs","repos_url":"https://api.github.com/users/liancheng/repos","events_url":"https://api.github.com/users/liancheng/events{/privacy}","received_events_url":"https://api.github.com/users/liancheng/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-09-24T20:47:14Z","updated_at":"2015-09-24T21:32:22Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/8907","html_url":"https://github.com/apache/spark/pull/8907","diff_url":"https://github.com/apache/spark/pull/8907.diff","patch_url":"https://github.com/apache/spark/pull/8907.patch"},"body":"When reading Parquet string and binary-backed decimal values, Parquet `Binary.getBytes` always returns a copied byte array, which is unnecessary. Since the underlying implementation of `Binary` values there is guaranteed to be `ByteArraySliceBackedBinary`, we can use `Binary.toByteBuffer.array()` to steal the underlying byte array without copying it.\r\n\r\nThis brings performance benefits when scanning Parquet string and binary-backed decimal columns. Note that, this trick doesn't cover binary-backed decimals with precision greater than 18.\r\n\r\nMy micro-benchmark result is that, this brings a ~15% performance boost for scanning TPC-DS `store_sales` table (scale factor 15)."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/143056520","html_url":"https://github.com/apache/spark/pull/8907#issuecomment-143056520","issue_url":"https://api.github.com/repos/apache/spark/issues/8907","id":143056520,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2015-09-24T21:32:22Z","updated_at":"2015-09-24T21:32:22Z","body":"Merged build started."}}