{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/4107","labels_url":"https://api.github.com/repos/apache/spark/issues/4107/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/4107/comments","events_url":"https://api.github.com/repos/apache/spark/issues/4107/events","html_url":"https://github.com/apache/spark/pull/4107","id":54818516,"number":4107,"title":"[SPARK-5325][SQL][WIP] Shrink the Hive shim layer","user":{"login":"liancheng","id":230655,"avatar_url":"https://avatars.githubusercontent.com/u/230655?v=3","gravatar_id":"","url":"https://api.github.com/users/liancheng","html_url":"https://github.com/liancheng","followers_url":"https://api.github.com/users/liancheng/followers","following_url":"https://api.github.com/users/liancheng/following{/other_user}","gists_url":"https://api.github.com/users/liancheng/gists{/gist_id}","starred_url":"https://api.github.com/users/liancheng/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/liancheng/subscriptions","organizations_url":"https://api.github.com/users/liancheng/orgs","repos_url":"https://api.github.com/users/liancheng/repos","events_url":"https://api.github.com/users/liancheng/events{/privacy}","received_events_url":"https://api.github.com/users/liancheng/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-01-19T22:08:52Z","updated_at":"2015-01-19T22:46:46Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/4107","html_url":"https://github.com/apache/spark/pull/4107","diff_url":"https://github.com/apache/spark/pull/4107.diff","patch_url":"https://github.com/apache/spark/pull/4107.patch"},"body":"This PR shrinks the Hive shim layer significantly by the following means:\r\n\r\n1. Re-implement some functions in a backwards compatible way, e.g. constant object inspector methods in `WritableConstantObjectInspectors`, and the `FileSinkDescWrapper` class\r\n2. For some simple functions that are not performance critical, re-implemented them with reflection tricks.\r\n\r\nMost functions are moved from `HiveShim` to `HiveCompat`. Constant object inspector related methods are moved to `o.a.h.h.s.o.p.WritableConstantObjectInspectors` because we need to call package private constructors.\r\n\r\nReflection utility function `callWithAlternativies` is introduced to simplify reflection tricks. It tries to invoke all provided reflected constructor calls, method calls, and static method calls one by one until one of them succeeds. For example, `HiveDecimal` objects are created via normal constructor in Hive 0.12.0, while `HiveDecimal.create` must be used in Hive 0.13.1. We can fix this by:\r\n\r\n```scala\r\n  def createDecimal(bd: JBigDecimal) = callWithAlternatives(\r\n    // For Hive 0.12.0\r\n    InvokeStatic[HiveDecimal](classOf[HiveDecimal], \"create\",\r\n      classOf[JBigDecimal] -> bd),\r\n\r\n    // For Hive 0.13.1\r\n    Construct(classOf[HiveDecimal],\r\n      classOf[JBigDecimal] -> bd))\r\n```\r\n\r\n<!-- Reviewable:start -->\r\n[<img src=\"https://reviewable.io/review_button.png\" height=40 alt=\"Review on Reviewable\"/>](https://reviewable.io/reviews/apache/spark/4107)\r\n<!-- Reviewable:end -->\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/70574795","html_url":"https://github.com/apache/spark/pull/4107#issuecomment-70574795","issue_url":"https://api.github.com/repos/apache/spark/issues/4107","id":70574795,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2015-01-19T22:46:46Z","updated_at":"2015-01-19T22:46:46Z","body":"  [Test build #25772 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25772/consoleFull) for   PR 4107 at commit [`e130d93`](https://github.com/apache/spark/commit/e130d935bcdc4cf06e97f44c659b9b9b3e910cc5).\n * This patch **fails to build**.\n * This patch merges cleanly.\n * This patch adds no public classes."}}