{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/4808","labels_url":"https://api.github.com/repos/apache/spark/issues/4808/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/4808/comments","events_url":"https://api.github.com/repos/apache/spark/issues/4808/events","html_url":"https://github.com/apache/spark/pull/4808","id":59200501,"number":4808,"title":"[SPARK-6055] [PySpark] fix incorrect __eq__ of DataType","user":{"login":"davies","id":40902,"avatar_url":"https://avatars.githubusercontent.com/u/40902?v=3","gravatar_id":"","url":"https://api.github.com/users/davies","html_url":"https://github.com/davies","followers_url":"https://api.github.com/users/davies/followers","following_url":"https://api.github.com/users/davies/following{/other_user}","gists_url":"https://api.github.com/users/davies/gists{/gist_id}","starred_url":"https://api.github.com/users/davies/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/davies/subscriptions","organizations_url":"https://api.github.com/users/davies/orgs","repos_url":"https://api.github.com/users/davies/repos","events_url":"https://api.github.com/users/davies/events{/privacy}","received_events_url":"https://api.github.com/users/davies/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":10,"created_at":"2015-02-27T07:31:14Z","updated_at":"2015-02-27T19:29:45Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/4808","html_url":"https://github.com/apache/spark/pull/4808","diff_url":"https://github.com/apache/spark/pull/4808.diff","patch_url":"https://github.com/apache/spark/pull/4808.patch"},"body":"The _eq_ of DataType is not correct, class cache is not use correctly (created class can not be find by dataType), then it will create lots of classes (saved in _cached_cls), never released.\r\n\r\nAlso, all same DataType have same hash code, there will be many object in a dict with the same hash code, end with hash attach, it's very slow to access this dict (depends on the implementation of CPython).\r\n\r\nThis PR also improve the performance of inferSchema (avoid the unnecessary converter of object). \r\n\r\ncc @pwendell  @JoshRosen "},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/76456279","html_url":"https://github.com/apache/spark/pull/4808#issuecomment-76456279","issue_url":"https://api.github.com/repos/apache/spark/issues/4808","id":76456279,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2015-02-27T19:29:45Z","updated_at":"2015-02-27T19:29:45Z","body":"Test FAILed.\nRefer to this link for build results (access rights to CI server needed): \nhttps://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28079/\nTest FAILed."}}