{"push_id":870945738,"size":33,"distinct_size":33,"ref":"refs/heads/master","head":"045a4f045821dcf60442f0600c2df1b79bddb536","before":"b362d50fca30693f97bd859984157bb8a76d48a1","commits":[{"sha":"75a292291062783129d02607302f91c85655975e","author":{"email":"2676c1e0be9d9df57419d05258d46b7142312de2@hortonworks.com","name":"jerryshao"},"message":"[SPARK-9065][STREAMING][PYSPARK] Add MessageHandler for Kafka Python API\n\nFixed the merge conflicts in #7410\n\nCloses #7410\n\nAuthor: Shixiong Zhu <shixiong@databricks.com>\nAuthor: jerryshao <saisai.shao@intel.com>\nAuthor: jerryshao <sshao@hortonworks.com>\n\nCloses #9742 from zsxwing/pr7410.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/75a292291062783129d02607302f91c85655975e"},{"sha":"ed8d1531f93f697c54bbaecefe08c37c32b0d391","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-11793][SQL] Dataset should set the resolved encoders internally for maps.\n\nI also wrote a test case -- but unfortunately the test case is not working due to SPARK-11795.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #9784 from rxin/SPARK-11503.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/ed8d1531f93f697c54bbaecefe08c37c32b0d391"},{"sha":"bf25f9bdfc7bd8533890c7df1b35afa912dc6d3d","author":{"email":"c437f8b7af73b0a10ecac061712ff9de6d41089e@databricks.com","name":"Davies Liu"},"message":"[SPARK-11016] Move RoaringBitmap to explicit Kryo serializer\n\nFix the serialization of RoaringBitmap with Kyro serializer\n\nThis PR came from https://github.com/metamx/spark/pull/1, thanks to drcrallen\n\nAuthor: Davies Liu <davies@databricks.com>\nAuthor: Charles Allen <charles@allen-net.com>\n\nCloses #9748 from davies/SPARK-11016.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/bf25f9bdfc7bd8533890c7df1b35afa912dc6d3d"},{"sha":"e33053ee0015025bbcfddb20cc9216c225bbe624","author":{"email":"bee1182b70536910b4db11bba3bcb19b1c82ecd8@hotmail.com","name":"Kent Yao"},"message":"[SPARK-11583] [CORE] MapStatus Using RoaringBitmap More Properly\n\nThis PR upgrade the version of RoaringBitmap to 0.5.10, to optimize the memory layout, will be much smaller when most of blocks are empty.\n\nThis PR is based on #9661 (fix conflicts), see all of the comments at https://github.com/apache/spark/pull/9661 .\n\nAuthor: Kent Yao <yaooqinn@hotmail.com>\nAuthor: Davies Liu <davies@databricks.com>\nAuthor: Charles Allen <charles@allen-net.com>\n\nCloses #9746 from davies/roaring_mapstatus.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/e33053ee0015025bbcfddb20cc9216c225bbe624"},{"sha":"98be8169f07eb0f1b8f01776c71d0e1ed3d5e4d5","author":{"email":"c437f8b7af73b0a10ecac061712ff9de6d41089e@databricks.com","name":"Davies Liu"},"message":"[SPARK-11737] [SQL] Fix serialization of UTF8String with Kyro\n\nThe default implementation of serialization UTF8String with Kyro may be not correct (BYTE_ARRAY_OFFSET could be different across JVM)\n\nAuthor: Davies Liu <davies@databricks.com>\n\nCloses #9704 from davies/kyro_string.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/98be8169f07eb0f1b8f01776c71d0e1ed3d5e4d5"},{"sha":"91f4b6f2db12650dfc33a576803ba8aeccf935dd","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-11797][SQL] collect, first, and take should use encoders for serialization\n\nThey were previously using Spark's default serializer for serialization.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #9787 from rxin/SPARK-11797.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/91f4b6f2db12650dfc33a576803ba8aeccf935dd"},{"sha":"8fb775ba874dd0488667bf299a7b49760062dc00","author":{"email":"452ae006f36be6c70342f33f64bb6f6eb552689c@gmail.com","name":"Yanbo Liang"},"message":"[SPARK-11755][R] SparkR should export \"predict\"\n\nThe bug described at [SPARK-11755](https://issues.apache.org/jira/browse/SPARK-11755), after exporting ```predict``` we can both get the help information from the SparkR and base R package like the following:\n```Java\n> help(predict)\nHelp on topic ‘predict’ was found in the following packages:\n\n  Package               Library\n  SparkR                /Users/yanboliang/data/trunk2/spark/R/lib\n  stats                 /Library/Frameworks/R.framework/Versions/3.2/Resources/library\n\nChoose one\n\n1: Make predictions from a model {SparkR}\n2: Model Predictions {stats}\n```\n\nAuthor: Yanbo Liang <ybliang8@gmail.com>\n\nCloses #9732 from yanboliang/spark-11755.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/8fb775ba874dd0488667bf299a7b49760062dc00"},{"sha":"446738e51fcda50cf2dc44123ff6bf12a1611dc0","author":{"email":"22f3e725b314afeca4d45e12683c560674029d21@gmail.com","name":"tedyu"},"message":"[SPARK-11761] Prevent the call to StreamingContext#stop() in the listener bus's thread\n\nSee discussion toward the tail of https://github.com/apache/spark/pull/9723\nFrom zsxwing :\n```\nThe user should not call stop or other long-time work in a listener since it will block the listener thread, and prevent from stopping SparkContext/StreamingContext.\n\nI cannot see an approach since we need to stop the listener bus's thread before stopping SparkContext/StreamingContext totally.\n```\nProposed solution is to prevent the call to StreamingContext#stop() in the listener bus's thread.\n\nAuthor: tedyu <yuzhihong@gmail.com>\n\nCloses #9741 from tedyu/master.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/446738e51fcda50cf2dc44123ff6bf12a1611dc0"},{"sha":"67a5132c21bc8338adbae80b33b85e8fa0ddda34","author":{"email":"d3a9d0249da6ad1bf16818d24a161eab252b511a@zju.edu.cn","name":"RoyGaoVLIS"},"message":"[SPARK-7013][ML][TEST] Add unit test for spark.ml StandardScaler\n\nI have added unit test for ML's StandardScaler By comparing with R's output, please review  for me.\nThx.\n\nAuthor: RoyGaoVLIS <roygao@zju.edu.cn>\n\nCloses #6665 from RoyGao/7013.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/67a5132c21bc8338adbae80b33b85e8fa0ddda34"},{"sha":"2f191c66b668fc97f82f44fd8336b6a4488c2f5d","author":{"email":"c437f8b7af73b0a10ecac061712ff9de6d41089e@databricks.com","name":"Davies Liu"},"message":"[SPARK-11643] [SQL] parse year with leading zero\n\nSupport the years between 0 <= year < 1000\n\nAuthor: Davies Liu <davies@databricks.com>\n\nCloses #9701 from davies/leading_zero.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/2f191c66b668fc97f82f44fd8336b6a4488c2f5d"},{"sha":"9154f89befb7a33d4853cea95efd7dc6b25d033b","author":{"email":"7853928d03425cf3be35be2fa0c5b99641435d26@gmail.com","name":"Xusen Yin"},"message":"[SPARK-11728] Replace example code in ml-ensembles.md using include_example\n\nJIRA issue https://issues.apache.org/jira/browse/SPARK-11728.\n\nThe ml-ensembles.md file contains `OneVsRestExample`. Instead of writing new code files of two `OneVsRestExample`s, I use two existing files in the examples directory, they are `OneVsRestExample.scala` and `JavaOneVsRestExample.scala`.\n\nAuthor: Xusen Yin <yinxusen@gmail.com>\n\nCloses #9716 from yinxusen/SPARK-11728.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/9154f89befb7a33d4853cea95efd7dc6b25d033b"},{"sha":"8019f66df5c65e21d6e4e7e8fbfb7d0471ba3e37","author":{"email":"563f5923eae9a92c8a3f34a26b67fa348a303ed1@databricks.com","name":"Wenchen Fan"},"message":"[SPARK-10186][SQL][FOLLOW-UP] simplify test\n\nAuthor: Wenchen Fan <wenchen@databricks.com>\n\nCloses #9783 from cloud-fan/postgre.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/8019f66df5c65e21d6e4e7e8fbfb7d0471ba3e37"},{"sha":"5e2b44474c2b838bebeffe5ba5cd72961b0cd31e","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-11802][SQL] Kryo-based encoder for opaque types in Datasets\n\nI also found a bug with self-joins returning incorrect results in the Dataset API. Two test cases attached and filed SPARK-11803.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #9789 from rxin/SPARK-11802.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/5e2b44474c2b838bebeffe5ba5cd72961b0cd31e"},{"sha":"1714350bddd78cd1398e1a816f675ab729001081","author":{"email":"8adc4758e0403b3bd961e4e414291eaa9ba64098@databricks.com","name":"Yin Huai"},"message":"[SPARK-11792][SQL] SizeEstimator cannot provide a good size estimation of UnsafeHashedRelations\n\nhttps://issues.apache.org/jira/browse/SPARK-11792\n\nRight now, SizeEstimator will \"think\" a small UnsafeHashedRelation is several GBs.\n\nAuthor: Yin Huai <yhuai@databricks.com>\n\nCloses #9788 from yhuai/SPARK-11792.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/1714350bddd78cd1398e1a816f675ab729001081"},{"sha":"b8f4379ba1c5c1a8f3b4c88bd97031dc8ad2dfea","author":{"email":"5762d5c6a40d8496c8912e8911d54dcd6cba0c5d@us.ibm.com","name":"somideshmukh"},"message":"[SPARK-10946][SQL] JDBC - Use Statement.executeUpdate instead of PreparedStatement.executeUpdate for DDLs\n\nNew changes with JDBCRDD\n\nAuthor: somideshmukh <somilde@us.ibm.com>\n\nCloses #9733 from somideshmukh/SomilBranch-1.1.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/b8f4379ba1c5c1a8f3b4c88bd97031dc8ad2dfea"},{"sha":"e62820c85fe02c70f9ed51b2e68d41ff8cfecd40","author":{"email":"b29f906215354b2d63cd7fb7b9919be54e0af7cd@apache.org","name":"Jean-Baptiste Onofré"},"message":"[SPARK-6541] Sort executors by ID (numeric)\n\n\"Force\" the executor ID sort with Int.\n\nAuthor: Jean-Baptiste Onofré <jbonofre@apache.org>\n\nCloses #9165 from jbonofre/SPARK-6541.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/e62820c85fe02c70f9ed51b2e68d41ff8cfecd40"},{"sha":"9631ca35275b0ce8a5219f975907ac36ed11f528","author":{"email":"291c18f3fb7528c712d9098b0e50a515ea0b91d5@cloudera.com","name":"Sean Owen"},"message":"[SPARK-11652][CORE] Remote code execution with InvokerTransformer\n\nUpdate to Commons Collections 3.2.2 to avoid any potential remote code execution vulnerability\n\nAuthor: Sean Owen <sowen@cloudera.com>\n\nCloses #9731 from srowen/SPARK-11652.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/9631ca35275b0ce8a5219f975907ac36ed11f528"},{"sha":"1429e0a2b562469146b6fa06051c85a00092e5b8","author":{"email":"9b76f4c6f3d148c944ce81a218332574510cf58c@iitk.ac.in","name":"Viveka Kulharia"},"message":"rmse was wrongly calculated\n\nIt was multiplying with U instaed of dividing by U\n\nAuthor: Viveka Kulharia <vivkul@iitk.ac.in>\n\nCloses #9771 from vivkul/patch-1.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/1429e0a2b562469146b6fa06051c85a00092e5b8"},{"sha":"3a6807fdf07b0e73d76502a6bd91ad979fde8b61","author":{"email":"ced9a24e2e494bb60341a9f26a61a78e7a11abad@apache.org","name":"Jeff Zhang"},"message":"[SPARK-11804] [PYSPARK] Exception raise when using Jdbc predicates opt…\n\n…ion in PySpark\n\nAuthor: Jeff Zhang <zjffdu@apache.org>\n\nCloses #9791 from zjffdu/SPARK-11804.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/3a6807fdf07b0e73d76502a6bd91ad979fde8b61"},{"sha":"a97d6f3a5861e9f2bbe36957e3b39f835f3e214c","author":{"email":"ffad4dc97bb49db436733c212788f8b59c50dd2b@gmail.com","name":"zero323"},"message":"[SPARK-11281][SPARKR] Add tests covering the issue.\n\nThe goal of this PR is to add tests covering the issue to ensure that is was resolved by [SPARK-11086](https://issues.apache.org/jira/browse/SPARK-11086).\n\nAuthor: zero323 <matthew.szymkiewicz@gmail.com>\n\nCloses #9743 from zero323/SPARK-11281-tests.","distinct":true,"url":"https://api.github.com/repos/zsxwing/spark/commits/a97d6f3a5861e9f2bbe36957e3b39f835f3e214c"}]}