{"push_id":742043357,"size":5,"distinct_size":5,"ref":"refs/heads/expose-memory-metrics","head":"eee54371beff1900e29e875ebb34c333f571dc1e","before":"d9b90155617f1f092dff3d4f72610ddcc2561562","commits":[{"sha":"1b0099fc62d02ff6216a76fbfe17a4ec5b2f3536","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-9411] [SQL] Make Tungsten page sizes configurable\n\nWe need to make page sizes configurable so we can reduce them in unit tests and increase them in real production workloads.  These sizes are now controlled by a new configuration, `spark.buffer.pageSize`.  The new default is 64 megabytes.\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #7741 from JoshRosen/SPARK-9411 and squashes the following commits:\n\na43c4db [Josh Rosen] Fix pow\n2c0eefc [Josh Rosen] Fix MAXIMUM_PAGE_SIZE_BYTES comment + value\nbccfb51 [Josh Rosen] Lower page size to 4MB in TestHive\nba54d4b [Josh Rosen] Make UnsafeExternalSorter's page size configurable\n0045aa2 [Josh Rosen] Make UnsafeShuffle's page size configurable\nbc734f0 [Josh Rosen] Rename configuration\ne614858 [Josh Rosen] Makes BytesToBytesMap page size configurable","distinct":true,"url":"https://api.github.com/repos/andrewor14/spark/commits/1b0099fc62d02ff6216a76fbfe17a4ec5b2f3536"},{"sha":"2cc212d56a1d50fe68d5816f71b27803de1f6389","author":{"email":"9835e449f9a666ddd8ba8aeb460af39a5130691b@databricks.com","name":"Feynman Liang"},"message":"[SPARK-6793] [MLLIB] OnlineLDAOptimizer LDA perplexity\n\nImplements `logPerplexity` in `OnlineLDAOptimizer`. Also refactors inference code into companion object to enable future reuse (e.g. `predict` method).\n\nAuthor: Feynman Liang <fliang@databricks.com>\n\nCloses #7705 from feynmanliang/SPARK-6793-perplexity and squashes the following commits:\n\n6da2c99 [Feynman Liang] Remove get* from LDAModel public API\n8381da6 [Feynman Liang] Code review comments\n17f7000 [Feynman Liang] Documentation typo fixes\n2f452a4 [Feynman Liang] Remove auxillary DistributedLDAModel constructor\na275914 [Feynman Liang] Prevent empty counts calls to variationalInference\n06d02d9 [Feynman Liang] Remove deprecated LocalLDAModel constructor\nafecb46 [Feynman Liang] Fix regression bug in sstats accumulator\n5a327a0 [Feynman Liang] Code review quick fixes\n998c03e [Feynman Liang] Fix style\n1cbb67d [Feynman Liang] Fix access modifier bug\n4362daa [Feynman Liang] Organize imports\n4f171f7 [Feynman Liang] Fix indendation\n2f049ce [Feynman Liang] Fix failing save/load tests\n7415e96 [Feynman Liang] Pick changes from big PR\n11e7c33 [Feynman Liang] Merge remote-tracking branch 'apache/master' into SPARK-6793-perplexity\nf8adc48 [Feynman Liang] Add logPerplexity, refactor variationalBound into a method\ncd521d6 [Feynman Liang] Refactor methods into companion class\n7f62a55 [Feynman Liang] --amend\nc62cb1e [Feynman Liang] Outer product for stats, revert Range slicing\naead650 [Feynman Liang] Range slice, in-place update, reduce transposes","distinct":true,"url":"https://api.github.com/repos/andrewor14/spark/commits/2cc212d56a1d50fe68d5816f71b27803de1f6389"},{"sha":"86505962e6c9da1ee18c6a3533e169a22e4f1665","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-9448][SQL] GenerateUnsafeProjection should not share expressions across instances.\n\nWe accidentally moved the list of expressions from the generated code instance to the class wrapper, and as a result, different threads are sharing the same set of expressions, which cause problems for expressions with mutable state.\n\nThis pull request fixed that problem, and also added unit tests for all codegen classes, except GeneratedOrdering (which will never need any expressions since sort now only accepts bound references.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #7759 from rxin/SPARK-9448 and squashes the following commits:\n\nc09b50f [Reynold Xin] [SPARK-9448][SQL] GenerateUnsafeProjection should not share expressions across instances.","distinct":true,"url":"https://api.github.com/repos/andrewor14/spark/commits/86505962e6c9da1ee18c6a3533e169a22e4f1665"},{"sha":"103d8cce78533b38b4f8060b30f7f455113bc6b5","author":{"email":"4c128750a9d45c2eaea031bb820de86def24c584@bimal-MBP.local","name":"Bimal Tandel"},"message":"[SPARK-8921] [MLLIB] Add @since tags to mllib.stat\n\nAuthor: Bimal Tandel <bimal@bimal-MBP.local>\n\nCloses #7730 from BimalTandel/branch_spark_8921 and squashes the following commits:\n\n3ea230a [Bimal Tandel] Spark 8921 add @since tags","distinct":true,"url":"https://api.github.com/repos/andrewor14/spark/commits/103d8cce78533b38b4f8060b30f7f455113bc6b5"},{"sha":"eee54371beff1900e29e875ebb34c333f571dc1e","author":{"email":"02e0a999c50b1f88df7a8f5a04e1b76b35ea6a88@databricks.com","name":"Andrew Or"},"message":"Merge branch 'master' of github.com:apache/spark into expose-memory-metrics\n\nConflicts:\n\tcore/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java\n\tsql/core/src/main/scala/org/apache/spark/sql/execution/GeneratedAggregate.scala","distinct":true,"url":"https://api.github.com/repos/andrewor14/spark/commits/eee54371beff1900e29e875ebb34c333f571dc1e"}]}