{"push_id":622692779,"size":24,"distinct_size":24,"ref":"refs/heads/string","head":"13d9d42c69212de4af7375a880c670006dcf6806","before":"fd113643c48b633eb505540a13b8fd4798c0197d","commits":[{"sha":"dfd2982bc7047732197f1d9ad77221e9c6076fc2","author":{"email":"50c4b4c4c26f83ee4bbc45a85db1c3c80b5aecdb@intel.com","name":"Cheng Hao"},"message":"[SQL][Minor] Use analyzed logical instead of unresolved in HiveComparisonTest\n\nSome internal unit test failed due to the logical plan node in pattern matching in `HiveComparisonTest`,  e.g.\nhttps://github.com/apache/spark/blob/master/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveComparisonTest.scala#L137\n\nWhich will may call the `output` function on an unresolved logical plan.\n\nAuthor: Cheng Hao <hao.cheng@intel.com>\n\nCloses #4946 from chenghao-intel/logical and squashes the following commits:\n\n432ecb3 [Cheng Hao] Use analyzed instead of logical in HiveComparisonTest","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/dfd2982bc7047732197f1d9ad77221e9c6076fc2"},{"sha":"947802cb0de581e51f8141f6663e896de3d753ce","author":{"email":"7acf0e49bc9acc50da7e38d41884e81d5f2df305@qq.com","name":"DoingDone9"},"message":"[SPARK-6243][SQL] The Operation of match did not conside the scenarios that order.dataType does not match NativeType\n\nIt did not conside that order.dataType does not match NativeType. So i add \"case other => ...\" for other cenarios.\n\nAuthor: DoingDone9 <799203320@qq.com>\n\nCloses #4959 from DoingDone9/case_ and squashes the following commits:\n\n6278846 [DoingDone9] Update rows.scala\ncb1852d [DoingDone9] Merge pull request #2 from apache/master\nc3f046f [DoingDone9] Merge pull request #1 from apache/master","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/947802cb0de581e51f8141f6663e896de3d753ce"},{"sha":"052dee0707830cfd3cd8821ecc3471a37ede294a","author":{"email":"17b9e1c64588c7fa6419b4d29dc1f4426279ba01@databricks.com","name":"Michael Armbrust"},"message":"[SPARK-6686][SQL] Use resolved output instead of names for toDF rename\n\nThis is a workaround for a problem reported on the user list.  This doesn't fix the core problem, but in general is a more robust way to do renames.\n\nAuthor: Michael Armbrust <michael@databricks.com>\n\nCloses #5337 from marmbrus/toDFrename and squashes the following commits:\n\n6a3159d [Michael Armbrust] [SPARK-6686][SQL] Use resolved output instead of names for toDF rename","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/052dee0707830cfd3cd8821ecc3471a37ede294a"},{"sha":"45134ec920c3766c22aefd4366b4b60ec99bd810","author":{"email":"8283fd24ce58e0bb0d6bf349e9064fef42c4d84e@cloudera.com","name":"Marcelo Vanzin"},"message":"[SPARK-6650] [core] Stop ExecutorAllocationManager when context stops.\n\nThis fixes the thread leak. I also changed the unit test to keep track\nof allocated contexts and make sure they're closed after tests are\nrun; this is needed since some tests use this pattern:\n\n    val sc = createContext()\n    doSomethingThatMayThrow()\n    sc.stop()\n\nAuthor: Marcelo Vanzin <vanzin@cloudera.com>\n\nCloses #5311 from vanzin/SPARK-6650 and squashes the following commits:\n\n652c73b [Marcelo Vanzin] Nits.\n5711512 [Marcelo Vanzin] More exception safety.\ncc5a744 [Marcelo Vanzin] Stop alloc manager before scheduler.\n9886f69 [Marcelo Vanzin] [SPARK-6650] [core] Stop ExecutorAllocationManager when context stops.","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/45134ec920c3766c22aefd4366b4b60ec99bd810"},{"sha":"4b82bd730a24f96d94dfea87420cfaa4253a5ccb","author":{"email":"8adc4758e0403b3bd961e4e414291eaa9ba64098@databricks.com","name":"Yin Huai"},"message":"[SPARK-6575][SQL] Converted Parquet Metastore tables no longer cache metadata\n\nhttps://issues.apache.org/jira/browse/SPARK-6575\n\nAuthor: Yin Huai <yhuai@databricks.com>\n\nCloses #5339 from yhuai/parquetRelationCache and squashes the following commits:\n\n83d9846 [Yin Huai] Remove unnecessary change.\nc0dc7a4 [Yin Huai] Cache converted parquet relations.","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/4b82bd730a24f96d94dfea87420cfaa4253a5ccb"},{"sha":"8a0aa81ca37d337423db60edb09cf264cc2c6498","author":{"email":"1bbfbf3ad5c4de88db64d0be3ca037b409d25231@huawei.com","name":"KaiXinXiaoLei"},"message":"[CORE] The descriptionof jobHistory config should be spark.history.fs.logDirectory\n\nThe config option  is spark.history.fs.logDirectory, not spark.fs.history.logDirectory. So the descriptionof  should be changed. Thanks.\n\nAuthor: KaiXinXiaoLei <huleilei1@huawei.com>\n\nCloses #5332 from KaiXinXiaoLei/historyConfig and squashes the following commits:\n\n5ffbfb5 [KaiXinXiaoLei] the describe of jobHistory config is error","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/8a0aa81ca37d337423db60edb09cf264cc2c6498"},{"sha":"6e1c1ec67bc4d7e5700f523ec08db6bb25bd2302","author":{"email":"05f4ff6cfd932210ddb781d81debbbf760fcab75@gmail.com","name":"freeman"},"message":"[SPARK-6345][STREAMING][MLLIB] Fix for training with prediction\n\nThis patch fixes a reported bug causing model updates to not properly propagate to model predictions during streaming regression. These minor changes in model declaration fix the problem, and I expanded the tests to include the scenario in which the bug was arising. The two new tests failed prior to the patch and now pass.\n\ncc mengxr\n\nAuthor: freeman <the.freeman.lab@gmail.com>\n\nCloses #5037 from freeman-lab/train-predict-fix and squashes the following commits:\n\n3af953e [freeman] Expand test coverage to include combined training and prediction\n8f84fc8 [freeman] Move model declaration","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/6e1c1ec67bc4d7e5700f523ec08db6bb25bd2302"},{"sha":"440ea31b76aa7e813436271fd63880c7bcd69157","author":{"email":"5d14e010a5b345b784fd604efcc6cebb79f97468@gmail.com","name":"zsxwing"},"message":"[SPARK-6621][Core] Fix the bug that calling EventLoop.stop in EventLoop.onReceive/onError/onStart doesn't call onStop\n\nAuthor: zsxwing <zsxwing@gmail.com>\n\nCloses #5280 from zsxwing/SPARK-6621 and squashes the following commits:\n\n521125e [zsxwing] Fix the bug that calling EventLoop.stop in EventLoop.onReceive and EventLoop.onError doesn't call onStop","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/440ea31b76aa7e813436271fd63880c7bcd69157"},{"sha":"c42c3fc7f7b79a1f6ce990d39b5d9d14ab19fcf0","author":{"email":"8adc4758e0403b3bd961e4e414291eaa9ba64098@databricks.com","name":"Yin Huai"},"message":"[SPARK-6575][SQL] Converted Parquet Metastore tables no longer cache metadata\n\nhttps://issues.apache.org/jira/browse/SPARK-6575\n\nAuthor: Yin Huai <yhuai@databricks.com>\n\nThis patch had conflicts when merged, resolved by\nCommitter: Cheng Lian <lian@databricks.com>\n\nCloses #5339 from yhuai/parquetRelationCache and squashes the following commits:\n\nb0e1a42 [Yin Huai] Address comments.\n83d9846 [Yin Huai] Remove unnecessary change.\nc0dc7a4 [Yin Huai] Cache converted parquet relations.","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/c42c3fc7f7b79a1f6ce990d39b5d9d14ab19fcf0"},{"sha":"82701ee25fda64f03899713bc56f82ca6f278151","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-6428] Turn on explicit type checking for public methods.\n\nThis builds on my earlier pull requests and turns on the explicit type checking in scalastyle.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #5342 from rxin/SPARK-6428 and squashes the following commits:\n\n7b531ab [Reynold Xin] import ordering\n2d9a8a5 [Reynold Xin] jl\ne668b1c [Reynold Xin] override\n9b9e119 [Reynold Xin] Parenthesis.\n82e0cf5 [Reynold Xin] [SPARK-6428] Turn on explicit type checking for public methods.","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/82701ee25fda64f03899713bc56f82ca6f278151"},{"sha":"b0d884f044fea1c954da77073f3556cd9ab1e922","author":{"email":"06fa905d7f2aaced6dc72e9511c71a2a51e8aead@exigencecorp.com","name":"Stephen Haberman"},"message":"[SPARK-6560][CORE] Do not suppress exceptions from writer.write.\n\nIf there is a failure in the Hadoop backend while calling\nwriter.write, we should remember this original exception,\nand try to call writer.close(), but if that fails as well,\nstill report the original exception.\n\nNote that, if writer.write fails, it is likely that writer\nwas left in an invalid state, and so actually makes it more\nlikely that writer.close will also fail. Which just increases\nthe chances for writer.write's exception to be suppressed.\n\nThis patch introduces an admittedly potentially too cute\nUtils.tryWithSafeFinally method to handle the try/finally\ngyrations.\n\nAuthor: Stephen Haberman <stephen@exigencecorp.com>\n\nCloses #5223 from stephenh/do_not_suppress_writer_exception and squashes the following commits:\n\nc7ad53f [Stephen Haberman] [SPARK-6560][CORE] Do not suppress exceptions from writer.write.","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/b0d884f044fea1c954da77073f3556cd9ab1e922"},{"sha":"b52c7f9fc87a1b9a039724e1dac8b30554f75196","author":{"email":"2e15c7978687116f75c8ca95b4e65d755c10220e@palantir.com","name":"Omede Firouz"},"message":"[MLLIB] Remove println in LogisticRegression.scala\n\nThere's no corresponding printing in linear regression. Here was my previous PR (something weird happened and I can't reopen it) https://github.com/apache/spark/pull/5272\n\nAuthor: Omede Firouz <ofirouz@palantir.com>\n\nCloses #5338 from oefirouz/println and squashes the following commits:\n\n3f3dbf4 [Omede Firouz] [MLLIB] Remove println","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/b52c7f9fc87a1b9a039724e1dac8b30554f75196"},{"sha":"512a2f191a6b53699373b6588f316b4437050425","author":{"email":"bb02c6aa60d4a0a091a9679b63db9e2367e07b50@me.com","name":"lewuathe"},"message":"[SPARK-6615][MLLIB] Python API for Word2Vec\n\nThis is the sub-task of SPARK-6254.\nWrap missing method for `Word2Vec` and `Word2VecModel`.\n\nAuthor: lewuathe <lewuathe@me.com>\n\nCloses #5296 from Lewuathe/SPARK-6615 and squashes the following commits:\n\nf14c304 [lewuathe] Reorder tests\n1d326b9 [lewuathe] Merge master\ne2bedfb [lewuathe] Modify test cases\nafb866d [lewuathe] [SPARK-6615] Python API for Word2Vec","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/512a2f191a6b53699373b6588f316b4437050425"},{"sha":"dc6dff248d8f5d7de22af64b0586dfe3885731df","author":{"email":"d24118b4297e1ecf0c990d6935f5f9e6e4fd7701@gmail.com","name":"Liang-Chi Hsieh"},"message":"[Minor][SQL] Fix typo\n\nJust fix a typo.\n\nAuthor: Liang-Chi Hsieh <viirya@gmail.com>\n\nCloses #5352 from viirya/fix_a_typo and squashes the following commits:\n\n303b2d2 [Liang-Chi Hsieh] Fix typo.","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/dc6dff248d8f5d7de22af64b0586dfe3885731df"},{"sha":"c23ba81b8cf86c3a085de8ddfef9403ff6fcd87f","author":{"email":"c135ccafd9da216e05a17f80c5af6fcd4e4020ba@asiainfo.com","name":"guowei2"},"message":"[SPARK-5203][SQL] fix union with different decimal type\n\n   When union non-decimal types with decimals, we use the following rules:\n      - FIRST `intTypeToFixed`, then fixed union decimals with precision/scale p1/s2 and p2/s2  will be promoted to\n      DecimalType(max(p1, p2), max(s1, s2))\n      - FLOAT and DOUBLE cause fixed-length decimals to turn into DOUBLE (this is the same as Hive,\n      but note that unlimited decimals are considered bigger than doubles in WidenTypes)\n\nAuthor: guowei2 <guowei2@asiainfo.com>\n\nCloses #4004 from guowei2/SPARK-5203 and squashes the following commits:\n\nff50f5f [guowei2] fix code style\n11df1bf [guowei2] fix decimal union with double, double->Decimal(15,15)\n0f345f9 [guowei2] fix structType merge with decimal\n101ed4d [guowei2] fix build error after rebase\n0b196e4 [guowei2] code style\nfe2c2ca [guowei2] handle union decimal precision in 'DecimalPrecision'\n421d840 [guowei2] fix union types for decimal precision\nef2c661 [guowei2] fix union with different decimal type","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/c23ba81b8cf86c3a085de8ddfef9403ff6fcd87f"},{"sha":"2c43ea38ee0db6b292c14baf6bc6f8d16f509c9d","author":{"email":"17d07ce14ed8a23e11d276b3691423f0ca203e51@capitalone.com","name":"Ilya Ganelin"},"message":"[SPARK-6492][CORE] SparkContext.stop() can deadlock when DAGSchedulerEventProcessLoop dies\n\nI've added a timeout and retry loop around the SparkContext shutdown code that should fix this deadlock. If a SparkContext shutdown is in progress when another thread comes knocking, it will wait for 10 seconds for the lock, then fall through where the outer loop will re-submit the request.\n\nAuthor: Ilya Ganelin <ilya.ganelin@capitalone.com>\n\nCloses #5277 from ilganeli/SPARK-6492 and squashes the following commits:\n\n8617a7e [Ilya Ganelin] Resolved merge conflict\n2fbab66 [Ilya Ganelin] Added MIMA Exclude\na0e2c70 [Ilya Ganelin] Deleted stale imports\nfa28ce7 [Ilya Ganelin] reverted to just having a single stopped\n76fc825 [Ilya Ganelin] Updated to use atomic booleans instead of the synchronized vars\n6e8a7f7 [Ilya Ganelin] Removing unecessary null check for now since i'm not fixing stop ordering yet\ncdf7073 [Ilya Ganelin] [SPARK-6492] Moved stopped=true back to the start of the shutdown sequence so this can be addressed in a seperate PR\n7fb795b [Ilya Ganelin] Spacing\nb7a0c5c [Ilya Ganelin] Import ordering\ndf8224f [Ilya Ganelin] Added comment for added lock\n343cb94 [Ilya Ganelin] [SPARK-6492] Added timeout/retry logic to fix a deadlock in SparkContext shutdown","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/2c43ea38ee0db6b292c14baf6bc6f8d16f509c9d"},{"sha":"88504b75ee610e14d7ceed8b038fa698a3d14f81","author":{"email":"5d14e010a5b345b784fd604efcc6cebb79f97468@gmail.com","name":"zsxwing"},"message":"[SPARK-6640][Core] Fix the race condition of creating HeartbeatReceiver and retrieving HeartbeatReceiver\n\nThis PR moved the code of creating `HeartbeatReceiver` above the code of creating `schedulerBackend` to resolve the race condition.\n\nAuthor: zsxwing <zsxwing@gmail.com>\n\nCloses #5306 from zsxwing/SPARK-6640 and squashes the following commits:\n\n840399d [zsxwing] Don't send TaskScheduler through Akka\na90616a [zsxwing] Fix docs\ndd202c7 [zsxwing] Fix typo\nd7c250d [zsxwing] Fix the race condition of creating HeartbeatReceiver and retrieving HeartbeatReceiver","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/88504b75ee610e14d7ceed8b038fa698a3d14f81"},{"sha":"ffe8cc9a25454ee4f451f6ee3ec6d1e934b47ca2","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"Closes #3158","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/ffe8cc9a25454ee4f451f6ee3ec6d1e934b47ca2"},{"sha":"14632b7942c02a332c4d3814fb6b2611e3f76fc7","author":{"email":"8283fd24ce58e0bb0d6bf349e9064fef42c4d84e@cloudera.com","name":"Marcelo Vanzin"},"message":"[SPARK-6688] [core] Always use resolved URIs in EventLoggingListener.\n\nAuthor: Marcelo Vanzin <vanzin@cloudera.com>\n\nCloses #5340 from vanzin/SPARK-6688 and squashes the following commits:\n\nccfddd9 [Marcelo Vanzin] Resolve at the source.\n20d2a34 [Marcelo Vanzin] [SPARK-6688] [core] Always use resolved URIs in EventLoggingListener.","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/14632b7942c02a332c4d3814fb6b2611e3f76fc7"},{"sha":"26b415e15970d02523f0df459557b09ffda0c8c1","author":{"email":"d24118b4297e1ecf0c990d6935f5f9e6e4fd7701@gmail.com","name":"Liang-Chi Hsieh"},"message":"[SPARK-6647][SQL] Make trait StringComparison as BinaryPredicate and fix unit tests of string data source Filter\n\nNow trait `StringComparison` is a `BinaryExpression`. In fact, it should be a `BinaryPredicate`.\n\nBy making `StringComparison` as `BinaryPredicate`, we can throw error when a `expressions.Predicate` can't translate to a data source `Filter` in function `selectFilters`.\n\nWithout this modification, because we will wrap a `Filter` outside the scanned results in `pruneFilterProjectRaw`, we can't detect about something is wrong in translating predicates to filters in `selectFilters`.\n\nThe unit test of #5285 demonstrates such problem. In that pr, even `expressions.Contains` is not properly translated to `sources.StringContains`, the filtering is still performed by the `Filter` and so the test passes.\n\nOf course, by doing this modification, all `expressions.Predicate` classes need to have its data source `Filter` correspondingly.\n\nThere is a small bug in `FilteredScanSuite` for doing `StringEndsWith` filter. This pr also fixes it.\n\nAuthor: Liang-Chi Hsieh <viirya@gmail.com>\n\nCloses #5309 from viirya/translate_predicate and squashes the following commits:\n\nb176385 [Liang-Chi Hsieh] Address comment.\n275a493 [Liang-Chi Hsieh] More properly test for StringStartsWith, StringEndsWith and StringContains.\ncaf2347 [Liang-Chi Hsieh] Make trait StringComparison as BinaryPredicate and throw error when Predicate can't translate to data source Filter.","distinct":true,"url":"https://api.github.com/repos/davies/spark/commits/26b415e15970d02523f0df459557b09ffda0c8c1"}]}