{"push_id":752211441,"size":943,"distinct_size":3,"ref":"refs/heads/SPARK-9570","head":"437a4d451147f179617628a672eaa795b3b76ea0","before":"a1e3649c8775d71ca78796b6544284e942ac1331","commits":[{"sha":"32e3cdaa647722671adcb5068bd5ffbf2f157806","author":{"email":"452ae006f36be6c70342f33f64bb6f6eb552689c@gmail.com","name":"Yanbo Liang"},"message":"[SPARK-7604] [MLLIB] Python API for PCA and PCAModel\n\nPython API for PCA and PCAModel\n\nAuthor: Yanbo Liang <ybliang8@gmail.com>\n\nCloses #6315 from yanboliang/spark-7604 and squashes the following commits:\n\n1d58734 [Yanbo Liang] remove transform() in PCAModel, use default behavior\n4d9d121 [Yanbo Liang] Python API for PCA and PCAModel","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/32e3cdaa647722671adcb5068bd5ffbf2f157806"},{"sha":"83cdfd84f8ca679e1ec451ed88b946e8e7f13a94","author":{"email":"e6f14efef9fae0ae8f92d04007faea68035873ce@databricks.com","name":"Cheng Lian"},"message":"[SPARK-8508] [SQL] Ignores a test case to cleanup unnecessary testing output until #6882 is merged\n\nCurrently [the test case for SPARK-7862] [1] writes 100,000 lines of integer triples to stderr and makes Jenkins build output unnecessarily large and it's hard to debug other build errors. A proper fix is on the way in #6882. This PR ignores this test case temporarily until #6882 is merged.\n\n[1]: https://github.com/apache/spark/pull/6404/files#diff-1ea02a6fab84e938582f7f87cc4d9ea1R641\n\nAuthor: Cheng Lian <lian@databricks.com>\n\nCloses #6925 from liancheng/spark-8508 and squashes the following commits:\n\n41e5b47 [Cheng Lian] Ignores the test case until #6882 is merged","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/83cdfd84f8ca679e1ec451ed88b946e8e7f13a94"},{"sha":"a1894422ad6b3335c84c73ba9466da6677d893cb","author":{"email":"461476587780aa9fa5611ea6dc3912c146a91760@databricks.com","name":"Joseph K. Bradley"},"message":"[SPARK-7715] [MLLIB] [ML] [DOC] Updated MLlib programming guide for release 1.4\n\nReorganized docs a bit.  Added migration guides.\n\n**Q**: Do we want to say more for the 1.3 -> 1.4 migration guide for ```spark.ml```?  It would be a lot.\n\nCC: mengxr\n\nAuthor: Joseph K. Bradley <joseph@databricks.com>\n\nCloses #6897 from jkbradley/ml-guide-1.4 and squashes the following commits:\n\n4bf26d6 [Joseph K. Bradley] tiny fix\n8085067 [Joseph K. Bradley] fixed spacing/layout issues in ml guide from previous commit in this PR\n6cd5c78 [Joseph K. Bradley] Updated MLlib programming guide for release 1.4","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/a1894422ad6b3335c84c73ba9466da6677d893cb"},{"sha":"47c1d5629373566df9d12fdc4ceb22f38b869482","author":{"email":"fcf9e35ad50aab44387f6ff709c4c4bf73ee4abe@gmail.com","name":"Mike Dusenberry"},"message":"[SPARK-7426] [MLLIB] [ML] Updated Attribute.fromStructField to allow any NumericType.\n\nUpdated `Attribute.fromStructField` to allow any `NumericType`, rather than just `DoubleType`, and added unit tests for a few of the other NumericTypes.\n\nAuthor: Mike Dusenberry <dusenberrymw@gmail.com>\n\nCloses #6540 from dusenberrymw/SPARK-7426_AttributeFactory.fromStructField_Should_Allow_NumericTypes and squashes the following commits:\n\n87fecb3 [Mike Dusenberry] Updated Attribute.fromStructField to allow any NumericType, rather than just DoubleType, and added unit tests for a few of the other NumericTypes.","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/47c1d5629373566df9d12fdc4ceb22f38b869482"},{"sha":"0818fdec3733ec5c0a9caa48a9c0f2cd25f84d13","author":{"email":"e6f14efef9fae0ae8f92d04007faea68035873ce@databricks.com","name":"Cheng Lian"},"message":"[SPARK-8406] [SQL] Adding UUID to output file name to avoid accidental overwriting\n\nThis PR fixes a Parquet output file name collision bug which may cause data loss.  Changes made:\n\n1.  Identify each write job issued by `InsertIntoHadoopFsRelation` with a UUID\n\n    All concrete data sources which extend `HadoopFsRelation` (Parquet and ORC for now) must use this UUID to generate task output file path to avoid name collision.\n\n2.  Make `TestHive` use a local mode `SparkContext` with 32 threads to increase parallelism\n\n    The major reason for this is that, the original parallelism of 2 is too low to reproduce the data loss issue.  Also, higher concurrency may potentially caught more concurrency bugs during testing phase. (It did help us spotted SPARK-8501.)\n\n3. `OrcSourceSuite` was updated to workaround SPARK-8501, which we detected along the way.\n\nNOTE: This PR is made a little bit more complicated than expected because we hit two other bugs on the way and have to work them around. See [SPARK-8501] [1] and [SPARK-8513] [2].\n\n[1]: https://github.com/liancheng/spark/tree/spark-8501\n[2]: https://github.com/liancheng/spark/tree/spark-8513\n\n----\n\nSome background and a summary of offline discussion with yhuai about this issue for better understanding:\n\nIn 1.4.0, we added `HadoopFsRelation` to abstract partition support of all data sources that are based on Hadoop `FileSystem` interface.  Specifically, this makes partition discovery, partition pruning, and writing dynamic partitions for data sources much easier.\n\nTo support appending, the Parquet data source tries to find out the max part number of part-files in the destination directory (i.e., `<id>` in output file name `part-r-<id>.gz.parquet`) at the beginning of the write job.  In 1.3.0, this step happens on driver side before any files are written.  However, in 1.4.0, this is moved to task side.  Unfortunately, for tasks scheduled later, they may see wrong max part number generated of files newly written by other finished tasks within the same job.  This actually causes a race condition.  In most cases, this only causes nonconsecutive part numbers in output file names.  But when the DataFrame contains thousands of RDD partitions, it's likely that two tasks may choose the same part number, then one of them gets overwritten by the other.\n\nBefore `HadoopFsRelation`, Spark SQL already supports appending data to Hive tables.  From a user's perspective, these two look similar.  However, they differ a lot internally.  When data are inserted into Hive tables via Spark SQL, `InsertIntoHiveTable` simulates Hive's behaviors:\n\n1.  Write data to a temporary location\n\n2.  Move data in the temporary location to the final destination location using\n\n    -   `Hive.loadTable()` for non-partitioned table\n    -   `Hive.loadPartition()` for static partitions\n    -   `Hive.loadDynamicPartitions()` for dynamic partitions\n\nThe important part is that, `Hive.copyFiles()` is invoked in step 2 to move the data to the destination directory (I found the name is kinda confusing since no \"copying\" occurs here, we are just moving and renaming stuff).  If a file in the source directory and another file in the destination directory happen to have the same name, say `part-r-00001.parquet`, the former is moved to the destination directory and renamed with a `_copy_N` postfix (`part-r-00001_copy_1.parquet`).  That's how Hive handles appending and avoids name collision between different write jobs.\n\nSome alternatives fixes considered for this issue:\n\n1.  Use a similar approach as Hive\n\n    This approach is not preferred in Spark 1.4.0 mainly because file metadata operations in S3 tend to be slow, especially for tables with lots of file and/or partitions.  That's why `InsertIntoHadoopFsRelation` just inserts to destination directory directly, and is often used together with `DirectParquetOutputCommitter` to reduce latency when working with S3.  This means, we don't have the chance to do renaming, and must avoid name collision from the very beginning.\n\n2.  Same as 1.3, just move max part number detection back to driver side\n\n    This isn't doable because unlike 1.3, 1.4 also takes dynamic partitioning into account.  When inserting into dynamic partitions, we don't know which partition directories will be touched on driver side before issuing the write job.  Checking all partition directories is simply too expensive for tables with thousands of partitions.\n\n3.  Add extra component to output file names to avoid name collision\n\n    This seems to be the only reasonable solution for now.  To be more specific, we need a JOB level unique identifier to identify all write jobs issued by `InsertIntoHadoopFile`.  Notice that TASK level unique identifiers can NOT be used.  Because in this way a speculative task will write to a different output file from the original task.  If both tasks succeed, duplicate output will be left behind.  Currently, the ORC data source adds `System.currentTimeMillis` to the output file name for uniqueness.  This doesn't work because of exactly the same reason.\n\n    That's why this PR adds a job level random UUID in `BaseWriterContainer` (which is used by `InsertIntoHadoopFsRelation` to issue write jobs).  The drawback is that record order is not preserved any more (output files of a later job may be listed before those of a earlier job).  However, we never promise to preserve record order when writing data, and Hive doesn't promise this either because the `_copy_N` trick breaks the order.\n\nAuthor: Cheng Lian <lian@databricks.com>\n\nCloses #6864 from liancheng/spark-8406 and squashes the following commits:\n\ndb7a46a [Cheng Lian] More comments\nf5c1133 [Cheng Lian] Addresses comments\n85c478e [Cheng Lian] Workarounds SPARK-8513\n088c76c [Cheng Lian] Adds comment about SPARK-8501\n99a5e7e [Cheng Lian] Uses job level UUID in SimpleTextRelation and avoids double task abortion\n4088226 [Cheng Lian] Works around SPARK-8501\n1d7d206 [Cheng Lian] Adds more logs\n8966bbb [Cheng Lian] Fixes Scala style issue\n18b7003 [Cheng Lian] Uses job level UUID to take speculative tasks into account\n3806190 [Cheng Lian] Lets TestHive use all cores by default\n748dbd7 [Cheng Lian] Adding UUID to output file name to avoid accidental overwriting","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/0818fdec3733ec5c0a9caa48a9c0f2cd25f84d13"},{"sha":"42a1f716fa35533507784be5e9117a984a03e62d","author":{"email":"90d3910746bfdbed36d8ed63997a5bd329b7fc0a@gmail.com","name":"Stefano Parmesan"},"message":"[SPARK-8429] [EC2] Add ability to set additional tags\n\nAdd the `--additional-tags` parameter that allows to set additional tags to all the created instances (masters and slaves).\n\nThe user can specify multiple tags by separating them with a comma (`,`), while each tag name and value should be separated by a colon (`:`); for example, `Task:MySparkProject,Env:production` would add two tags, `Task` and `Env`, with the given values.\n\nAuthor: Stefano Parmesan <s.parmesan@gmail.com>\n\nCloses #6857 from armisael/patch-1 and squashes the following commits:\n\nc5ac92c [Stefano Parmesan] python style (pep8)\n8e614f1 [Stefano Parmesan] Set multiple tags in a single request\nbfc56af [Stefano Parmesan] Address SPARK-7900 by inceasing sleep time\ndaf8615 [Stefano Parmesan] Add ability to set additional tags","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/42a1f716fa35533507784be5e9117a984a03e62d"},{"sha":"ba8a4537fee7d85f968cccf8d1c607731daae307","author":{"email":"ade00d7d41ee0746f906c5a6541e6765081dd18d@gmail.com","name":"Pradeep Chhetri"},"message":"[SPARK-8482] Added M4 instances to the list.\n\nAWS recently added M4 instances (https://aws.amazon.com/blogs/aws/the-new-m4-instance-type-bonus-price-reduction-on-m3-c4/).\n\nAuthor: Pradeep Chhetri <pradeep.chhetri89@gmail.com>\n\nCloses #6899 from pradeepchhetri/master and squashes the following commits:\n\n4f4ea79 [Pradeep Chhetri] Added t2.large instance\n3d2bb6c [Pradeep Chhetri] Added M4 instances to the list","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/ba8a4537fee7d85f968cccf8d1c607731daae307"},{"sha":"5d89d9f00ba4d6d0767a4c4964d3af324bf6f14b","author":{"email":"bc4a65f227ddcd81bb538c67e810a3e11696e8bf@gmail.com","name":"Yu ISHIKAWA"},"message":"[SPARK-8511] [PYSPARK] Modify a test to remove a saved model in `regression.py`\n\n[[SPARK-8511] Modify a test to remove a saved model in `regression.py` - ASF JIRA](https://issues.apache.org/jira/browse/SPARK-8511)\n\nAuthor: Yu ISHIKAWA <yuu.ishikawa@gmail.com>\n\nCloses #6926 from yu-iskw/SPARK-8511 and squashes the following commits:\n\n7cd0948 [Yu ISHIKAWA] Use `shutil.rmtree()` to temporary directories for saving model testings, instead of `os.removedirs()`\n4a01c9e [Yu ISHIKAWA] [SPARK-8511][pyspark] Modify a test to remove a saved model in `regression.py`","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/5d89d9f00ba4d6d0767a4c4964d3af324bf6f14b"},{"sha":"da7bbb9435dae9a3bedad578599d96ea858f349e","author":{"email":"ab56adc4e61f1b489818a700493ea97fb9f9e75e@outlook.com","name":"Wenchen Fan"},"message":"[SPARK-8104] [SQL] auto alias expressions in analyzer\n\nCurrently we auto alias expression in parser. However, during parser phase we don't have enough information to do the right alias. For example, Generator that has more than 1 kind of element need MultiAlias, ExtractValue don't need Alias if it's in middle of a ExtractValue chain.\n\nAuthor: Wenchen Fan <cloud0fan@outlook.com>\n\nCloses #6647 from cloud-fan/alias and squashes the following commits:\n\n552eba4 [Wenchen Fan] fix python\n5b5786d [Wenchen Fan] fix agg\n73a90cb [Wenchen Fan] fix case-preserve of ExtractValue\n4cfd23c [Wenchen Fan] fix order by\nd18f401 [Wenchen Fan] refine\n9f07359 [Wenchen Fan] address comments\n39c1aef [Wenchen Fan] small fix\n33640ec [Wenchen Fan] auto alias expressions in analyzer","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/da7bbb9435dae9a3bedad578599d96ea858f349e"},{"sha":"5ab9fcfb01a0ad2f6c103f67c1a785d3b49e33f0","author":{"email":"8adc4758e0403b3bd961e4e414291eaa9ba64098@databricks.com","name":"Yin Huai"},"message":"[SPARK-8532] [SQL] In Python's DataFrameWriter, save/saveAsTable/json/parquet/jdbc always override mode\n\nhttps://issues.apache.org/jira/browse/SPARK-8532\n\nThis PR has two changes. First, it fixes the bug that save actions (i.e. `save/saveAsTable/json/parquet/jdbc`) always override mode. Second, it adds input argument `partitionBy` to `save/saveAsTable/parquet`.\n\nAuthor: Yin Huai <yhuai@databricks.com>\n\nCloses #6937 from yhuai/SPARK-8532 and squashes the following commits:\n\nf972d5d [Yin Huai] davies's comment.\nd37abd2 [Yin Huai] style.\nd21290a [Yin Huai] Python doc.\n889eb25 [Yin Huai] Minor refactoring and add partitionBy to save, saveAsTable, and parquet.\n7fbc24b [Yin Huai] Use None instead of \"error\" as the default value of mode since JVM-side already uses \"error\" as the default value.\nd696dff [Yin Huai] Python style.\n88eb6c4 [Yin Huai] If mode is \"error\", do not call mode method.\nc40c461 [Yin Huai] Regression test.","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/5ab9fcfb01a0ad2f6c103f67c1a785d3b49e33f0"},{"sha":"afe35f0519bc7dcb85010a7eedcff854d4fc313a","author":{"email":"9835e449f9a666ddd8ba8aeb460af39a5130691b@databricks.com","name":"Feynman Liang"},"message":"[SPARK-8455] [ML] Implement n-gram feature transformer\n\nImplementation of n-gram feature transformer for ML.\n\nAuthor: Feynman Liang <fliang@databricks.com>\n\nCloses #6887 from feynmanliang/ngram-featurizer and squashes the following commits:\n\nd2c839f [Feynman Liang] Make n > input length yield empty output\n9fadd36 [Feynman Liang] Add empty and corner test cases, fix names and spaces\nfe93873 [Feynman Liang] Implement n-gram feature transformer","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/afe35f0519bc7dcb85010a7eedcff854d4fc313a"},{"sha":"b1f3a489efc6f4f9d172344c3345b9b38ae235e0","author":{"email":"bc4a65f227ddcd81bb538c67e810a3e11696e8bf@gmail.com","name":"Yu ISHIKAWA"},"message":"[SPARK-8537] [SPARKR] Add a validation rule about the curly braces in SparkR to `.lintr`\n\n[[SPARK-8537] Add a validation rule about the curly braces in SparkR to `.lintr` - ASF JIRA](https://issues.apache.org/jira/browse/SPARK-8537)\n\nAuthor: Yu ISHIKAWA <yuu.ishikawa@gmail.com>\n\nCloses #6940 from yu-iskw/SPARK-8537 and squashes the following commits:\n\n7eec1a0 [Yu ISHIKAWA] [SPARK-8537][SparkR] Add a validation rule about the curly braces in SparkR to `.lintr`","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/b1f3a489efc6f4f9d172344c3345b9b38ae235e0"},{"sha":"50d3242d6a5530a51dacab249e3f3d49e2d50635","author":{"email":"07c870ffe79a3d57c30fc67475f42d361ec88a03@gmail.com","name":"BenFradet"},"message":"[SPARK-8356] [SQL] Reconcile callUDF and callUdf\n\nDeprecates ```callUdf``` in favor of ```callUDF```.\n\nAuthor: BenFradet <benjamin.fradet@gmail.com>\n\nCloses #6902 from BenFradet/SPARK-8356 and squashes the following commits:\n\nef4e9d8 [BenFradet] deprecated callUDF, use udf instead\n9b1de4d [BenFradet] reinstated unit test for the deprecated callUdf\ncbd80a5 [BenFradet] deprecated callUdf in favor of callUDF","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/50d3242d6a5530a51dacab249e3f3d49e2d50635"},{"sha":"96aa01378e3b3dbb4601d31c7312a311cb65b22e","author":{"email":"c437f8b7af73b0a10ecac061712ff9de6d41089e@databricks.com","name":"Davies Liu"},"message":"[SPARK-8492] [SQL] support binaryType in UnsafeRow\n\nSupport BinaryType in UnsafeRow, just like StringType.\n\nAlso change the layout of StringType and BinaryType in UnsafeRow, by combining offset and size together as Long, which will limit the size of Row to under 2G (given that fact that any single buffer can not be bigger than 2G in JVM).\n\nAuthor: Davies Liu <davies@databricks.com>\n\nCloses #6911 from davies/unsafe_bin and squashes the following commits:\n\nd68706f [Davies Liu] update comment\n519f698 [Davies Liu] address comment\n98a964b [Davies Liu] Merge branch 'master' of github.com:apache/spark into unsafe_bin\n180b49d [Davies Liu] fix zero-out\n22e4c0a [Davies Liu] zero-out padding bytes\n6abfe93 [Davies Liu] fix style\n447dea0 [Davies Liu] support binaryType in UnsafeRow","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/96aa01378e3b3dbb4601d31c7312a311cb65b22e"},{"sha":"1dfb0f7b2aed5ee6d07543fdeac8ff7c777b63b9","author":{"email":"02e0a999c50b1f88df7a8f5a04e1b76b35ea6a88@databricks.com","name":"Andrew Or"},"message":"[HOTFIX] [TESTS] Typo mqqt -> mqtt\n\nThis was introduced in #6866.","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/1dfb0f7b2aed5ee6d07543fdeac8ff7c777b63b9"},{"sha":"860a49ef20cea5711a7f54de0053ea33647e56a7","author":{"email":"ab56adc4e61f1b489818a700493ea97fb9f9e75e@outlook.com","name":"Wenchen Fan"},"message":"[SPARK-7153] [SQL] support all integral type ordinal in GetArrayItem\n\nfirst convert `ordinal` to `Number`, then convert to int type.\n\nAuthor: Wenchen Fan <cloud0fan@outlook.com>\n\nCloses #5706 from cloud-fan/7153 and squashes the following commits:\n\n915db79 [Wenchen Fan] fix 7153","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/860a49ef20cea5711a7f54de0053ea33647e56a7"},{"sha":"6b7f2ceafdcbb014791909747c2210b527305df9","author":{"email":"c437f8b7af73b0a10ecac061712ff9de6d41089e@databricks.com","name":"Davies Liu"},"message":"[SPARK-8307] [SQL] improve timestamp from parquet\n\nThis PR change to convert julian day to unix timestamp directly (without Calendar and Timestamp).\n\ncc adrian-wang rxin\n\nAuthor: Davies Liu <davies@databricks.com>\n\nCloses #6759 from davies/improve_ts and squashes the following commits:\n\n849e301 [Davies Liu] Merge branch 'master' of github.com:apache/spark into improve_ts\nb0e4cad [Davies Liu] Merge branch 'master' of github.com:apache/spark into improve_ts\n8e2d56f [Davies Liu] address comments\n634b9f5 [Davies Liu] fix mima\n4891efb [Davies Liu] address comment\nbfc437c [Davies Liu] fix build\nae5979c [Davies Liu] Merge branch 'master' of github.com:apache/spark into improve_ts\n602b969 [Davies Liu] remove jodd\n2f2e48c [Davies Liu] fix test\n8ace611 [Davies Liu] fix mima\n212143b [Davies Liu] fix mina\nc834108 [Davies Liu] Merge branch 'master' of github.com:apache/spark into improve_ts\na3171b8 [Davies Liu] Merge branch 'master' of github.com:apache/spark into improve_ts\n5233974 [Davies Liu] fix scala style\n361fd62 [Davies Liu] address comments\nea196d4 [Davies Liu] improve timestamp from parquet","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/6b7f2ceafdcbb014791909747c2210b527305df9"},{"sha":"13321e65559f6354ec1287a690580fd6f498ef89","author":{"email":"50c4b4c4c26f83ee4bbc45a85db1c3c80b5aecdb@intel.com","name":"Cheng Hao"},"message":"[SPARK-7859] [SQL] Collect_set() behavior differences which fails the unit test under jdk8\n\nTo reproduce that:\n```\nJAVA_HOME=/home/hcheng/Java/jdk1.8.0_45 | build/sbt -Phadoop-2.3 -Phive  'test-only org.apache.spark.sql.hive.execution.HiveWindowFunctionQueryWithoutCodeGenSuite'\n```\n\nA simple workaround to fix that is update the original query, for getting the output size instead of the exact elements of the array (output by collect_set())\n\nAuthor: Cheng Hao <hao.cheng@intel.com>\n\nCloses #6402 from chenghao-intel/windowing and squashes the following commits:\n\n99312ad [Cheng Hao] add order by for the select clause\nedf8ce3 [Cheng Hao] update the code as suggested\n7062da7 [Cheng Hao] fix the collect_set() behaviour differences under different versions of JDK","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/13321e65559f6354ec1287a690580fd6f498ef89"},{"sha":"c4d2343966cbae40a8271a2e6cad66227d2f8249","author":{"email":"cbb7353e6d953ef360baf960c122346276c6e320@databricks.com","name":"Patrick Wendell"},"message":"MAINTENANCE: Automated closing of pull requests.\n\nThis commit exists to close the following pull requests on Github:\n\nCloses #2849 (close requested by 'srowen')\nCloses #2786 (close requested by 'andrewor14')\nCloses #4678 (close requested by 'JoshRosen')\nCloses #5457 (close requested by 'andrewor14')\nCloses #3346 (close requested by 'andrewor14')\nCloses #6518 (close requested by 'andrewor14')\nCloses #5403 (close requested by 'pwendell')\nCloses #2110 (close requested by 'srowen')","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/c4d2343966cbae40a8271a2e6cad66227d2f8249"},{"sha":"44fa7df64daa55bd6eb1f2c219a9701b34e1c2a3","author":{"email":"bc4a65f227ddcd81bb538c67e810a3e11696e8bf@gmail.com","name":"Yu ISHIKAWA"},"message":"[SPARK-8548] [SPARKR] Remove the trailing whitespaces from the SparkR files\n\n[[SPARK-8548] Remove the trailing whitespaces from the SparkR files - ASF JIRA](https://issues.apache.org/jira/browse/SPARK-8548)\n\n- This is the result of `lint-r`\n    https://gist.github.com/yu-iskw/0019b37a2c1167f33986\n\nAuthor: Yu ISHIKAWA <yuu.ishikawa@gmail.com>\n\nCloses #6945 from yu-iskw/SPARK-8548 and squashes the following commits:\n\n0bd567a [Yu ISHIKAWA] [SPARK-8548][SparkR] Remove the trailing whitespaces from the SparkR files","distinct":false,"url":"https://api.github.com/repos/nssalian/spark/commits/44fa7df64daa55bd6eb1f2c219a9701b34e1c2a3"}]}