{"push_id":680538165,"size":287,"distinct_size":287,"ref":"refs/heads/master_nravi","head":"9e59c7167fffddd824d56ddc533e5c63291fa343","before":"171a53cd443a4283ea632af4e74c5eae674a5e4c","commits":[{"sha":"ad92af9dbbd0c4e1224cca26da166382ed4f15b9","author":{"email":"7d45abe39fd242ee2f6575da28f13d66568418b3@oss.nttdata.co.jp","name":"Kousuke Saruta"},"message":"[SPARK-7664] [WEBUI] DAG visualization: Fix incorrect link paths of DAG.\n\nIn JobPage, we can jump a StagePage when we click corresponding box of DAG viz but the link path is incorrect.\n\nWhen we click a box like as follows ...\n![screenshot_from_2015-05-15 19 24 25](https://cloud.githubusercontent.com/assets/4736016/7651528/5f7ef824-fb3c-11e4-9518-8c9ade2dff7a.png)\n\nWe jump to index page.\n![screenshot_from_2015-05-15 19 24 45](https://cloud.githubusercontent.com/assets/4736016/7651534/6d666274-fb3c-11e4-971c-c3f2dc2b1da2.png)\n\nAuthor: Kousuke Saruta <sarutak@oss.nttdata.co.jp>\n\nCloses #6184 from sarutak/fix-link-path-of-dag-viz and squashes the following commits:\n\nfaba3ba [Kousuke Saruta] Fix a incorrect link","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/ad92af9dbbd0c4e1224cca26da166382ed4f15b9"},{"sha":"8e3822a0794b8b18436bd63d6859d40139a77090","author":{"email":"b8ea1e04f138102929bea9d39c280da3d91bca8b@gmail.com","name":"ehnalis"},"message":"[SPARK-7504] [YARN] NullPointerException when initializing SparkContext in YARN-cluster mode\n\nAdded a simple checking for SparkContext.\nAlso added two rational checking against null at AM object.\n\nAuthor: ehnalis <zoltan.zvara@gmail.com>\n\nCloses #6083 from ehnalis/cluster and squashes the following commits:\n\n926bd96 [ehnalis] Moved check to SparkContext.\n7c89b6e [ehnalis] Remove false line.\nea2a5fe [ehnalis] [SPARK-7504] [YARN] NullPointerException when initializing SparkContext in YARN-cluster mode\n4924e01 [ehnalis] [SPARK-7504] [YARN] NullPointerException when initializing SparkContext in YARN-cluster mode\n39e4fa3 [ehnalis] SPARK-7504 [YARN] NullPointerException when initializing SparkContext in YARN-cluster mode\n9f287c5 [ehnalis] [SPARK-7504] [YARN] NullPointerException when initializing SparkContext in YARN-cluster mode","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/8e3822a0794b8b18436bd63d6859d40139a77090"},{"sha":"9b6cf285d0b60848b01b6c7e3421e8ac850a88ab","author":{"email":"7d45abe39fd242ee2f6575da28f13d66568418b3@oss.nttdata.co.jp","name":"Kousuke Saruta"},"message":"[SPARK-7296] Add timeline visualization for stages in the UI.\n\nThis PR builds on #2342 by adding a timeline view for the Stage page,\nshowing how tasks spend their time.\n\nWith this timeline, we can understand following things of a Stage.\n\n* When/where each task ran\n* Total duration of each task\n* Proportion of the time each task spends\n\nAlso, this timeline view can scrollable and zoomable.\n\nAuthor: Kousuke Saruta <sarutak@oss.nttdata.co.jp>\n\nCloses #5843 from sarutak/stage-page-timeline and squashes the following commits:\n\n4ba9604 [Kousuke Saruta] Fixed the order of legends\n16bb552 [Kousuke Saruta] Removed border of legend area\n2e5d605 [Kousuke Saruta] Modified warning message\n16cb2e6 [Kousuke Saruta] Merge branch 'master' of https://github.com/apache/spark into stage-page-timeline\n7ae328f [Kousuke Saruta] Modified code style\nd5f794a [Kousuke Saruta] Fixed performance issues more\n64e6642 [Kousuke Saruta] Merge branch 'master' of https://github.com/apache/spark into stage-page-timeline\ne4a3354 [Kousuke Saruta] minor code style change\n878e3b8 [Kousuke Saruta] Fixed a bug that tooltip remains\nb9d8f1b [Kousuke Saruta] Fixed performance issue\nac8842b [Kousuke Saruta] Fixed layout\n2319739 [Kousuke Saruta] Modified appearances more\n81903ab [Kousuke Saruta] Modified appearances\na79dcc3 [Kousuke Saruta] Modified appearance\n55a390c [Kousuke Saruta] Ignored scalastyle for a line-comment\n29eae3e [Kousuke Saruta] limited to longest 1000 tasks\n2a9e376 [Kousuke Saruta] Minor cleanup\n385b6d2 [Kousuke Saruta] Added link feature\nba1ac3e [Kousuke Saruta] Fixed style\n2ae8520 [Kousuke Saruta] Updated bootstrap-tooltip.js from 2.2.2 to 2.3.2\naf430f1 [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into stage-page-timeline\ne694b8e [Kousuke Saruta] Added timeline view to StagePage\n8f6610c [Kousuke Saruta] Fixed conflict\nb587cf2 [Kousuke Saruta] initial commit\n11fe67d [Kousuke Saruta] Fixed conflict\n79ac03d [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into timeline-viewer-feature\na91abd3 [Kousuke Saruta] Merge branch 'master' of https://github.com/apache/spark into timeline-viewer-feature\nef34a5b [Kousuke Saruta] Implement tooltip using bootstrap\nb09d0c5 [Kousuke Saruta] Move `stroke` and `fill` attribute of rect elements to css\nd3c63c8 [Kousuke Saruta] Fixed a little bit bugs\na36291b [Kousuke Saruta] Merge branch 'master' of https://github.com/apache/spark into timeline-viewer-feature\n28714b6 [Kousuke Saruta] Fixed highlight issue\n0dc4278 [Kousuke Saruta] Addressed most of Patrics's feedbacks\n8110acf [Kousuke Saruta] Added scroll limit to Job timeline\n974a64a [Kousuke Saruta] Removed unused function\nee7a7f0 [Kousuke Saruta] Refactored\n6a91872 [Kousuke Saruta] Temporary commit\n6693f34 [Kousuke Saruta] Added link to job/stage box in the timeline in order to move to corresponding row when we click\n8f88222 [Kousuke Saruta] Added job/stage description\naeed4b1 [Kousuke Saruta] Removed stage timeline\nfc1696c [Kousuke Saruta] Merge branch 'timeline-viewer-feature' of github.com:sarutak/spark into timeline-viewer-feature\n999ccd4 [Kousuke Saruta] Improved scalability\n0fc6a31 [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into timeline-viewer-feature\n19815ae [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into timeline-viewer-feature\n68b7540 [Kousuke Saruta] Merge branch 'timeline-viewer-feature' of github.com:sarutak/spark into timeline-viewer-feature\n52b5f0b [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into timeline-viewer-feature\ndec85db [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into timeline-viewer-feature\nfcdab7d [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into timeline-viewer-feature\ndab7cc1 [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into timeline-viewer-feature\n09cce97 [Kousuke Saruta] Cleanuped\n16f82cf [Kousuke Saruta] Cleanuped\n9fb522e [Kousuke Saruta] Cleanuped\nd05f2c2 [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into timeline-viewer-feature\ne85e9aa [Kousuke Saruta] Cleanup: Added TimelineViewUtils.scala\na76e569 [Kousuke Saruta] Removed unused setting in timeline-view.css\n5ce1b21 [Kousuke Saruta] Added vis.min.js, vis.min.css and vis.map to .rat-exclude\n082f709 [Kousuke Saruta] Added Timeline-View feature for Applications, Jobs and Stages","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/9b6cf285d0b60848b01b6c7e3421e8ac850a88ab"},{"sha":"50da9e89161faa0ecdc1feb3ffee6c822a742034","author":{"email":"3b63531f765b09216524bb8f92815227f77df3d2@intel.com","name":"qhuang"},"message":"[SPARK-7226] [SPARKR] Support math functions in R DataFrame\n\nAuthor: qhuang <qian.huang@intel.com>\n\nCloses #6170 from hqzizania/master and squashes the following commits:\n\nf20c39f [qhuang] add tests units and fixes\n2a7d121 [qhuang] use a function name more familiar to R users\n07aa72e [qhuang] Support math functions in R DataFrame","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/50da9e89161faa0ecdc1feb3ffee6c822a742034"},{"sha":"6e77105e11ff81bfd84561f4e1121111f686df21","author":{"email":"53fdba43521c8d2c2b4ff3a6f81e9f27ad86e7ac@gmail.com","name":"Iulian Dragos"},"message":"[SPARK-7677] [STREAMING] Add Kafka modules to the 2.11 build.\n\nThis is somewhat related to [SPARK-6154](https://issues.apache.org/jira/browse/SPARK-6154), though it only touches Kafka, not the jline dependency for thriftserver.\n\nI tested this locally on 2.11 (./run-tests) and everything looked good (I had to disable mima, because `MimaBuild` harcodes 2.10 for the previous version -- that's another PR).\n\nAuthor: Iulian Dragos <jaguarul@gmail.com>\n\nCloses #6149 from dragos/issue/spark-2.11-kafka and squashes the following commits:\n\naa15d99 [Iulian Dragos] Add Kafka modules to the 2.11 build.","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/6e77105e11ff81bfd84561f4e1121111f686df21"},{"sha":"c8696337e2a5878f3171eb574c0a1365d45814c9","author":{"email":"d24118b4297e1ecf0c990d6935f5f9e6e4fd7701@gmail.com","name":"Liang-Chi Hsieh"},"message":"[SPARK-7556] [ML] [DOC] Add user guide for spark.ml Binarizer, including Scala, Java and Python examples\n\nJIRA: https://issues.apache.org/jira/browse/SPARK-7556\n\nAuthor: Liang-Chi Hsieh <viirya@gmail.com>\n\nCloses #6116 from viirya/binarizer_doc and squashes the following commits:\n\n40cb677 [Liang-Chi Hsieh] Better print out.\n5b7ef1d [Liang-Chi Hsieh] Make examples more clear.\n1bf9c09 [Liang-Chi Hsieh] For comments.\n6cf8cba [Liang-Chi Hsieh] Add user guide for Binarizer.","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/c8696337e2a5878f3171eb574c0a1365d45814c9"},{"sha":"e74545647684b3047248ca3cfee894ac5378dead","author":{"email":"284a00ea9901227c42561196ff8576aed3ac5631@gmail.com","name":"Kay Ousterhout"},"message":"[SPARK-7676] Bug fix and cleanup of stage timeline view\n\ncc pwendell sarutak\n\nThis commit cleans up some unnecessary code, eliminates the feature where when you mouse-over a box in the timeline, the corresponding task is highlighted in the table (because that feature is only useful in the rare case when you have a very small number of tasks, in which case it's easy to figure out the mapping anyway), and fixes a bug where nothing shows up if you try to visualize a stage with only 1 task.\n\nAuthor: Kay Ousterhout <kayousterhout@gmail.com>\n\nCloses #6202 from kayousterhout/SPARK-7676 and squashes the following commits:\n\ndfd29d4 [Kay Ousterhout] [SPARK-7676] Bug fix and cleanup of stage timeline view","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/e74545647684b3047248ca3cfee894ac5378dead"},{"sha":"2c04c8a1aed34cce420b3d30d9e885daa6e03d74","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-7563] OutputCommitCoordinator.stop() should only run on the driver\n\nThis fixes a bug where an executor that exits can cause the driver's OutputCommitCoordinator to stop. To fix this, we use an `isDriver` flag and check it in `stop()`.\n\nSee https://issues.apache.org/jira/browse/SPARK-7563 for more details.\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #6197 from JoshRosen/SPARK-7563 and squashes the following commits:\n\n04b2cc5 [Josh Rosen] [SPARK-7563] OutputCommitCoordinator.stop() should only be executed on the driver","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/2c04c8a1aed34cce420b3d30d9e885daa6e03d74"},{"sha":"cc12a86fb049f2be1f45baf461d202ec356ccf8f","author":{"email":"89674bb0dc6f2e03b27640b006f61a9dd0e29d81@hw11853.local","name":"Ram Sriharsha"},"message":"[SPARK-7575] [ML] [DOC] Example code for OneVsRest\n\nJava and Scala examples for OneVsRest. Fixes the base classifier to be Logistic Regression and accepts the configuration parameters of the base classifier.\n\nAuthor: Ram Sriharsha <rsriharsha@hw11853.local>\n\nCloses #6115 from harsha2010/SPARK-7575 and squashes the following commits:\n\n87ad3c7 [Ram Sriharsha] extra line\nf5d9891 [Ram Sriharsha] Merge branch 'master' into SPARK-7575\n7076084 [Ram Sriharsha] cleanup\ndfd660c [Ram Sriharsha] cleanup\n8703e4f [Ram Sriharsha] update doc\ncb23995 [Ram Sriharsha] fix commandline options for JavaOneVsRestExample\n69e91f8 [Ram Sriharsha] cleanup\n7f4e127 [Ram Sriharsha] cleanup\nd4c40d0 [Ram Sriharsha] Code Review fixes\n461eb38 [Ram Sriharsha] cleanup\ne0106d9 [Ram Sriharsha] Fix typo\n935cf56 [Ram Sriharsha] Try to match Java and Scala Example Commandline options\n5323ff9 [Ram Sriharsha] cleanup\n196a59a [Ram Sriharsha] cleanup\n6adfa0c [Ram Sriharsha] Style Fix\n8cfc5d5 [Ram Sriharsha] [SPARK-7575] Example code for OneVsRest","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/cc12a86fb049f2be1f45baf461d202ec356ccf8f"},{"sha":"adfd366814499c0540a15dd6017091ba8c0f05da","author":{"email":"c437f8b7af73b0a10ecac061712ff9de6d41089e@databricks.com","name":"Davies Liu"},"message":"[SPARK-7073] [SQL] [PySpark] Clean up SQL data type hierarchy in Python\n\nAuthor: Davies Liu <davies@databricks.com>\n\nCloses #6206 from davies/sql_type and squashes the following commits:\n\n33d6860 [Davies Liu] [SPARK-7073] [SQL] [PySpark] Clean up SQL data type hierarchy in Python","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/adfd366814499c0540a15dd6017091ba8c0f05da"},{"sha":"d7b69946cb21cd2781c9ad3e691e54b28efbbf3d","author":{"email":"c437f8b7af73b0a10ecac061712ff9de6d41089e@databricks.com","name":"Davies Liu"},"message":"[SPARK-7543] [SQL] [PySpark] split dataframe.py into multiple files\n\ndataframe.py is splited into column.py, group.py and dataframe.py:\n```\n   360 column.py\n  1223 dataframe.py\n   183 group.py\n```\n\nAuthor: Davies Liu <davies@databricks.com>\n\nCloses #6201 from davies/split_df and squashes the following commits:\n\nfc8f5ab [Davies Liu] split dataframe.py into multiple files","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/d7b69946cb21cd2781c9ad3e691e54b28efbbf3d"},{"sha":"deb411335a09b91eb1f75421d77e1c3686719621","author":{"email":"6f926b8fae3893ce5f7e882614cfe038839d4fc8@ussuning.com","name":"AiHe"},"message":"[SPARK-7473] [MLLIB] Add reservoir sample in RandomForest\n\nreservoir feature sample by using existing api\n\nAuthor: AiHe <ai.he@ussuning.com>\n\nCloses #5988 from AiHe/reservoir and squashes the following commits:\n\ne7a41ac [AiHe] remove non-robust testing case\n28ffb9a [AiHe] set seed as rng.nextLong\n37459e1 [AiHe] set fixed seed\n1e98a4c [AiHe] [MLLIB][tree] Add reservoir sample in RandomForest","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/deb411335a09b91eb1f75421d77e1c3686719621"},{"sha":"578bfeeff514228f6fd4b07a536815fbb3510f7e","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-7654][SQL] DataFrameReader and DataFrameWriter for input/output API\n\nThis patch introduces DataFrameWriter and DataFrameReader.\n\nDataFrameReader interface, accessible through SQLContext.read, contains methods that create DataFrames. These methods used to reside in SQLContext. Example usage:\n```scala\nsqlContext.read.json(\"...\")\nsqlContext.read.parquet(\"...\")\n```\n\nDataFrameWriter interface, accessible through DataFrame.write, implements a builder pattern to avoid the proliferation of options in writing DataFrame out. It currently implements:\n- mode\n- format (e.g. \"parquet\", \"json\")\n- options (generic options passed down into data sources)\n- partitionBy (partitioning columns)\nExample usage:\n```scala\ndf.write.mode(\"append\").format(\"json\").partitionBy(\"date\").saveAsTable(\"myJsonTable\")\n```\n\nTODO:\n\n- [ ] Documentation update\n- [ ] Move JDBC into reader / writer?\n- [ ] Deprecate the old interfaces\n- [ ] Move the generic load interface into reader.\n- [ ] Update example code and documentation\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #6175 from rxin/reader-writer and squashes the following commits:\n\nb146c95 [Reynold Xin] Deprecation of old APIs.\nbd8abdf [Reynold Xin] Fixed merge conflict.\n26abea2 [Reynold Xin] Added general load methods.\n244fbec [Reynold Xin] Added equivalent to example.\n4f15d92 [Reynold Xin] Added documentation for partitionBy.\n7e91611 [Reynold Xin] [SPARK-7654][SQL] DataFrameReader and DataFrameWriter for input/output API.","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/578bfeeff514228f6fd4b07a536815fbb3510f7e"},{"sha":"d41ae4344c07064de03a120804830886e1614d92","author":{"email":"0b23808f93757c320fe116d2f52ebb427397edcb@gmail.com","name":"FavioVazquez"},"message":"[SPARK-7671] Fix wrong URLs in MLlib Data Types Documentation\n\nThere is a mistake in the URL of Matrices in the MLlib Data Types documentation (Local matrix scala section), the URL points to https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.mllib.linalg.Matrices which is a mistake, since Matrices is an object that implements factory methods for Matrix that does not have a companion class. The correct link should point to https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.mllib.linalg.Matrices$\n\nThere is another mistake, in the Local Vector section in Scala, Java and Python\n\nIn the Scala section the URL of Vectors points to the trait Vector (https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.mllib.linalg.Vector) and not to the factory methods implemented in Vectors.\n\nThe correct link should be: https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.mllib.linalg.Vectors$\n\nIn the Java section the URL of Vectors points to the Interface Vector (https://spark.apache.org/docs/latest/api/java/org/apache/spark/mllib/linalg/Vector.html) and not to the Class Vectors\n\nThe correct link should be:\nhttps://spark.apache.org/docs/latest/api/java/org/apache/spark/mllib/linalg/Vectors.html\n\nIn the Python section the URL of Vectors points to the class Vector (https://spark.apache.org/docs/latest/api/python/pyspark.mllib.html#pyspark.mllib.linalg.Vector) and not the Class Vectors\n\nThe correct link should be:\nhttps://spark.apache.org/docs/latest/api/python/pyspark.mllib.html#pyspark.mllib.linalg.Vectors\n\nAuthor: FavioVazquez <favio.vazquezp@gmail.com>\n\nCloses #6196 from FavioVazquez/fix-typo-matrices-mllib-datatypes and squashes the following commits:\n\n3e9efd5 [FavioVazquez] - Fixed wrong URLs in the MLlib Data Types Documentation\n9af7074 [FavioVazquez] Merge remote-tracking branch 'upstream/master'\nedab1ef [FavioVazquez] Merge remote-tracking branch 'upstream/master'\nb2e2f8c [FavioVazquez] Merge remote-tracking branch 'upstream/master'","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/d41ae4344c07064de03a120804830886e1614d92"},{"sha":"1fd33815f47478f5f2e8b55b90757819b8cb5247","author":{"email":"291c18f3fb7528c712d9098b0e50a515ea0b91d5@cloudera.com","name":"Sean Owen"},"message":"[SPARK-4556] [BUILD] binary distribution assembly can't run in local mode\n\nAdd note on building a runnable distribution with make-distribution.sh\n\nAuthor: Sean Owen <sowen@cloudera.com>\n\nCloses #6186 from srowen/SPARK-4556 and squashes the following commits:\n\n4002966 [Sean Owen] Add pointer to --help flag\n9fa7883 [Sean Owen] Add note on building a runnable distribution with make-distribution.sh","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/1fd33815f47478f5f2e8b55b90757819b8cb5247"},{"sha":"0ac8b01a07840f199bbc79fb845762284aead6de","author":{"email":"125178f65a43413a83dc2cc3f6b1e1d4a7409a12@cloudera.com","name":"Nishkam Ravi"},"message":"[SPARK-7672] [CORE] Use int conversion in translating kryoserializer.buffer.mb to kryoserializer.buffer\n\nIn translating spark.kryoserializer.buffer.mb to spark.kryoserializer.buffer, use of toDouble will lead to \"Fractional values not supported\" error even when spark.kryoserializer.buffer.mb is an integer.\nilganeli, andrewor14\n\nAuthor: Nishkam Ravi <nravi@cloudera.com>\nAuthor: nishkamravi2 <nishkamravi@gmail.com>\nAuthor: nravi <nravi@c1704.halxg.cloudera.com>\n\nCloses #6198 from nishkamravi2/master_nravi and squashes the following commits:\n\n171a53c [nishkamravi2] Update SparkConfSuite.scala\n5261bf6 [Nishkam Ravi] Add a test for deprecated config spark.kryoserializer.buffer.mb\n5190f79 [Nishkam Ravi] In translating from deprecated spark.kryoserializer.buffer.mb to spark.kryoserializer.buffer use int conversion since fractions are not permissible\n059ce82 [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark into master_nravi\neaa13b5 [nishkamravi2] Update Client.scala\n981afd2 [Nishkam Ravi] Check for read permission before initiating copy\n1b81383 [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark into master_nravi\n0f1abd0 [nishkamravi2] Update Utils.scala\n474e3bf [nishkamravi2] Update DiskBlockManager.scala\n97c383e [nishkamravi2] Update Utils.scala\n8691e0c [Nishkam Ravi] Add a try/catch block around Utils.removeShutdownHook\n2be1e76 [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark into master_nravi\n1c13b79 [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark into master_nravi\nbad4349 [nishkamravi2] Update Main.java\n36a6f87 [Nishkam Ravi] Minor changes and bug fixes\nb7f4ae7 [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark into master_nravi\n4a45d6a [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark into master_nravi\n458af39 [Nishkam Ravi] Locate the jar using getLocation, obviates the need to pass assembly path as an argument\nd9658d6 [Nishkam Ravi] Changes for SPARK-6406\nccdc334 [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark into master_nravi\n3faa7a4 [Nishkam Ravi] Launcher library changes (SPARK-6406)\n345206a [Nishkam Ravi] spark-class merge Merge branch 'master_nravi' of https://github.com/nishkamravi2/spark into master_nravi\nac58975 [Nishkam Ravi] spark-class changes\n06bfeb0 [nishkamravi2] Update spark-class\n35af990 [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark into master_nravi\n32c3ab3 [nishkamravi2] Update AbstractCommandBuilder.java\n4bd4489 [nishkamravi2] Update AbstractCommandBuilder.java\n746f35b [Nishkam Ravi] \"hadoop\" string in the assembly name should not be mandatory (everywhere else in spark we mandate spark-assembly*hadoop*.jar)\nbfe96e0 [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark into master_nravi\nee902fa [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark into master_nravi\nd453197 [nishkamravi2] Update NewHadoopRDD.scala\n6f41a1d [nishkamravi2] Update NewHadoopRDD.scala\n0ce2c32 [nishkamravi2] Update HadoopRDD.scala\nf7e33c2 [Nishkam Ravi] Merge branch 'master_nravi' of https://github.com/nishkamravi2/spark into master_nravi\nba1eb8b [Nishkam Ravi] Try-catch block around the two occurrences of removeShutDownHook. Deletion of semi-redundant occurrences of expensive operation inShutDown.\n71d0e17 [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark into master_nravi\n494d8c0 [nishkamravi2] Update DiskBlockManager.scala\n3c5ddba [nishkamravi2] Update DiskBlockManager.scala\nf0d12de [Nishkam Ravi] Workaround for IllegalStateException caused by recent changes to BlockManager.stop\n79ea8b4 [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark into master_nravi\nb446edc [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark into master_nravi\n5c9a4cb [nishkamravi2] Update TaskSetManagerSuite.scala\n535295a [nishkamravi2] Update TaskSetManager.scala\n3e1b616 [Nishkam Ravi] Modify test for maxResultSize\n9f6583e [Nishkam Ravi] Changes to maxResultSize code (improve error message and add condition to check if maxResultSize > 0)\n5f8f9ed [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark into master_nravi\n636a9ff [nishkamravi2] Update YarnAllocator.scala\n8f76c8b [Nishkam Ravi] Doc change for yarn memory overhead\n35daa64 [Nishkam Ravi] Slight change in the doc for yarn memory overhead\n5ac2ec1 [Nishkam Ravi] Remove out\ndac1047 [Nishkam Ravi] Additional documentation for yarn memory overhead issue\n42c2c3d [Nishkam Ravi] Additional changes for yarn memory overhead issue\n362da5e [Nishkam Ravi] Additional changes for yarn memory overhead\nc726bd9 [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark into master_nravi\nf00fa31 [Nishkam Ravi] Improving logging for AM memoryOverhead\n1cf2d1e [nishkamravi2] Update YarnAllocator.scala\nebcde10 [Nishkam Ravi] Modify default YARN memory_overhead-- from an additive constant to a multiplier (redone to resolve merge conflicts)\n2e69f11 [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark into master_nravi\nefd688a [Nishkam Ravi] Merge branch 'master' of https://github.com/apache/spark\n2b630f9 [nravi] Accept memory input as \"30g\", \"512M\" instead of an int value, to be consistent with rest of Spark\n3bf8fad [nravi] Merge branch 'master' of https://github.com/apache/spark\n5423a03 [nravi] Merge branch 'master' of https://github.com/apache/spark\neb663ca [nravi] Merge branch 'master' of https://github.com/apache/spark\ndf2aeb1 [nravi] Improved fix for ConcurrentModificationIssue (Spark-1097, Hadoop-10456)\n6b840f0 [nravi] Undo the fix for SPARK-1758 (the problem is fixed)\n5108700 [nravi] Fix in Spark for the Concurrent thread modification issue (SPARK-1097, HADOOP-10456)\n681b36f [nravi] Fix for SPARK-1758: failing test org.apache.spark.JavaAPISuite.wholeTextFiles","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/0ac8b01a07840f199bbc79fb845762284aead6de"},{"sha":"47e7ffe36b8a8a246fe9af522aff480d19c0c8a6","author":{"email":"5d14e010a5b345b784fd604efcc6cebb79f97468@gmail.com","name":"zsxwing"},"message":"[SPARK-7655][Core][SQL] Remove 'scala.concurrent.ExecutionContext.Implicits.global' in 'ask' and 'BroadcastHashJoin'\n\nBecause both `AkkaRpcEndpointRef.ask` and `BroadcastHashJoin` uses `scala.concurrent.ExecutionContext.Implicits.global`. However, because the tasks in `BroadcastHashJoin` are usually long-running tasks, which will occupy all threads in `global`. Then `ask` cannot get a chance to process the replies.\n\nFor `ask`, actually the tasks are very simple, so we can use `MoreExecutors.sameThreadExecutor()`. For `BroadcastHashJoin`, it's better to use `ThreadUtils.newDaemonCachedThreadPool`.\n\nAuthor: zsxwing <zsxwing@gmail.com>\n\nCloses #6200 from zsxwing/SPARK-7655-2 and squashes the following commits:\n\ncfdc605 [zsxwing] Remove redundant imort and minor doc fix\ncf83153 [zsxwing] Add \"sameThread\" and \"newDaemonCachedThreadPool with maxThreadNumber\" to ThreadUtils\n08ad0ee [zsxwing] Remove 'scala.concurrent.ExecutionContext.Implicits.global' in 'ask' and 'BroadcastHashJoin'","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/47e7ffe36b8a8a246fe9af522aff480d19c0c8a6"},{"sha":"ce6391296a061bc352386080a2ee96bb63fcc4ac","author":{"email":"e6f14efef9fae0ae8f92d04007faea68035873ce@databricks.com","name":"Cheng Lian"},"message":"[HOTFIX] [SQL] Fixes DataFrameWriter.mode(String)\n\nWe forgot an assignment there.\n\n/cc rxin\n\nAuthor: Cheng Lian <lian@databricks.com>\n\nCloses #6212 from liancheng/fix-df-writer and squashes the following commits:\n\n711fbb0 [Cheng Lian] Adds a test case\n3b72d78 [Cheng Lian] Fixes DataFrameWriter.mode(String)","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/ce6391296a061bc352386080a2ee96bb63fcc4ac"},{"sha":"1b4e710e5cdb00febb4c5920d81e77c2e3966a8b","author":{"email":"0f1b6c21e0a79dff0c9e6fc6e738b2412190e920@us.ibm.com","name":"Matthew Brandyberry"},"message":"[BUILD] update jblas dependency version to 1.2.4\n\njblas 1.2.4 includes native library support for PPC64LE.\n\nAuthor: Matthew Brandyberry <mbrandy@us.ibm.com>\n\nCloses #6199 from mtbrandy/jblas-1.2.4 and squashes the following commits:\n\n9df9301 [Matthew Brandyberry] [BUILD] update jblas dependency version to 1.2.4","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/1b4e710e5cdb00febb4c5920d81e77c2e3966a8b"},{"sha":"161d0b4a41f453b21adde46a86e16c2743752799","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-7654][MLlib] Migrate MLlib to the DataFrame reader/writer API.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #6211 from rxin/mllib-reader and squashes the following commits:\n\n79a2cb9 [Reynold Xin] [SPARK-7654][MLlib] Migrate MLlib to the DataFrame reader/writer API.","distinct":true,"url":"https://api.github.com/repos/nishkamravi2/spark/commits/161d0b4a41f453b21adde46a86e16c2743752799"}]}