{"push_id":675842658,"size":53,"distinct_size":1,"ref":"refs/heads/SPARKC-125","head":"c99151b54b02b101ec42ad088182780ae60bdebc","before":"5844e81f2caaf1de88bff65318f154fe7ddadf0a","commits":[{"sha":"ac09bff385f0a1a36d3d98f9f7d0143347c09a9f","author":{"email":"b5b613d15b30463c8392df72ca75b43fdf79ac5c@gmail.com","name":"Jacek Lewandowski"},"message":"SPARKC-130: Update versions of Scala to 2.10.5 / 2.11.6 and of Spark to 1.2.2","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/ac09bff385f0a1a36d3d98f9f7d0143347c09a9f"},{"sha":"2575706b47ba67d0c602cbb10facb79519937081","author":{"email":"b5b613d15b30463c8392df72ca75b43fdf79ac5c@gmail.com","name":"Jacek Lewandowski"},"message":"SPARKC-130: Turn on Java API for Scala 2.11","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/2575706b47ba67d0c602cbb10facb79519937081"},{"sha":"67badab016b1942b9518fa369cf902e7a96d3046","author":{"email":"b5b613d15b30463c8392df72ca75b43fdf79ac5c@gmail.com","name":"Jacek Lewandowski"},"message":"SPARKC-138: Added a way to run integration tests on vanilla Spark","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/67badab016b1942b9518fa369cf902e7a96d3046"},{"sha":"541560ca7718dc29d1715e8b3f447416c51fa377","author":{"email":"b5b613d15b30463c8392df72ca75b43fdf79ac5c@gmail.com","name":"Jacek Lewandowski"},"message":"SPARKC-138: Added few fixes","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/541560ca7718dc29d1715e8b3f447416c51fa377"},{"sha":"f78216a38d3b2ece59c37487e1fb468f35b203a4","author":{"email":"6addb84ccccbad488a813eb79fdeb9d39326b14e@datastax.com","name":"Jacek Lewandowski"},"message":"Update CHANGES.txt","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/f78216a38d3b2ece59c37487e1fb468f35b203a4"},{"sha":"8a0403e188e6ecff06f376ed09551aaabad180f0","author":{"email":"ad30e78512680ac1b6cd039d9137fcfff37a35f4@yahoo.com","name":"Alex Liu"},"message":"SPARKC-112 Implement basic data source API functions\n\nCreate CassandraSourceRelation class implementing BaseRelation,\nInsertableRelation and PrunedFilteredScan. This class creates\nsource relation which can read from and write to Cassandra\ntable. By default some filters and selected columns are\npushed down to CQL. It can be disable by set push_down to false\nin table options.\n\nCreate DefaultSource class implementing RelationProvider,\nSchemaRelationProvider and CreatableRelationProvider. Spark\nSQL creates source provider instances by reflection. It can\ncreate CassandraSourceRelation based on parameters map.\n\nUser can store table, keyspace, cluster, pushdown, read conf,\nwrite conf, Cassandra connector conf and table size settings.\nBecause Spark SQL can't parse option key String with dot, so\nall read conf, write conf, connector conf properties are\nconverted from dot to _.\n\nPushDown class is copied from CassandraTableScans and it's\nrefactored into a new class. Because CassandraTableScans will\nbe removed from the code in the later ticket, so I don't\nrefactor it.\n\nRddTransformer class is copied from CassandraTableScan.\nBecause CassandraTableScans will be removed in the later ticket,\nso I don't refactor it.\n\nDataTypeConverter is copied from CassandraRelation. Because\nCassandraRelation class will be removed in later ticket, so\nI don't refactore it.\n\nBasic example to create a temp table\n\n CREATE TEMPORARY TABLE tmpTable\n USING org.apache.spark.sql.cassandra\n OPTIONS (\n   c_table \"table\",\n   keyspace \"keyspace\",\n   cluster \"test_cluster\",\n   push_down \"true\",\n   spark_cassandra_input_page_row_size \"10\",\n   spark_cassandra_output_consistency_level \"ONE\",\n   spark_cassandra_connection_timeout_ms \"1000\" )","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/8a0403e188e6ecff06f376ed09551aaabad180f0"},{"sha":"c99c138b3683e1b55c5238300cc24ef4c1ad56d3","author":{"email":"ad30e78512680ac1b6cd039d9137fcfff37a35f4@yahoo.com","name":"Alex Liu"},"message":"SPARKC-112 Clean up tests\n\nRemove some duplicate tests and code clean up","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/c99c138b3683e1b55c5238300cc24ef4c1ad56d3"},{"sha":"926125e5c8a5f0ba53708ee45024b017130b0446","author":{"email":"ad30e78512680ac1b6cd039d9137fcfff37a35f4@yahoo.com","name":"Alex Liu"},"message":"SPARKC-112 Move Quote method to a object and refactor code base\n\nSince there are many places using quote method, create a object\nQuote and remove quote method from many classes.","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/926125e5c8a5f0ba53708ee45024b017130b0446"},{"sha":"48a96c2db6a06d44b02150bc773c33d4f3965113","author":{"email":"ad30e78512680ac1b6cd039d9137fcfff37a35f4@yahoo.com","name":"Alex Liu"},"message":"SPARKC-112 Move  RddTransfer class to parent class member methods\n\nIt makes code cleaner, so remove RddTransfer class, and move it\nmethods to parent class member methods","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/48a96c2db6a06d44b02150bc773c33d4f3965113"},{"sha":"5a69eb328ffa6acc0023382dc3d48c7ded13d967","author":{"email":"ad30e78512680ac1b6cd039d9137fcfff37a35f4@yahoo.com","name":"Alex Liu"},"message":"SPARKC-112 Add more CreateRelation documentation, rename mehtod\n\nAdd detail usage for CreateRelation, rename\nreadWriteConnectionConfAndTableSize  to buildConfs.\nAdd warning logging message for data mapping from VarInt\nto Catalyst decimal.","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/5a69eb328ffa6acc0023382dc3d48c7ded13d967"},{"sha":"ef2cf0d44191af6bb0559d0ea3c5ee185eeed455","author":{"email":"ad30e78512680ac1b6cd039d9137fcfff37a35f4@yahoo.com","name":"Alex Liu"},"message":"SPARKC-112 Refactor PushDown class\n\nRename some methods. Use collect with partial function than\nfilter.","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/ef2cf0d44191af6bb0559d0ea3c5ee185eeed455"},{"sha":"e1668237935b581f5cdad8f2d0c58dc844c786d9","author":{"email":"ad30e78512680ac1b6cd039d9137fcfff37a35f4@yahoo.com","name":"Alex Liu"},"message":"SPARKC-112 Change PushDown to FilterPushdown object\n\nOnly use it to get pushdown filters, so an object\nis better than a class","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/e1668237935b581f5cdad8f2d0c58dc844c786d9"},{"sha":"31944469aae7600a997bff10e9d83e3c7e3d6055","author":{"email":"ad30e78512680ac1b6cd039d9137fcfff37a35f4@yahoo.com","name":"Alex Liu"},"message":"SPARKC-112 Use partial function in FilterPushdown class\n\nRefactor code to use partial functions instead of filter","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/31944469aae7600a997bff10e9d83e3c7e3d6055"},{"sha":"a171a706e7aa5c0321c7e580ca880bd010cca1a2","author":{"email":"ad30e78512680ac1b6cd039d9137fcfff37a35f4@yahoo.com","name":"Alex Liu"},"message":"SPARKC-112 Small code formatt change in FilterPushdown class","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/a171a706e7aa5c0321c7e580ca880bd010cca1a2"},{"sha":"4b8f2deb7400a01229b22eb11c96d16be0a41bfb","author":{"email":"ad30e78512680ac1b6cd039d9137fcfff37a35f4@yahoo.com","name":"Alex Liu"},"message":"SPARKC-112 Dynamically retrieve table size from C* system table\n\nTable size may change, so retrieve it when Catalyst uses it.","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/4b8f2deb7400a01229b22eb11c96d16be0a41bfb"},{"sha":"b8fbac79ea38381ddc744b1bb0e29d5c32de21b8","author":{"email":"b4f99bbaf9153e145146d8bed3992790fe0edce1@gmail.com","name":"Russell Spitzer"},"message":"SPARKC-155: Repart uses only DC Local Hosts\n\nThis fixes a small bug where the repartition code could\nselect a replica Endpoint for a given C* key that was not resident in\nthe same datacenter as the one the connector had chosen. Assuming that\nthere were no valid endpoints in the datacenter the code would simply\nchoose a random endpoint in the datacenter as teh preferredLocation.\nThe new code only chooses from replica endpoints  which are in the same\ndatacenter as the CassandraConnector connected to.","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/b8fbac79ea38381ddc744b1bb0e29d5c32de21b8"},{"sha":"5b5e5174bfb336e097a6121236289634b7c2e032","author":{"email":"43b54a04fac7e323e7c38332b5cdf6c96b4d442d@datastax.com","name":"Piotr Kolaczkowski"},"message":"SPARKC-155: Update set of hosts in _config in openSession\n\nA full list of hosts should be saved in _config when opening\na new session, for the future use, in case some of the initial\nhosts go down.","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/5b5e5174bfb336e097a6121236289634b7c2e032"},{"sha":"0f5215d903324ac55b531cf171dd97b4a00ecf59","author":{"email":"ad30e78512680ac1b6cd039d9137fcfff37a35f4@yahoo.com","name":"Alex Liu"},"message":"SPARKC-112 Replace buildConfs to methods to build each conf\n\nbuildConfs build a few conf objects, split it to\nseveral methods","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/0f5215d903324ac55b531cf171dd97b4a00ecf59"},{"sha":"2c49093b5dd5d27d970c2294a7a8d25c7255859f","author":{"email":"ad30e78512680ac1b6cd039d9137fcfff37a35f4@yahoo.com","name":"Alex Liu"},"message":"SPARKC-112 Replace predicate by filter in FilterPushdown\n\nSimply replace any predicate by filter in FilterPushdown\nobject. We may need check whether there is any place to\nbe improved for filters later.","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/2c49093b5dd5d27d970c2294a7a8d25c7255859f"},{"sha":"3525fde1231ee68d66af1d14c19f5b977408001c","author":{"email":"ad30e78512680ac1b6cd039d9137fcfff37a35f4@yahoo.com","name":"Alex Liu"},"message":"SPARKC-112 Pass Map to CassandraSourceRelation to build Cassandra Conf\nBuild Cassandra Conf based on sparkConf and conf map from\ntable options.\n\nClone sparkConf, and override any Cassandra conf settings\nby table option Cassandra conf settings. Then build\nReadConf, WriteConf, CassandraConnectorConf and\ntableSizeInBytes.","distinct":false,"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/commits/3525fde1231ee68d66af1d14c19f5b977408001c"}]}