{"action":"created","issue":{"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/issues/527","labels_url":"https://api.github.com/repos/datastax/spark-cassandra-connector/issues/527/labels{/name}","comments_url":"https://api.github.com/repos/datastax/spark-cassandra-connector/issues/527/comments","events_url":"https://api.github.com/repos/datastax/spark-cassandra-connector/issues/527/events","html_url":"https://github.com/datastax/spark-cassandra-connector/pull/527","id":55750937,"number":527,"title":"SPARKC-25: Initial work to allow RDD's to be used as Keys for C*","user":{"login":"jacek-lewandowski","id":6516951,"avatar_url":"https://avatars.githubusercontent.com/u/6516951?v=3","gravatar_id":"","url":"https://api.github.com/users/jacek-lewandowski","html_url":"https://github.com/jacek-lewandowski","followers_url":"https://api.github.com/users/jacek-lewandowski/followers","following_url":"https://api.github.com/users/jacek-lewandowski/following{/other_user}","gists_url":"https://api.github.com/users/jacek-lewandowski/gists{/gist_id}","starred_url":"https://api.github.com/users/jacek-lewandowski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jacek-lewandowski/subscriptions","organizations_url":"https://api.github.com/users/jacek-lewandowski/orgs","repos_url":"https://api.github.com/users/jacek-lewandowski/repos","events_url":"https://api.github.com/users/jacek-lewandowski/events{/privacy}","received_events_url":"https://api.github.com/users/jacek-lewandowski/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-01-28T12:35:17Z","updated_at":"2015-03-09T13:04:24Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/pulls/527","html_url":"https://github.com/datastax/spark-cassandra-connector/pull/527","diff_url":"https://github.com/datastax/spark-cassandra-connector/pull/527.diff","patch_url":"https://github.com/datastax/spark-cassandra-connector/pull/527.patch"},"body":"Added function keyByReplica\r\n    For keying RDD's based on their cassandra replica placement\r\nAdded function partitionByReplica\r\n    Which keys an RDD based on replica placement then creates\r\n    new partitions based on their replicas"},"comment":{"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/issues/comments/77849068","html_url":"https://github.com/datastax/spark-cassandra-connector/pull/527#issuecomment-77849068","issue_url":"https://api.github.com/repos/datastax/spark-cassandra-connector/issues/527","id":77849068,"user":{"login":"pkolaczk","id":1352795,"avatar_url":"https://avatars.githubusercontent.com/u/1352795?v=3","gravatar_id":"","url":"https://api.github.com/users/pkolaczk","html_url":"https://github.com/pkolaczk","followers_url":"https://api.github.com/users/pkolaczk/followers","following_url":"https://api.github.com/users/pkolaczk/following{/other_user}","gists_url":"https://api.github.com/users/pkolaczk/gists{/gist_id}","starred_url":"https://api.github.com/users/pkolaczk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pkolaczk/subscriptions","organizations_url":"https://api.github.com/users/pkolaczk/orgs","repos_url":"https://api.github.com/users/pkolaczk/repos","events_url":"https://api.github.com/users/pkolaczk/events{/privacy}","received_events_url":"https://api.github.com/users/pkolaczk/received_events","type":"User","site_admin":false},"created_at":"2015-03-09T13:04:24Z","updated_at":"2015-03-09T13:04:24Z","body":"Two tests are failing:\r\n```\r\n - should be lazy and must not throw IOException if the table was not found at the RDD initialization time *** FAILED *** (2 milliseconds)\r\n[info]   java.io.IOException: Table not found: read_test.unknown_table\r\n[info]   at com.datastax.spark.connector.rdd.CassandraTableRowReader$class.tableDef(CassandraTableRowReader.scala:48)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraTableScanRDD.tableDef$lzycompute(CassandraTableScanRDD.scala:47)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraTableScanRDD.tableDef(CassandraTableScanRDD.scala:47)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraTableRowReader$class.rowReader(CassandraTableRowReader.scala:43)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraTableScanRDD.rowReader$lzycompute(CassandraTableScanRDD.scala:47)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraTableScanRDD.rowReader(CassandraTableScanRDD.scala:47)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraTableRowReader$class.$init$(CassandraTableRowReader.scala:105)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraTableScanRDD.<init>(CassandraTableScanRDD.scala:47)\r\n[info]   at com.datastax.spark.connector.SparkContextFunctions.cassandraTable(SparkContextFunctions.scala:50)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraRDDSpec$$anonfun$37.apply$mcV$sp(CassandraRDDSpec.scala:367)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraRDDSpec$$anonfun$37.apply(CassandraRDDSpec.scala:367)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraRDDSpec$$anonfun$37.apply(CassandraRDDSpec.scala:367)\r\n[info]   at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)\r\n[info]   at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)\r\n[info]   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)\r\n[info]   at org.scalatest.Transformer.apply(Transformer.scala:22)\r\n[info]   at org.scalatest.Transformer.apply(Transformer.scala:20)\r\n[info]   at org.scalatest.FlatSpecLike$$anon$1.apply(FlatSpecLike.scala:1647)\r\n[info]   at org.scalatest.Suite$class.withFixture(Suite.scala:1122)\r\n[info]   at org.scalatest.FlatSpec.withFixture(FlatSpec.scala:1683)\r\n[info]   at org.scalatest.FlatSpecLike$class.invokeWithFixture$1(FlatSpecLike.scala:1644)\r\n[info]   at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)\r\n[info]   at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)\r\n[info]   at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)\r\n[info]   at org.scalatest.FlatSpecLike$class.runTest(FlatSpecLike.scala:1656)\r\n[info]   at org.scalatest.FlatSpec.runTest(FlatSpec.scala:1683)\r\n[info]   at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)\r\n[info]   at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)\r\n[info]   at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413)\r\n[info]   at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)\r\n[info]   at scala.collection.immutable.List.foreach(List.scala:318)\r\n[info]   at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)\r\n[info]   at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:390)\r\n[info]   at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:427)\r\n[info]   at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)\r\n[info]   at scala.collection.immutable.List.foreach(List.scala:318)\r\n[info]   at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)\r\n[info]   at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396)\r\n[info]   at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483)\r\n[info]   at org.scalatest.FlatSpecLike$class.runTests(FlatSpecLike.scala:1714)\r\n[info]   at org.scalatest.FlatSpec.runTests(FlatSpec.scala:1683)\r\n[info]   at org.scalatest.Suite$class.run(Suite.scala:1424)\r\n[info]   at org.scalatest.FlatSpec.org$scalatest$FlatSpecLike$$super$run(FlatSpec.scala:1683)\r\n[info]   at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)\r\n[info]   at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)\r\n[info]   at org.scalatest.SuperEngine.runImpl(Engine.scala:545)\r\n[info]   at org.scalatest.FlatSpecLike$class.run(FlatSpecLike.scala:1760)\r\n[info]   at org.scalatest.FlatSpec.run(FlatSpec.scala:1683)\r\n[info]   at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:466)\r\n[info]   at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:677)\r\n[info]   at sbt.ForkMain$Run$2.call(ForkMain.java:294)\r\n[info]   at sbt.ForkMain$Run$2.call(ForkMain.java:284)\r\n[info]   at java.util.concurrent.FutureTask.run(FutureTask.java:262)\r\n[info]   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\r\n[info]   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\r\n[info]   at java.lang.Thread.run(Thread.java:745)\r\n - should allow to use empty RDD on undefined table *** FAILED *** (1 millisecond)\r\n[info]   java.io.IOException: Table not found: unknown_ks.unknown_table\r\n[info]   at com.datastax.spark.connector.rdd.CassandraTableRowReader$class.tableDef(CassandraTableRowReader.scala:48)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraTableScanRDD.tableDef$lzycompute(CassandraTableScanRDD.scala:47)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraTableScanRDD.tableDef(CassandraTableScanRDD.scala:47)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraTableRowReader$class.rowReader(CassandraTableRowReader.scala:43)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraTableScanRDD.rowReader$lzycompute(CassandraTableScanRDD.scala:47)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraTableScanRDD.rowReader(CassandraTableScanRDD.scala:47)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraTableRowReader$class.$init$(CassandraTableRowReader.scala:105)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraTableScanRDD.<init>(CassandraTableScanRDD.scala:47)\r\n[info]   at com.datastax.spark.connector.SparkContextFunctions.cassandraTable(SparkContextFunctions.scala:50)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraRDDSpec$$anonfun$63.apply$mcV$sp(CassandraRDDSpec.scala:640)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraRDDSpec$$anonfun$63.apply(CassandraRDDSpec.scala:639)\r\n[info]   at com.datastax.spark.connector.rdd.CassandraRDDSpec$$anonfun$63.apply(CassandraRDDSpec.scala:639)\r\n[info]   at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)\r\n[info]   at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)\r\n[info]   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)\r\n[info]   at org.scalatest.Transformer.apply(Transformer.scala:22)\r\n[info]   at org.scalatest.Transformer.apply(Transformer.scala:20)\r\n[info]   at org.scalatest.FlatSpecLike$$anon$1.apply(FlatSpecLike.scala:1647)\r\n[info]   at org.scalatest.Suite$class.withFixture(Suite.scala:1122)\r\n[info]   at org.scalatest.FlatSpec.withFixture(FlatSpec.scala:1683)\r\n[info]   at org.scalatest.FlatSpecLike$class.invokeWithFixture$1(FlatSpecLike.scala:1644)\r\n[info]   at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)\r\n[info]   at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)\r\n[info]   at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)\r\n[info]   at org.scalatest.FlatSpecLike$class.runTest(FlatSpecLike.scala:1656)\r\n[info]   at org.scalatest.FlatSpec.runTest(FlatSpec.scala:1683)\r\n[info]   at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)\r\n[info]   at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)\r\n[info]   at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413)\r\n[info]   at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)\r\n[info]   at scala.collection.immutable.List.foreach(List.scala:318)\r\n[info]   at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)\r\n[info]   at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:390)\r\n[info]   at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:427)\r\n[info]   at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)\r\n[info]   at scala.collection.immutable.List.foreach(List.scala:318)\r\n[info]   at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)\r\n[info]   at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396)\r\n[info]   at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483)\r\n[info]   at org.scalatest.FlatSpecLike$class.runTests(FlatSpecLike.scala:1714)\r\n[info]   at org.scalatest.FlatSpec.runTests(FlatSpec.scala:1683)\r\n[info]   at org.scalatest.Suite$class.run(Suite.scala:1424)\r\n[info]   at org.scalatest.FlatSpec.org$scalatest$FlatSpecLike$$super$run(FlatSpec.scala:1683)\r\n[info]   at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)\r\n[info]   at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)\r\n[info]   at org.scalatest.SuperEngine.runImpl(Engine.scala:545)\r\n[info]   at org.scalatest.FlatSpecLike$class.run(FlatSpecLike.scala:1760)\r\n[info]   at org.scalatest.FlatSpec.run(FlatSpec.scala:1683)\r\n[info]   at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:466)\r\n[info]   at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:677)\r\n[info]   at sbt.ForkMain$Run$2.call(ForkMain.java:294)\r\n[info]   at sbt.ForkMain$Run$2.call(ForkMain.java:284)\r\n[info]   at java.util.concurrent.FutureTask.run(FutureTask.java:262)\r\n[info]   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\r\n[info]   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\r\n[info]   at java.lang.Thread.run(Thread.java:745)\r\n```"}}