Spark & Scala - Cannot Filter null Values from RDD
<p>i tried to filter null values from RDD but failed. Here's my code :</p>

<pre><code>val hBaseRDD = sc.newAPIHadoopRDD(conf, classOf[TableInputFormat],
      classOf[org.apache.hadoop.hbase.io.ImmutableBytesWritable],
      classOf[org.apache.hadoop.hbase.client.Result])

val raw_hbaserdd = hBaseRDD.map{
  kv =&gt; kv._2
}

val Ratings = raw_hbaseRDD.map {
      result =&gt;  val x = Bytes.toString(result.getValue(Bytes.toBytes("data"),Bytes.toBytes("user")))
                 val y = Bytes.toString(result.getValue(Bytes.toBytes("data"),Bytes.toBytes("item")))
                 val z = Bytes.toString(result.getValue(Bytes.toBytes("data"),Bytes.toBytes("rating")))

                 (x,y, z)
    }
Ratings.filter ( x =&gt; x._1 != null )

Ratings.foreach(println)
</code></pre>

<p>when Debugging,  null value still appeared after Filter :</p>

<pre><code>(3359,1494,4)
(null,null,null)
(28574,1542,5)
(null,null,null)
(12062,1219,5)
(14068,1459,3)
</code></pre>

<p>any Better idea ?</p>