why use spark core API (RDD) when you can do most of it in spark-sql
<p>I am learning spark for big data processing. People recommend using <code>HiveContext</code> over <code>SparkSqlContext</code>. and also recommend using <code>dataframes</code> instead of directly using an <code>rdd</code>.</p>

<p>Spark-sql seems is highly optmized for query planner, so it seems that using spark-sql is a better option than using Core api (RDD) via scala ( or python...) . Is there something I am missing  ?</p>