Converting Dataframe to RDD reduces partitions
<p>In our code, Dataframe was created as :</p>

<pre><code>DataFrame DF = hiveContext.sql("select * from table_instance");
</code></pre>

<p>When I convert my dataframe to rdd and try to get its number of partitions as</p>

<pre><code>RDD&lt;Row&gt; newRDD = Df.rdd();
System.out.println(newRDD.getNumPartitions());
</code></pre>

<p>It reduces the number of partitions to 1(1 is printed in the console). Originally my dataframe has 102 partitions . </p>

<p><strong>UPDATE:</strong></p>

<p>While reading , I repartitoned the dataframe :</p>

<pre><code>DataFrame DF = hiveContext.sql("select * from table_instance").repartition(200);
</code></pre>

<p>and then converted to rdd , so it gave me 200 partitions only. 
Does </p>

<blockquote>
  <p>JavaSparkContext</p>
</blockquote>

<p>has a role to play in this? When we convert a dataframe to rdd , is default minimum partitions flag also considered at the spark context level?</p>

<p>UPDATE:</p>

<p>I made a seperate sample program in which I read the exact same table into dataframe and converted to rdd. No extra stage was created for RDD conversion and the partition count was also correct. I am now wondering what different am I doing in my main program. </p>

<p>Please let me know if my understanding is wrong here.</p>