SparkSQL: Am I doing in right?
<p>Here is how I use Spark-SQL in a little application I am working with.
I have two Hbase tables say t1,t2.</p>

<ul>
<li><p>My input being a csv file, I parse each and every line and query(SparkSQL) the table t1. I write the output to another file.</p></li>
<li><p>Now I parse the second file and query the second table and I apply certain functions over the result and I output the data.
the table t1 hast the purchase details and t2 has the list of items that were added to cart along with the time frame by each user.</p></li>
</ul>

<p><strong>Input</strong> -> CustomerID(list of it in a csv file)<br>
<strong>Output</strong> - > A csv file in a particular format mentioned below.  </p>

<p>CustomerID, Details of the item he brought,First item he added to cart,All the items he added to cart until purchase. </p>

<p>For a input of 1100 records, It takes two hours to complete the whole process!</p>

<p>I was wondering if I could speed up the process but I am struck.
Any help?</p>