Oracle ROWID for Sqoop Split-By Column
<p>I have a huge oracle table (Transaction), the data in my oracle table has skew data on the column "Customer id" due to which the few mappers take time in hours to finish the job while other mappers finish the job in minutes. I couldn't see any other option to avoid the skewing data as this is the only column can be splited by. We can combine other columns like Customer ID, Batch ID, SEQ NUM to come with multi column split but I understood that sqoop doesn't support multi column in split by.  </p>

<p>My objective is to pull the transaction data for a specific period (i.e. batch date unique for a month of data). </p>

<p>I tried the below options in sqoop with 10 mappers.</p>

<pre><code>--split-by "my column name"           //for example customer id
--where "my query condition"         //for example batch date
</code></pre>

<p>Now I am thinking of using the ROWID which might split the rows evenly between the mappers. I thought of using the boundary query to get the MIN &amp; MAX ROW ID. Below is Sqoop command I want to use.</p>

<pre><code>sqoop import \
--table Transaction \
--split-by ROWID \
--where "BATCH_DT=TO_DATE('03/31/2016','MM/DD/YYYY')" \
--boundary-query "SELECT MIN(ROWID) AS MIN, MAX(ROWID) AS MAXL FROM Transaction WHERE BATCH_DT=TO_DATE('03/31/2016','MM/DD/YYYY') GROUP BY CUSTOMERID, BATCHNO,BATCHSEQNO " \
--num-mappers 10 \
--target-dir /user/trans
</code></pre>

<p>Need advise if this would be right option or is there any other way.</p>

<p>Also I would like to know if we can use multi split-by column name by any chance.</p>