Azure Data Lake Analytics: Combine overlapping time duration using U-SQL
<p>I want to remove overlapping time duration from CSV data placed in Azure Data Lake Store using U-SQL and combine those rows. Data set contains start time and end time with several other attributes for each record. Here is an example:</p>

<pre><code>Start Time - End Time - Usar Name
5:00 AM - 6:00 AM - ABC
5:00 AM - 6:00 AM - XYZ
8:00 AM - 9:00 AM - ABC
8:00 AM - 10:00 AM - ABC
10:00 AM - 2:00 PM - ABC
7:00 AM - 11:00 AM - ABC
9:00 AM - 11:00 AM - ABC
11:00 AM - 11:30 AM - ABC
</code></pre>

<p>After removing overlap, output data set will look like:</p>

<pre><code>Start Time - End Time - Usar Name
5:00 AM - 6:00 AM - ABC
5:00 AM - 6:00 AM - XYZ
7:00 AM - 2:00 PM - ABC
</code></pre>

<p>Note that CSV contains enormous amount of data and consists of several GB of size. I am trying to solve this problem but no luck. I want to avoid U-SQL User defined operators for Azure Data Lake Analytics jobs and looking for some efficient solution from within U-SQL.</p>