Split DateTimeIndex data based on hour/minute/second
<p>I have time-series data that I would like to split based on hour, or minute, or second. This is generally user-defined. I would like to know how it can be done. </p>

<p>For example, consider the following: </p>

<pre><code>test = pd.DataFrame({'TIME': pd.date_range(start='2016-09-30',
                                           freq='600s', periods=20)})
test['X'] = np.arange(20)
</code></pre>

<p>The output is: </p>

<pre><code>    TIME                X
0   2016-09-30 00:00:00 0
1   2016-09-30 00:10:00 1
2   2016-09-30 00:20:00 2
3   2016-09-30 00:30:00 3
4   2016-09-30 00:40:00 4
5   2016-09-30 00:50:00 5
6   2016-09-30 01:00:00 6
7   2016-09-30 01:10:00 7
8   2016-09-30 01:20:00 8
9   2016-09-30 01:30:00 9
10  2016-09-30 01:40:00 10
11  2016-09-30 01:50:00 11
12  2016-09-30 02:00:00 12
13  2016-09-30 02:10:00 13
14  2016-09-30 02:20:00 14
15  2016-09-30 02:30:00 15
16  2016-09-30 02:40:00 16
17  2016-09-30 02:50:00 17
18  2016-09-30 03:00:00 18
19  2016-09-30 03:10:00 19
</code></pre>

<p>Suppose I want to split it by hour. I would like the following as one chunk which I can then save to a file. </p>

<pre><code>    TIME                X
0   2016-09-30 00:00:00 0
1   2016-09-30 00:10:00 1
2   2016-09-30 00:20:00 2
3   2016-09-30 00:30:00 3
4   2016-09-30 00:40:00 4
5   2016-09-30 00:50:00 5
</code></pre>

<p>The second chunk would be: </p>

<pre><code>    TIME                X
0   2016-09-30 01:00:00 6
1   2016-09-30 01:10:00 7
2   2016-09-30 01:20:00 8
3   2016-09-30 01:30:00 9
4   2016-09-30 01:40:00 10
5   2016-09-30 01:50:00 11
</code></pre>

<p>and so on...</p>

<p>Note I can do it purely based on logical conditions such as, </p>

<pre><code> df[(df['TIME'] &gt;= '2016-09-30 00:00:00') &amp;
    (df['TIME'] &lt;= '2016-09-30 00:50:00')]
</code></pre>

<p>and repeat....</p>

<p>but what if my sampling changes? Is there a way to create a mask or something that takes less amount of code and is efficient? I have 10 GB of data. </p>