How to reprocess a batched Kafka Stream
<ul>
<li>I want to batch messages based on the timestamp the message was
created. </li>
<li>Furthermore, I want to batch these messages in fixed time
windows (of 1 minute).  </li>
<li>Only after the passing of the window, the batch
should be pushed downstream.</li>
</ul>

<p>For this to work, the Processor API seems more or less fitting (a la  <a href="https://stackoverflow.com/questions/39104352/kstream-batch-process-windows/39123110#39123110">KStream batch process windows</a>):</p>

<pre><code>public void process(String key, SensorData sensorData) {
    //epochTime is rounded to our prefered time window (1 minute)
    long epochTime = Sensordata.epochTime;

    ArrayList&lt;SensorData&gt; data = messageStore.get(epochTime);
    if (data == null) {
        data = new ArrayList&lt;SensorData&gt;();
    }

    data.add(sensorData);
    messageStore.put(id, data);
    this.context.commit();
}

@Override
public void init(ProcessorContext context) {
    this.context = context;
    this.context.schedule(60000); // equal to 1 minute
}

@Override
public void punctuate(long streamTime) {
    KeyValueIterator&lt;String, ArrayList&lt;SensorData&gt;&gt; it = messageStore.all();
    while (it.hasNext()) {
        KeyValue&lt;String, ArrayList&lt;SensorData&gt;&gt; entry = it.next();
        this.context.forward(entry.key, entry.value);
    }
    //reset messageStore
}
</code></pre>

<p>However, this approach has one major downside with : we don't use Kafka Streams windows.</p>

<ul>
<li>out-of-order messages are not considered.</li>
<li>When operating in real-time, punctuation schedule should be equal to desired batch time window. If we set it to short, the batch will be forwarded and downstream computation will start to quickly. If set to long, and punctuation is triggered when a batch window is not finished yet, same problem.</li>
<li>also, replaying historic data whilst keeping the punctuation schedule (1 minute) will trigger the first computation only after 1 minute. If so, that will blow up the statestore and also feels wrong.</li>
</ul>

<p>Taking these points into consideration, I should use Kafka Streams windows. But this is only possible in the Kafka Streams DSL...</p>

<p>Any toughts on this would be awesome.</p>