Filter/aggregate one elasticsearch index of time series data by timestamps found in another index
<h2>The Data</h2>

<p>So I have reams of different types of time series data.  Currently i've chosen to put each type of data into their own index because with the exception of 4 fields, all of the data is very different.  Also the data is sampled at different rates and are not guaranteed to have common timestamps across the same sub-second window so fusing them all into one large document is also not a trivial task.</p>

<h2>The Goal</h2>

<p>One of our common use cases that i'm trying to see if I can solve entirely in Elasticsearch is to return an aggregation result of one index based on the time windows returned from a query of another index.  Pictorially:</p>

<p><a href="https://i.stack.imgur.com/vBoc9.png" rel="nofollow"><img src="https://i.stack.imgur.com/vBoc9.png" alt="What i&#39;d like to accomplish"></a></p>

<p>This is what I want to accomplish.  </p>

<h2>Some Considerations</h2>

<p>For small enough signal transitions on the "condition" data, I can just use a date histogram and some combination of a top hits sub aggregation, but this quickly breaks down when I have 10,000's or 100,000's of occurrences of "the condition".  Further this is just one "case", I have 100's of sets of similar situations that i'd like to get the overall min/max from.</p>

<p>The comparisons are basically amongst what I would consider to be <em>sibling</em> level documents or indices, so there doesn't seem to be any obvious parent->child relationship that would be flexible enough over the long run, at least with how the data is currently structured.</p>

<p>It feels like there should be an elegant solution instead of brute force building the date ranges outside of Elasticsearch with the results of one query and feeding 100's of time ranges into another query.</p>

<p>Looking through the documentation it feels like some combination of Elasticsearch scripting and some of the pipelined aggregations are going to be what i want, but no definitive solutions are jumping out at me.  I could really use some pointers in the right direction from the community.</p>

<p>Thanks.</p>