ElasticSearch (search_context_missing_exception) with Search::ElasticSearch::Scroll
<p>I'm using Search::Elasticsearch and Search::Elasticsearch::Scroll for search and scroll into my elasticsearch server.</p>

<p>In scrolling process, for some querys, I'm seeing the next errors while I'm scrolling the search results:</p>

<pre><code>2016/03/22 11:03:38 - 265885 FATAL: [Daemon.pm][8221]: Something gone wrong, error $VAR1 = bless( {
 'msg' =&gt; '[Missing] ** [http://localhost:9200]-[404] Not Found, called from sub Search::Elasticsearch::Scroll::next at searcher.pl line 92. With vars: {\'body\' =&gt; {\'hits\' =&gt; {\'hits\' =&gt; [],\'max_score\' =&gt; \'0\',\'total\' =&gt; 5215},\'timed_out\' =&gt; bless( do{\\(my $o = 0)}, \'JSON::XS::Boolean\' ),\'_shards\' =&gt; {\'failures\' =&gt; [{\'index\' =&gt; undef,\'reason\' =&gt; {\'reason\' =&gt; \'No search context found for id [4920053]\',\'type\' =&gt; \'search_context_missing_exception\'},\'shard\' =&gt; -1},{\'index\' =&gt; undef,\'reason\' =&gt; {\'reason\' =&gt; \'No search context found for id [5051485]\',\'type\' =&gt; \'search_context_missing_exception\'},\'shard\' =&gt; -1},{\'index\' =&gt; undef,\'reason\' =&gt; {\'reason\' =&gt; \'No search context found for id [4920059]\',\'type\' =&gt; \'search_context_missing_exception\'},\'shard\' =&gt; -1},{\'index\' =&gt; undef,\'reason\' =&gt; {\'reason\' =&gt; \'No search context found for id [5051496]\',\'type\' =&gt; \'search_context_missing_exception\'},\'shard\' =&gt; -1},{\'index\' =&gt; undef,\'reason\' =&gt; {\'reason\' =&gt; \'No search context found for id [5051500]\',\'type\' =&gt; \'search_context_missing_exception\'},\'shard\' =&gt; -1}],\'failed\' =&gt; 5,\'successful\' =&gt; 0,\'total\' =&gt; 5},\'_scroll_id\' =&gt; \'c2NhbjswOzE7dG90YWxfaGl0czo1MjE1Ow==\',\'took\' =&gt; 2},\'request\' =&gt; {\'serialize\' =&gt; \'std\',\'path\' =&gt; \'/_search/scroll\',\'ignore\' =&gt; [],\'mime_type\' =&gt; \'application/json\',\'body\' =&gt; \'c2Nhbjs1OzQ5MjAwNTM6bHExbENzRDVReEc0OV9UMUgzd3Vkdzs1MDUxNDg1OnJrQ3lsUkRKVHRxRWRWeURoOTB4WVE7NDkyMDA1OTpscTFsQ3NENVF4RzQ5X1QxSDN3dWR3OzUwNTE0OTY6cmtDeWxSREpUdHFFZFZ5RGg5MHhZUTs1MDUxNTAwOnJrQ3lsUkRKVHRxRWRWeURoOTB4WVE7MTt0b3RhbF9oaXRzOjUyMTU7\',\'qs\' =&gt; {\'scroll\' =&gt; \'1m\'},\'method\' =&gt; \'GET\'},\'status_code\' =&gt; 404}
',
 'stack' =&gt; [
              [
                'searcher.pl',
                92,
                'Search::Elasticsearch::Scroll::next'
              ]
            ],
 'text' =&gt; '[http://localhost:9200]-[404] Not Found',
 'vars' =&gt; {
             'body' =&gt; {
                         'hits' =&gt; {
                                     'hits' =&gt; [],
                                     'max_score' =&gt; '0',
                                     'total' =&gt; 5215
                                   },
                         'timed_out' =&gt; bless( do{\(my $o = 0)}, 'JSON::XS::Boolean' ),
                         '_shards' =&gt; {
                                        'failures' =&gt; [
                                                        {
                                                          'index' =&gt; undef,
                                                          'reason' =&gt; {
                                                                        'reason' =&gt; 'No search context found for id [4920053]',
                                                                        'type' =&gt; 'search_context_missing_exception'
                                                                      },
                                                          'shard' =&gt; -1
                                                        },
                                                        {
                                                          'index' =&gt; undef,
                                                          'reason' =&gt; {
                                                                        'reason' =&gt; 'No search context found for id [5051485]',
                                                                        'type' =&gt; 'search_context_missing_exception'
                                                                      },
                                                          'shard' =&gt; -1
                                                        },
                                                        {
                                                          'index' =&gt; undef,
                                                          'reason' =&gt; {
                                                                        'reason' =&gt; 'No search context found for id [4920059]',
                                                                        'type' =&gt; 'search_context_missing_exception'
                                                                      },
                                                          'shard' =&gt; -1
                                                        },
                                                        {
                                                          'index' =&gt; undef,
                                                          'reason' =&gt; {
                                                                        'reason' =&gt; 'No search context found for id [5051496]',
                                                                        'type' =&gt; 'search_context_missing_exception'
                                                                      },
                                                          'shard' =&gt; -1

                                                                                                        },
                                                        {
                                                          'index' =&gt; undef,
                                                          'reason' =&gt; {
                                                                        'reason' =&gt; 'No search context found for id [5051500]',
                                                                        'type' =&gt; 'search_context_missing_exception'
                                                                      },
                                                          'shard' =&gt; -1
                                                        }
                                                      ],
                                        'failed' =&gt; 5,
                                        'successful' =&gt; 0,
                                        'total' =&gt; 5
                                      },
                         '_scroll_id' =&gt; 'c2NhbjswOzE7dG90YWxfaGl0czo1MjE1Ow==',
                         'took' =&gt; 2
                       },
             'request' =&gt; {
                            'serialize' =&gt; 'std',
                            'path' =&gt; '/_search/scroll',
                            'ignore' =&gt; [],
                            'mime_type' =&gt; 'application/json',
                            'body' =&gt; 'c2Nhbjs1OzQ5MjAwNTM6bHExbENzRDVReEc0OV9UMUgzd3Vkdzs1MDUxNDg1OnJrQ3lsUkRKVHRxRWRWeURoOTB4WVE7NDkyMDA1OTpscTFsQ3NENVF4RzQ5X1QxSDN3dWR3OzUwNTE0OTY6cmtDeWxSREpUdHFFZFZ5RGg5MHhZUTs1MDUxNTAwOnJrQ3lsUkRKVHRxRWRWeURoOTB4WVE7MTt0b3RhbF9oaXRzOjUyMTU7',
                            'qs' =&gt; {
                                      'scroll' =&gt; '1m'
                                    },
                            'method' =&gt; 'GET'
                          },
             'status_code' =&gt; 404
           },
 'type' =&gt; 'Missing'
}, 'Search::Elasticsearch::Error::Missing' );
</code></pre>

<p>The code I'm using is the next one (simplified) :</p>

<pre><code># Retrieve scroll 
my $scroll = $self-&gt;getScrollBySignature($item);

# Retrieve all affected documents ids 
while (my @docs = $scroll-&gt;next(500)) {
    # Do stuff with @docs
}
</code></pre>

<p>The function getScrollBySignature have the next code in order to call to elasticSearch</p>

<pre><code>my $scroll = $self-&gt;{ELASTIC}-&gt;scroll_helper(
    index              =&gt; $self-&gt;{INDEXES},
    search_type        =&gt; 'scan',
    ignore_unavailable =&gt; 1,
    body               =&gt; {
        size  =&gt; $self-&gt;{PAGINATION},
        query =&gt; {
            filtered =&gt; {
                filter =&gt; {
                    bool =&gt; {
                        must =&gt; [{term =&gt; {signature_id =&gt; $item-&gt;{profileId}}}, {terms =&gt; {channel_type_id =&gt; $type}}]
                    }
                }
            }
        }
    }
);
</code></pre>

<p>As you can see, I'm doing the scroll without passing scroll parameter then as documentation says, the time that scroll is alive is 1 min.</p>

<p>The elasticSearch is a cluster of 3 servers, and the query that ends with that error retrieves a bit more than 5000 docs.</p>

<p>My first solution was to update the life time for scroll to 5 minutes and the error didn't appear. </p>

<p>The question is, as I understand every time I'm calling $scroll->next() the life time off scroll affected is upgraded 1m more, then how is possible to receive those context related errors? </p>

<p>I'm doing something in a bad manner?</p>

<p>Thank you all.</p>