{"action":"closed","issue":{"url":"https://api.github.com/repos/elastic/kibana/issues/3994","labels_url":"https://api.github.com/repos/elastic/kibana/issues/3994/labels{/name}","comments_url":"https://api.github.com/repos/elastic/kibana/issues/3994/comments","events_url":"https://api.github.com/repos/elastic/kibana/issues/3994/events","html_url":"https://github.com/elastic/kibana/issues/3994","id":81293580,"number":3994,"title":"Kibana 4 - Courier Fetch: X of 5 shards failed","user":{"login":"Ginja","id":1631363,"avatar_url":"https://avatars.githubusercontent.com/u/1631363?v=3","gravatar_id":"","url":"https://api.github.com/users/Ginja","html_url":"https://github.com/Ginja","followers_url":"https://api.github.com/users/Ginja/followers","following_url":"https://api.github.com/users/Ginja/following{/other_user}","gists_url":"https://api.github.com/users/Ginja/gists{/gist_id}","starred_url":"https://api.github.com/users/Ginja/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Ginja/subscriptions","organizations_url":"https://api.github.com/users/Ginja/orgs","repos_url":"https://api.github.com/users/Ginja/repos","events_url":"https://api.github.com/users/Ginja/events{/privacy}","received_events_url":"https://api.github.com/users/Ginja/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-05-27T05:55:37Z","updated_at":"2015-05-27T16:40:41Z","closed_at":"2015-05-27T16:40:41Z","body":"Hi,\r\n\r\nI'm having issues viewing Netflow data using Kibana 4.0.2 (Build 6004). The error message I get when I click the Discover tab is a yellow banner that reads:\r\n\r\n```\r\nCourier Fetch: 5 of 5 shards failed.\r\n```\r\n\r\nI am however able to view this data using Kibana 3, which leads me to believe this is a bug (or a change) in how Kibana 4 queries Elasticsearch.\r\n\r\nI should also mention that I can view this data in Kibana 4 if I let Elasticsearch create a mapping dynamically for this index, but I would like to use a custom one so that it's optimized. I've included the mapping I'm trying to use below, and I don't see anything wrong with it. Netflow data includes bigger numbers than `\"type\": \"long\"` can handle, so I needed to use `\"type\": \"string\"` for some fields.\r\n\r\nRequest \r\n```bash\r\ncurl -XPOST http://fqdn.omitted.com:5601/elasticsearch/_msearch?timeout=0&ignore_unavailable=true&preference=1432705287095\r\n```\r\n\r\nRequest Payload\r\n```json\r\n{\r\n  \"index\": \"customindex-*\",\r\n  \"ignore_unavailable\": true\r\n}\\n\r\n{\r\n  \"size\": 500,\r\n  \"sort\": {\r\n    \"@timestamp\": \"desc\"\r\n  },\r\n  \"highlight\": {\r\n    \"pre_tags\": [\r\n      \"@kibana-highlighted-field@\"\r\n    ],\r\n    \"post_tags\": [\r\n      \"@/kibana-highlighted-field@\"\r\n    ],\r\n    \"fields\": {\r\n      \"*\": {}\r\n    }\r\n  },\r\n  \"aggs\": {\r\n    \"2\": {\r\n      \"date_histogram\": {\r\n        \"field\": \"@timestamp\",\r\n        \"interval\": \"30s\",\r\n        \"pre_zone\": \"-07:00\",\r\n        \"pre_zone_adjust_large_interval\": true,\r\n        \"min_doc_count\": 0,\r\n        \"extended_bounds\": {\r\n          \"min\": 1432704390127,\r\n          \"max\": 1432705290128\r\n        }\r\n      }\r\n    }\r\n  },\r\n  \"query\": {\r\n    \"filtered\": {\r\n      \"query\": {\r\n        \"match_all\": {}\r\n      },\r\n      \"filter\": {\r\n        \"bool\": {\r\n          \"must\": [\r\n            {\r\n              \"range\": {\r\n                \"@timestamp\": {\r\n                  \"gte\": 1432704390134,\r\n                  \"lte\": 1432705290134\r\n                }\r\n              }\r\n            }\r\n          ],\r\n          \"must_not\": []\r\n        }\r\n      }\r\n    }\r\n  },\r\n  \"fields\": [\r\n    \"*\",\r\n    \"_source\"\r\n  ],\r\n  \"script_fields\": {},\r\n  \"fielddata_fields\": [\r\n    \"@timestamp\"\r\n  ]\r\n}\r\n```\r\n\r\nResponse\r\n```json\r\n{\r\n  \"responses\": [\r\n    {\r\n      \"took\": 44,\r\n      \"timed_out\": false,\r\n      \"_shards\": {\r\n        \"total\": 5,\r\n        \"successful\": 0,\r\n        \"failed\": 5,\r\n        \"failures\": [\r\n          {\r\n            \"index\": \"customindex-2015.05.27\",\r\n            \"shard\": 0,\r\n            \"status\": 500,\r\n            \"reason\": \"RemoteTransportException[[fqdn.omitted.com][inet[/192.168.1.110:9300]][indices:data/read/search[phase/fetch/id]]]; nested: ElasticsearchIllegalStateException[No matching token for number_type [BIG_INTEGER]]; \"\r\n          },\r\n          {\r\n            \"index\": \"customindex-2015.05.27\",\r\n            \"shard\": 1,\r\n            \"status\": 500,\r\n            \"reason\": \"RemoteTransportException[[fqdn.omitted.com][inet[/192.168.1.110:9300]][indices:data/read/search[phase/fetch/id]]]; nested: ElasticsearchIllegalStateException[No matching token for number_type [BIG_INTEGER]]; \"\r\n          },\r\n          {\r\n            \"index\": \"customindex-2015.05.27\",\r\n            \"shard\": 2,\r\n            \"status\": 500,\r\n            \"reason\": \"RemoteTransportException[[fqdn.omitted.com][inet[/192.168.1.110:9300]][indices:data/read/search[phase/fetch/id]]]; nested: ElasticsearchIllegalStateException[No matching token for number_type [BIG_INTEGER]]; \"\r\n          },\r\n          {\r\n            \"index\": \"customindex-2015.05.27\",\r\n            \"shard\": 3,\r\n            \"status\": 500,\r\n            \"reason\": \"RemoteTransportException[[fqdn.omitted.com][inet[/192.168.112.177:9300]][indices:data/read/search[phase/fetch/id]]]; nested: ElasticsearchIllegalStateException[No matching token for number_type [BIG_INTEGER]]; \"\r\n          },\r\n          {\r\n            \"index\": \"customindex-2015.05.27\",\r\n            \"shard\": 4,\r\n            \"status\": 500,\r\n            \"reason\": \"RemoteTransportException[[fqdn.omitted.com][inet[/192.168.1.110:9300]][indices:data/read/search[phase/fetch/id]]]; nested: ElasticsearchIllegalStateException[No matching token for number_type [BIG_INTEGER]]; \"\r\n          }\r\n        ]\r\n      },\r\n      \"hits\": {\r\n        \"total\": 86519,\r\n        \"max_score\": null,\r\n        \"hits\": []\r\n      },\r\n      \"aggregations\": {\r\n        \"2\": {\r\n          \"buckets\": [\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:16:30.000Z\",\r\n              \"key\": 1432703790000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:17:00.000Z\",\r\n              \"key\": 1432703820000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:17:30.000Z\",\r\n              \"key\": 1432703850000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:18:00.000Z\",\r\n              \"key\": 1432703880000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:18:30.000Z\",\r\n              \"key\": 1432703910000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:19:00.000Z\",\r\n              \"key\": 1432703940000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:19:30.000Z\",\r\n              \"key\": 1432703970000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:20:00.000Z\",\r\n              \"key\": 1432704000000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:20:30.000Z\",\r\n              \"key\": 1432704030000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:21:00.000Z\",\r\n              \"key\": 1432704060000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:21:30.000Z\",\r\n              \"key\": 1432704090000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:22:00.000Z\",\r\n              \"key\": 1432704120000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:22:30.000Z\",\r\n              \"key\": 1432704150000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:23:00.000Z\",\r\n              \"key\": 1432704180000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:23:30.000Z\",\r\n              \"key\": 1432704210000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:24:00.000Z\",\r\n              \"key\": 1432704240000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:24:30.000Z\",\r\n              \"key\": 1432704270000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:25:00.000Z\",\r\n              \"key\": 1432704300000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:25:30.000Z\",\r\n              \"key\": 1432704330000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:26:00.000Z\",\r\n              \"key\": 1432704360000,\r\n              \"doc_count\": 0\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:26:30.000Z\",\r\n              \"key\": 1432704390000,\r\n              \"doc_count\": 4209\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:27:00.000Z\",\r\n              \"key\": 1432704420000,\r\n              \"doc_count\": 7270\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:27:30.000Z\",\r\n              \"key\": 1432704450000,\r\n              \"doc_count\": 6646\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:28:00.000Z\",\r\n              \"key\": 1432704480000,\r\n              \"doc_count\": 7181\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:28:30.000Z\",\r\n              \"key\": 1432704510000,\r\n              \"doc_count\": 6612\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:29:00.000Z\",\r\n              \"key\": 1432704540000,\r\n              \"doc_count\": 6753\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:29:30.000Z\",\r\n              \"key\": 1432704570000,\r\n              \"doc_count\": 6509\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:30:00.000Z\",\r\n              \"key\": 1432704600000,\r\n              \"doc_count\": 10295\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:30:30.000Z\",\r\n              \"key\": 1432704630000,\r\n              \"doc_count\": 13073\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:31:00.000Z\",\r\n              \"key\": 1432704660000,\r\n              \"doc_count\": 14627\r\n            },\r\n            {\r\n              \"key_as_string\": \"2015-05-27T05:31:30.000Z\",\r\n              \"key\": 1432704690000,\r\n              \"doc_count\": 3344\r\n            }\r\n          ]\r\n        }\r\n      }\r\n    }\r\n  ]\r\n}\r\n```\r\n\r\nTemplate & Mapping\r\n```json\r\n{\r\n  \"template\": \"customindex-*\",\r\n  \"settings\": {\r\n    \"index.refresh_integererval\": \"5s\",\r\n    \"index.number_of_shards\": \"5\"\r\n  },\r\n  \"mappings\": {\r\n    \"_default_\": {\r\n      \"_all\": {\r\n        \"enabled\": false\r\n      }\r\n    },\r\n    \"netflow\": {\r\n      \"properties\": {\r\n        \"@timestamp\": {\r\n          \"type\": \"date\",\r\n          \"format\": \"dateOptionalTime\"\r\n        },\r\n        \"@version\": {\r\n          \"type\": \"string\"\r\n        },\r\n        \"host\": {\r\n          \"type\": \"string\"\r\n        },\r\n        \"netflow\": {\r\n          \"properties\": {\r\n            \"flow_seq_num\": {\r\n              \"type\": \"long\"\r\n            },\r\n            \"flowset_id\": {\r\n              \"type\": \"long\"\r\n            },\r\n            \"nf_f_conn_id\": {\r\n              \"type\": \"long\"\r\n            },\r\n            \"nf_f_dst_addr_ipv4\": {\r\n              \"type\": \"string\",\r\n              \"index\": \"not_analyzed\"\r\n            },\r\n            \"nf_f_dst_intf_id\": {\r\n              \"type\": \"long\"\r\n            },\r\n            \"nf_f_dst_port\": {\r\n              \"type\": \"long\"\r\n            },\r\n            \"nf_f_egress_acl_id\": {\r\n              \"type\": \"string\",\r\n              \"index\": \"not_analyzed\"\r\n            },\r\n            \"nf_f_event_time_msec\": {\r\n              \"type\": \"string\",\r\n              \"index\": \"not_analyzed\"\r\n            },\r\n            \"nf_f_flow_create_time_msec\": {\r\n              \"type\": \"string\",\r\n              \"index\": \"not_analyzed\"\r\n            },\r\n            \"nf_f_fwd_flow_delta_bytes\": {\r\n              \"type\": \"string\",\r\n              \"index\": \"not_analyzed\"\r\n            },\r\n            \"nf_f_rev_flow_delta_bytes\": {\r\n              \"type\": \"string\",\r\n              \"index\": \"not_analyzed\"\r\n            },\r\n            \"nf_f_flow_bytes\": {\r\n              \"type\": \"long\"\r\n            },\r\n            \"nf_f_fw_event\": {\r\n              \"type\": \"long\"\r\n            },\r\n            \"nf_f_fw_ext_event\": {\r\n              \"type\": \"long\"\r\n            },\r\n            \"nf_f_icmp_code\": {\r\n              \"type\": \"long\"\r\n            },\r\n            \"nf_f_icmp_type\": {\r\n              \"type\": \"long\"\r\n            },\r\n            \"nf_f_icmp_type_ipv6\": {\r\n              \"type\": \"string\",\r\n              \"index\": \"not_analyzed\"\r\n            },\r\n            \"nf_f_icmp_code_ipv6\": {\r\n              \"type\": \"string\",\r\n              \"index\": \"not_analyzed\"\r\n            },\r\n            \"nf_f_ingress_acl_id\": {\r\n              \"type\": \"string\",\r\n              \"index\": \"not_analyzed\"\r\n            },\r\n            \"nf_f_protocol\": {\r\n              \"type\": \"long\"\r\n            },\r\n            \"nf_f_src_addr_ipv4\": {\r\n              \"type\": \"string\",\r\n              \"index\": \"not_analyzed\"\r\n            },\r\n            \"nf_f_src_intf_id\": {\r\n              \"type\": \"long\"\r\n            },\r\n            \"nf_f_src_port\": {\r\n              \"type\": \"long\"\r\n            },\r\n            \"nf_f_username\": {\r\n              \"type\": \"string\"\r\n            },\r\n            \"nf_f_xlate_dst_addr_ipv4\": {\r\n              \"type\": \"string\"\r\n            },\r\n            \"nf_f_xlate_dst_port\": {\r\n              \"type\": \"long\"\r\n            },\r\n            \"nf_f_xlate_src_addr_ipv4\": {\r\n              \"type\": \"string\",\r\n              \"index\": \"not_analyzed\"\r\n            },\r\n            \"nf_f_xlate_src_port\": {\r\n              \"type\": \"long\"\r\n            },\r\n            \"version\": {\r\n              \"type\": \"long\"\r\n            }\r\n          }\r\n        },\r\n        \"type\": {\r\n          \"type\": \"string\"\r\n        }\r\n      }\r\n    }\r\n  }\r\n}\r\n```"}}