{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/15267","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/15267/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/15267/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/15267/events","html_url":"https://github.com/elastic/elasticsearch/issues/15267","id":120644830,"number":15267,"title":"Field data loading is forbidden on path","user":{"login":"velion","id":11274206,"avatar_url":"https://avatars.githubusercontent.com/u/11274206?v=3","gravatar_id":"","url":"https://api.github.com/users/velion","html_url":"https://github.com/velion","followers_url":"https://api.github.com/users/velion/followers","following_url":"https://api.github.com/users/velion/following{/other_user}","gists_url":"https://api.github.com/users/velion/gists{/gist_id}","starred_url":"https://api.github.com/users/velion/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/velion/subscriptions","organizations_url":"https://api.github.com/users/velion/orgs","repos_url":"https://api.github.com/users/velion/repos","events_url":"https://api.github.com/users/velion/events{/privacy}","received_events_url":"https://api.github.com/users/velion/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-12-06T16:45:25Z","updated_at":"2015-12-10T08:41:49Z","closed_at":null,"body":"when i Click “visualize” and then ,display error \"Visualize:java.lang.lllegalStateException:Field daVisualize\",or i used dashboard module ,before logstash auto update index,it was normal;when time is 8:00 am,dashboard module will not work,and display the error.\r\n\r\nerror details:\r\n\r\n1.kibana page error:\r\nError: Request to Elasticsearch failed: {\"error\":{\"root_cause\":[{\"type\":\"exception\",\"reason\":\"java.lang.IllegalStateException: Field data loading is forbidden on path\"}],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[{\"shard\":0,\"index\":\"logstash-2015.11.29\",\"node\":\"ODyds4KgQsqrxczx1ana8A\",\"reason\":{\"type\":\"exception\",\"reason\":\"java.lang.IllegalStateException: Field data loading is forbidden on path\",\"caused_by\":{\"type\":\"unchecked_execution_exception\",\"reason\":\"java.lang.IllegalStateException: Field data loading is forbidden on path\",\"caused_by\":{\"type\":\"illegal_state_exception\",\"reason\":\"Field data loading is forbidden on path\"}}}}]}} KbnError@http://www.xxx.cn:80/bundles/c ... 64:30 RequestFailure@http://www.xxx.cn:80/bundles/c ... 97:19 http://www.xxx.cn:80/bundles/k ... 05:57 http://www.xxx.cn:80/bundles/c ... 91:28 http://www.xxx.cn:80/bundles/c ... 60:31 map@[native code] map@http://www.xxx.cn:80/bundles/c ... 59:34 callResponseHandlers@http://www.xxx.cn:80/bundles/k ... 77:26 http://www.xxx.cn:80/bundles/k ... 84:37 processQueue@http://www.xxx.cn:80/bundles/c ... 09:31 http://www.xxx.cn:80/bundles/c ... 25:40 $eval@http://www.xxx.cn:80/bundles/c ... 53:29 $digest@http://www.xxx.cn:80/bundles/c ... 64:37 $apply@http://www.xxx.cn:80/bundles/c ... 61:32 done@http://www.xxx.cn:80/bundles/c ... 10:54 completeRequest@http://www.xxx.cn:80/bundles/c ... 08:16 requestLoaded@http://www.xxx.cn:80/bundles/commons.bundle.js:37749:25\r\n\r\n2.elasticsearch error log:\r\n\r\n[2015-11-30 00:04:53,395][DEBUG][action.search.type       ] [Thumbelina] [logstash-2015.11.29][2], node[acrTX4O0RciN8ppbSdfoww], [P], v[4], s[STARTED], a[id=ogwkZP0yQCCgatW0_tnvnw]: Failed to execute [org.elasticsearch.action.search.SearchRequest@aef2ffe] lastShard [true]\r\nRemoteTransportException[[Thumbelina][192.168.1.76:9300][indices:data/read/search[phase/query]]]; nested: QueryPhaseExecutionException[Query Failed [Failed to execute main query]]; nested: ElasticsearchException[java.lang.IllegalStateException: Field data loading is forbidden on response]; nested: UncheckedExecutionException[java.lang.IllegalStateException: Field data loading is forbidden on response]; nested: IllegalStateException[Field data loading is forbidden on response];\r\nCaused by: QueryPhaseExecutionException[Query Failed [Failed to execute main query]]; nested: ElasticsearchException[java.lang.IllegalStateException: Field data loading is forbidden on response]; nested: UncheckedExecutionException[java.lang.IllegalStateException: Field data loading is forbidden on response]; nested: IllegalStateException[Field data loading is forbidden on response];\r\n        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:343)\r\n        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:106)\r\n        at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:363)\r\n        at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:375)\r\n        at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:368)\r\n        at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:365)\r\n        at org.elasticsearch.shield.transport.ShieldServerTransportService$ProfileSecuredRequestHandler.messageReceived(ShieldServerTransportService.java:165)\r\n        at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:350)\r\n        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)\r\n        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n        at java.lang.Thread.run(Thread.java:745)\r\nCaused by: ElasticsearchException[java.lang.IllegalStateException: Field data loading is forbidden on response]; nested: UncheckedExecutionException[java.lang.IllegalStateException: Field data loading is forbidden on response]; nested: IllegalStateException[Field data loading is forbidden on response];\r\n        at org.elasticsearch.index.fielddata.plain.AbstractIndexFieldData.load(AbstractIndexFieldData.java:82)\r\n        at org.elasticsearch.search.aggregations.support.ValuesSource$Bytes$FieldData.bytesValues(ValuesSource.java:195)\r\n        at org.elasticsearch.search.aggregations.bucket.terms.StringTermsAggregator.getLeafCollector(StringTermsAggregator.java:73)\r\n        at org.elasticsearch.search.aggregations.AggregatorBase.getLeafCollector(AggregatorBase.java:132)\r\n        at org.elasticsearch.search.aggregations.AggregatorBase.getLeafCollector(AggregatorBase.java:38)\r\n        at org.apache.lucene.search.MultiCollector.getLeafCollector(MultiCollector.java:117)\r\n        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:763)\r\n        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:486)\r\n        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:324)\r\n        ... 11 more\r\nVian  00:18:36\r\nCaused by: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalStateException: Field data loading is forbidden on response\r\n        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2203)\r\n        at com.google.common.cache.LocalCache.get(LocalCache.java:3937)\r\n        at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4739)\r\n        at org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache.load(IndicesFieldDataCache.java:156)\r\n        at org.elasticsearch.index.fielddata.plain.AbstractIndexFieldData.load(AbstractIndexFieldData.java:76)\r\n        … 19 more\r\nCaused by: java.lang.IllegalStateException: Field data loading is forbidden on response\r\n        at org.elasticsearch.index.fielddata.plain.DisabledIndexFieldData.fail(DisabledIndexFieldData.java:68)\r\n        at org.elasticsearch.index.fielddata.plain.DisabledIndexFieldData.loadDirect(DisabledIndexFieldData.java:54)\r\n        at org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache$1.call(IndicesFieldDataCache.java:163)\r\n        at org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache$1.call(IndicesFieldDataCache.java:156)\r\n        at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4742)\r\n        at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)\r\n        at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)\r\n        at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)\r\n        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)\r\n        … 23 more\r\n"},"comment":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/comments/163534485","html_url":"https://github.com/elastic/elasticsearch/issues/15267#issuecomment-163534485","issue_url":"https://api.github.com/repos/elastic/elasticsearch/issues/15267","id":163534485,"user":{"login":"cybacolt","id":730769,"avatar_url":"https://avatars.githubusercontent.com/u/730769?v=3","gravatar_id":"","url":"https://api.github.com/users/cybacolt","html_url":"https://github.com/cybacolt","followers_url":"https://api.github.com/users/cybacolt/followers","following_url":"https://api.github.com/users/cybacolt/following{/other_user}","gists_url":"https://api.github.com/users/cybacolt/gists{/gist_id}","starred_url":"https://api.github.com/users/cybacolt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cybacolt/subscriptions","organizations_url":"https://api.github.com/users/cybacolt/orgs","repos_url":"https://api.github.com/users/cybacolt/repos","events_url":"https://api.github.com/users/cybacolt/events{/privacy}","received_events_url":"https://api.github.com/users/cybacolt/received_events","type":"User","site_admin":false},"created_at":"2015-12-10T08:41:49Z","updated_at":"2015-12-10T08:41:49Z","body":"this seems to have had a good outcome (visualizations are working, at least initially)... be aware this will wipe all previous index data:\r\n```\r\nservice elasticsearch stop\r\nservice logstash stop\r\napt-cache remove elasticsearch\r\nmv /var/lib/elasticsearch /var/lib/elasticsearch.back\r\napt-get install elasticsearch=2.0.1\r\nservice elasticsearch restart\r\nservice logstash restart\r\n```\r\n\r\nthen repeat these steps to reinstall 2.1.0 with the good 2.0.1 `/var/lib/elasticsearch` contents.\r\n\r\nperhaps i'd broken my indexes somehow... not sure... eitherway, its working atm."}}