{"action":"created","issue":{"url":"https://api.github.com/repos/Graylog2/graylog2-server/issues/1002","labels_url":"https://api.github.com/repos/Graylog2/graylog2-server/issues/1002/labels{/name}","comments_url":"https://api.github.com/repos/Graylog2/graylog2-server/issues/1002/comments","events_url":"https://api.github.com/repos/Graylog2/graylog2-server/issues/1002/events","html_url":"https://github.com/Graylog2/graylog2-server/issues/1002","id":59087495,"number":1002,"title":"Exception raised during index rotation","user":{"login":"jframeau","id":498115,"avatar_url":"https://avatars.githubusercontent.com/u/498115?v=3","gravatar_id":"","url":"https://api.github.com/users/jframeau","html_url":"https://github.com/jframeau","followers_url":"https://api.github.com/users/jframeau/followers","following_url":"https://api.github.com/users/jframeau/following{/other_user}","gists_url":"https://api.github.com/users/jframeau/gists{/gist_id}","starred_url":"https://api.github.com/users/jframeau/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jframeau/subscriptions","organizations_url":"https://api.github.com/users/jframeau/orgs","repos_url":"https://api.github.com/users/jframeau/repos","events_url":"https://api.github.com/users/jframeau/events{/privacy}","received_events_url":"https://api.github.com/users/jframeau/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/Graylog2/graylog2-server/labels/bug","name":"bug","color":"e11d21"}],"state":"closed","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/Graylog2/graylog2-server/milestones/35","html_url":"https://github.com/Graylog2/graylog2-server/milestones/1.2.0","labels_url":"https://api.github.com/repos/Graylog2/graylog2-server/milestones/35/labels","id":1119344,"number":35,"title":"1.2.0","description":"","creator":{"login":"lennartkoopmann","id":35022,"avatar_url":"https://avatars.githubusercontent.com/u/35022?v=3","gravatar_id":"","url":"https://api.github.com/users/lennartkoopmann","html_url":"https://github.com/lennartkoopmann","followers_url":"https://api.github.com/users/lennartkoopmann/followers","following_url":"https://api.github.com/users/lennartkoopmann/following{/other_user}","gists_url":"https://api.github.com/users/lennartkoopmann/gists{/gist_id}","starred_url":"https://api.github.com/users/lennartkoopmann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lennartkoopmann/subscriptions","organizations_url":"https://api.github.com/users/lennartkoopmann/orgs","repos_url":"https://api.github.com/users/lennartkoopmann/repos","events_url":"https://api.github.com/users/lennartkoopmann/events{/privacy}","received_events_url":"https://api.github.com/users/lennartkoopmann/received_events","type":"User","site_admin":false},"open_issues":22,"closed_issues":66,"state":"open","created_at":"2015-05-19T21:34:25Z","updated_at":"2015-08-25T12:09:09Z","due_on":null,"closed_at":null},"comments":6,"created_at":"2015-02-26T15:22:03Z","updated_at":"2015-08-25T12:09:09Z","closed_at":"2015-08-25T12:09:09Z","body":"CentOS 6.6\r\ngraylog-server v1.0.0 (Oracle Corporation 1.7.0_75 / Linux 2.6.32-504.8.1.el6.x86_64) \r\nelasticsearch-1.3.9\r\n\r\nLogs from Graylog\r\n===============\r\n    2015-02-26T14:56:10.985+01:00 INFO  [IndexRotationThread] Deflector index <graylog2_151> should be rotated, Pointing deflector to new index now!\r\n    2015-02-26T14:56:10.985+01:00 INFO  [Deflector] Cycling deflector to next index now.\r\n    2015-02-26T14:56:11.034+01:00 INFO  [Deflector] Cycling from <graylog2_151> to <graylog2_152>\r\n    2015-02-26T14:56:11.034+01:00 INFO  [Deflector] Creating index target <graylog2_152>...\r\n    2015-02-26T14:56:11.915+01:00 INFO  [Deflector] Done!\r\n    2015-02-26T14:56:11.915+01:00 INFO  [Deflector] Pointing deflector to new target index....\r\n    2015-02-26T14:56:11.947+01:00 INFO  [SystemJobManager] Submitted SystemJob <38eda8a2-bdbf-11e4-9ee3-0050568576b3> [org.graylog2.indexer.SetIndexReadOnlyJob]\r\n    2015-02-26T14:56:11.948+01:00 INFO  [RebuildIndexRangesJob] Re-calculating index ranges.\r\n    2015-02-26T14:56:11.948+01:00 INFO  [SystemJobManager] Submitted SystemJob <38edf6c1-bdbf-11e4-9ee3-0050568576b3> [org.graylog2.indexer.ranges.RebuildIndexRangesJob]\r\n    2015-02-26T14:56:11.949+01:00 INFO  [Deflector] Done!\r\n    2015-02-26T14:56:13.570+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_133] in [1448ms].\r\n    2015-02-26T14:56:13.876+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_149] in [305ms].\r\n    2015-02-26T14:56:14.096+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_137] in [219ms].\r\n    2015-02-26T14:56:14.257+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_136] in [161ms].\r\n    2015-02-26T14:56:14.480+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_135] in [222ms].\r\n    2015-02-26T14:56:14.659+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_134] in [178ms].\r\n    2015-02-26T14:56:14.875+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_145] in [215ms].\r\n    2015-02-26T14:56:15.074+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_146] in [198ms].\r\n    2015-02-26T14:56:15.273+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_147] in [198ms].\r\n    2015-02-26T14:56:15.508+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_139] in [235ms].\r\n    2015-02-26T14:56:15.706+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_148] in [197ms].\r\n    2015-02-26T14:56:15.915+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_138] in [208ms].\r\n    2015-02-26T14:56:16.114+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_141] in [198ms].\r\n    2015-02-26T14:56:16.294+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_142] in [180ms].\r\n    2015-02-26T14:56:16.518+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_143] in [223ms].\r\n    2015-02-26T14:56:16.718+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_144] in [200ms].\r\n    2015-02-26T14:56:17.837+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_151] in [1118ms].\r\n    2015-02-26T14:56:18.169+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_150] in [331ms].\r\n    2015-02-26T14:56:18.188+01:00 INFO  [RebuildIndexRangesJob] Calculated range of [graylog2_140] in [18ms].\r\n    2015-02-26T14:56:18.235+01:00 INFO  [RebuildIndexRangesJob] Done calculating index ranges for 19 indices. Took 6112ms.\r\n    2015-02-26T14:56:18.236+01:00 INFO  [SystemJobManager] SystemJob <38edf6c1-bdbf-11e4-9ee3-0050568576b3> [org.graylog2.indexer.ranges.RebuildIndexRangesJob] finished in 6288ms.\r\n    2015-02-26T14:56:20.890+01:00 WARN  [IndexRotationThread] Deflector is pointing to [graylog2_152], not the newest one: [graylog2_151]. Re-pointing.\r\n    2015-02-26T14:56:21.030+01:00 INFO  [IndexRotationThread] Deflector index <graylog2_151> should be rotated, Pointing deflector to new index now!\r\n    2015-02-26T14:56:21.030+01:00 INFO  [Deflector] Cycling deflector to next index now.\r\n    2015-02-26T14:56:21.084+01:00 INFO  [Deflector] Cycling from <graylog2_151> to <graylog2_152>\r\n    2015-02-26T14:56:21.084+01:00 INFO  [Deflector] Creating index target <graylog2_152>...\r\n    2015-02-26T14:56:21.210+01:00 ERROR [IndexRotationThread] Couldn't point deflector to a new index\r\n    org.elasticsearch.indices.IndexAlreadyExistsException: [graylog2_152] already exists\r\n\r\n\r\nLogs at the same time from EL:\r\n=========================\r\n```\r\n[2015-02-26 14:56:11,607][INFO ][cluster.metadata         ] [Hamilton Slade] [graylog2_152] creating index, cause [api], shards [4]/[0], mappings []\r\n[2015-02-26 14:56:11,889][INFO ][cluster.metadata         ] [Hamilton Slade] [graylog2_152] create_mapping [message]\r\n[2015-02-26 14:56:31,680][INFO ][monitor.jvm              ] [Hamilton Slade] [gc][young][18830][5192] duration [754ms], collections [1]/[1.2s], total [754ms]/[1.4m], memory [1.9gb]->[1.4gb]/[3.9gb], all_pools {[young] [489.6mb]->[3.5mb]/[532.5mb]}{[survivor] [10.3mb]->[38.8mb]/[66.5mb]}{[old] [1.4gb]->[1.4gb]/[3.3gb]}\r\n[2015-02-26 15:10:08,467][INFO ][monitor.jvm              ] [Hamilton Slade] [gc][young][19645][5357] duration [890ms], collections [1]/[1.3s], total [890ms]/[1.5m], memory [1.9gb]->[1.5gb]/[3.9gb], all_pools {[young] [412.2mb]->[8.8mb]/[532.5mb]}{[survivor] [12.9mb]->[5.3mb]/[66.5mb]}{[old] [1.5gb]->[1.5gb]/[3.3gb]}\r\n[2015-02-26 15:11:18,802][WARN ][monitor.jvm              ] [Hamilton Slade] [gc][young][19714][5373] duration [2s], collections [1]/[2.3s], total [2s]/[1.5m], memory [1.8gb]->[1.5gb]/[3.9gb], all_pools {[young] [292.2mb]->[10.6mb]/[532.5mb]}{[survivor] [7.4mb]->[699.7kb]/[66.5mb]}{[old] [1.5gb]->[1.5gb]/[3.3gb]}\r\n[2015-02-26 15:25:05,046][WARN ][monitor.jvm              ] [Hamilton Slade] [gc][young][20538][5541] duration [1.6s], collections [1]/[2.3s], total [1.6s]/[1.6m], memory [2gb]->[1.6gb]/[3.9gb], all_pools {[young] [499.2mb]->[8.3mb]/[532.5mb]}{[survivor] [19.3mb]->[5.5mb]/[66.5mb]}{[old] [1.5gb]->[1.5gb]/[3.3gb]}\r\n```\r\nThx for this great piece of code\r\n\r\njfr"},"comment":{"url":"https://api.github.com/repos/Graylog2/graylog2-server/issues/comments/134565629","html_url":"https://github.com/Graylog2/graylog2-server/issues/1002#issuecomment-134565629","issue_url":"https://api.github.com/repos/Graylog2/graylog2-server/issues/1002","id":134565629,"user":{"login":"joschi","id":43951,"avatar_url":"https://avatars.githubusercontent.com/u/43951?v=3","gravatar_id":"","url":"https://api.github.com/users/joschi","html_url":"https://github.com/joschi","followers_url":"https://api.github.com/users/joschi/followers","following_url":"https://api.github.com/users/joschi/following{/other_user}","gists_url":"https://api.github.com/users/joschi/gists{/gist_id}","starred_url":"https://api.github.com/users/joschi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joschi/subscriptions","organizations_url":"https://api.github.com/users/joschi/orgs","repos_url":"https://api.github.com/users/joschi/repos","events_url":"https://api.github.com/users/joschi/events{/privacy}","received_events_url":"https://api.github.com/users/joschi/received_events","type":"User","site_admin":false},"created_at":"2015-08-25T12:09:09Z","updated_at":"2015-08-25T12:09:09Z","body":"This should be fixed in Graylog 1.2.0. Please re-open the ticket if you still have this problem with Graylog 1.2.0."}}