{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/13819","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/13819/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/13819/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/13819/events","html_url":"https://github.com/elastic/elasticsearch/issues/13819","id":108538714,"number":13819,"title":"Should forced merges be IO throttled?","user":{"login":"mikemccand","id":796508,"avatar_url":"https://avatars.githubusercontent.com/u/796508?v=3","gravatar_id":"","url":"https://api.github.com/users/mikemccand","html_url":"https://github.com/mikemccand","followers_url":"https://api.github.com/users/mikemccand/followers","following_url":"https://api.github.com/users/mikemccand/following{/other_user}","gists_url":"https://api.github.com/users/mikemccand/gists{/gist_id}","starred_url":"https://api.github.com/users/mikemccand/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mikemccand/subscriptions","organizations_url":"https://api.github.com/users/mikemccand/orgs","repos_url":"https://api.github.com/users/mikemccand/repos","events_url":"https://api.github.com/users/mikemccand/events{/privacy}","received_events_url":"https://api.github.com/users/mikemccand/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/:Core","name":":Core","color":"009800"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/discuss","name":"discuss","color":"fbca04"}],"state":"open","locked":false,"assignee":{"login":"mikemccand","id":796508,"avatar_url":"https://avatars.githubusercontent.com/u/796508?v=3","gravatar_id":"","url":"https://api.github.com/users/mikemccand","html_url":"https://github.com/mikemccand","followers_url":"https://api.github.com/users/mikemccand/followers","following_url":"https://api.github.com/users/mikemccand/following{/other_user}","gists_url":"https://api.github.com/users/mikemccand/gists{/gist_id}","starred_url":"https://api.github.com/users/mikemccand/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mikemccand/subscriptions","organizations_url":"https://api.github.com/users/mikemccand/orgs","repos_url":"https://api.github.com/users/mikemccand/repos","events_url":"https://api.github.com/users/mikemccand/events{/privacy}","received_events_url":"https://api.github.com/users/mikemccand/received_events","type":"User","site_admin":false},"milestone":null,"comments":1,"created_at":"2015-09-27T14:17:02Z","updated_at":"2015-09-27T16:01:54Z","closed_at":null,"body":"With the new auto-IO throttling in Lucene's ConcurrentMergeScheduler as of Lucene 5.0 (ES 2.0), forced merges (optimize) are never throttled by default.\r\n\r\nI think this is fair: the user asked for the index to merge down to 1 (default) segment so it should happen as quickly as possible.  And users generally should force merge only when the index is otherwise idle ... 5.0 has been out for a while now and I haven't heard user complaints about this.\r\n\r\nCMS has a setting to change the default MB/sec for forced merges, but we haven't exposed it in ES, and this is a behavior change from before (in ES 1.x) when all merges IO (natural and forced) were held to the shared default of 20 MB/sec.\r\n\r\nNote that users can still set the old IO store throttling (`indices/index.store.throttle.max_bytes_per_sec`) and it will still apply to all merges.\r\n\r\nSo we could do nothing here, and if that turns out to be a problem, users can still use the old way (but we need to re-document this if so).\r\n\r\nOr we can open up a MB/sec setting for forced merges ... I don't really like that: ES has too many settings, and \"MB/sec\" is too raw/low-level for a user who's kicking off a forced merge to necessarily know the right value for.\r\n\r\nOr we could hardwire the force merge MB/sec to be a function of the current natural merge MB/sec auto IO throttle, maybe just `==`."},"comment":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/comments/143571158","html_url":"https://github.com/elastic/elasticsearch/issues/13819#issuecomment-143571158","issue_url":"https://api.github.com/repos/elastic/elasticsearch/issues/13819","id":143571158,"user":{"login":"rmuir","id":504194,"avatar_url":"https://avatars.githubusercontent.com/u/504194?v=3","gravatar_id":"","url":"https://api.github.com/users/rmuir","html_url":"https://github.com/rmuir","followers_url":"https://api.github.com/users/rmuir/followers","following_url":"https://api.github.com/users/rmuir/following{/other_user}","gists_url":"https://api.github.com/users/rmuir/gists{/gist_id}","starred_url":"https://api.github.com/users/rmuir/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rmuir/subscriptions","organizations_url":"https://api.github.com/users/rmuir/orgs","repos_url":"https://api.github.com/users/rmuir/repos","events_url":"https://api.github.com/users/rmuir/events{/privacy}","received_events_url":"https://api.github.com/users/rmuir/received_events","type":"User","site_admin":false},"created_at":"2015-09-27T16:01:54Z","updated_at":"2015-09-27T16:01:54Z","body":"> I think this is fair: the user asked for the index to merge down to 1 (default) segment so it should happen as quickly as possible.\r\n\r\n+1, they asked to do this."}}