{"action":"closed","issue":{"url":"https://api.github.com/repos/rethinkdb/docs/issues/787","labels_url":"https://api.github.com/repos/rethinkdb/docs/issues/787/labels{/name}","comments_url":"https://api.github.com/repos/rethinkdb/docs/issues/787/comments","events_url":"https://api.github.com/repos/rethinkdb/docs/issues/787/events","html_url":"https://github.com/rethinkdb/docs/issues/787","id":86424232,"number":787,"title":"Document emergency repair command","user":{"login":"timmaxw","id":316661,"avatar_url":"https://avatars.githubusercontent.com/u/316661?v=3","gravatar_id":"","url":"https://api.github.com/users/timmaxw","html_url":"https://github.com/timmaxw","followers_url":"https://api.github.com/users/timmaxw/followers","following_url":"https://api.github.com/users/timmaxw/following{/other_user}","gists_url":"https://api.github.com/users/timmaxw/gists{/gist_id}","starred_url":"https://api.github.com/users/timmaxw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/timmaxw/subscriptions","organizations_url":"https://api.github.com/users/timmaxw/orgs","repos_url":"https://api.github.com/users/timmaxw/repos","events_url":"https://api.github.com/users/timmaxw/events{/privacy}","received_events_url":"https://api.github.com/users/timmaxw/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":{"login":"chipotle","id":467928,"avatar_url":"https://avatars.githubusercontent.com/u/467928?v=3","gravatar_id":"","url":"https://api.github.com/users/chipotle","html_url":"https://github.com/chipotle","followers_url":"https://api.github.com/users/chipotle/followers","following_url":"https://api.github.com/users/chipotle/following{/other_user}","gists_url":"https://api.github.com/users/chipotle/gists{/gist_id}","starred_url":"https://api.github.com/users/chipotle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chipotle/subscriptions","organizations_url":"https://api.github.com/users/chipotle/orgs","repos_url":"https://api.github.com/users/chipotle/repos","events_url":"https://api.github.com/users/chipotle/events{/privacy}","received_events_url":"https://api.github.com/users/chipotle/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/rethinkdb/docs/milestones/8","html_url":"https://github.com/rethinkdb/docs/milestones/2.1","labels_url":"https://api.github.com/repos/rethinkdb/docs/milestones/8/labels","id":1045737,"number":8,"title":"2.1","description":null,"creator":{"login":"VeXocide","id":167416,"avatar_url":"https://avatars.githubusercontent.com/u/167416?v=3","gravatar_id":"","url":"https://api.github.com/users/VeXocide","html_url":"https://github.com/VeXocide","followers_url":"https://api.github.com/users/VeXocide/followers","following_url":"https://api.github.com/users/VeXocide/following{/other_user}","gists_url":"https://api.github.com/users/VeXocide/gists{/gist_id}","starred_url":"https://api.github.com/users/VeXocide/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/VeXocide/subscriptions","organizations_url":"https://api.github.com/users/VeXocide/orgs","repos_url":"https://api.github.com/users/VeXocide/repos","events_url":"https://api.github.com/users/VeXocide/events{/privacy}","received_events_url":"https://api.github.com/users/VeXocide/received_events","type":"User","site_admin":false},"open_issues":4,"closed_issues":5,"state":"open","created_at":"2015-03-30T22:32:47Z","updated_at":"2015-06-30T23:39:40Z","due_on":null,"closed_at":null},"comments":2,"created_at":"2015-06-09T03:14:08Z","updated_at":"2015-06-30T23:39:40Z","closed_at":"2015-06-30T23:39:40Z","body":"We need to document the new `emergency_repair` mode of `table.reconfigure()` for 2.1.\r\n\r\n##### Background\r\n\r\nNormally, failure recovery in RethinkDB works as follows: Suppose that some servers have died or have been isolated from the rest of the cluster. As long as more than half of the (voting) replicas for each shard of a table are still available, the table will automatically regain availability by failing over to the server that are still available. The admin can then either wait for the missing servers to come back online; or add new replacement servers by using the `reconfigure()` command or by writing to `rethinkdb.table_config`. Unlike in RethinkDB 2.0, there is no need to \"permanently remove\" the old servers from the cluster; the old servers can return later with no difficulty. During this whole process, the consistency/linearizability guarantees are preserved and the table never loses availability for more than a short time.\r\n\r\nHowever, this all assumes that more than half of the voting replicas are available for each shard. If half or more of the voting replicas for a shard are lost, then the normal recovery methods will not work. The user has two options: They can simply wait for the missing servers to come back, or they can run the emergency repair command.\r\n\r\n##### Syntax\r\n\r\nThe syntax is as follows:\r\n```js\r\nr.table(...).reconfigure({emergencyRepair: \"unsafe_rollback\" [, dryRun: <bool>]})\r\nr.table(...).reconfigure({emergencyRepair: \"unsafe_rollback_or_erase\" [, dryRun: <bool>]})\r\n```\r\nEmergency repair is effectively a separate command from the normal `reconfigure()` command, even though it uses the same term. So if the `emergencyRepair` optarg is provided, the normal `reconfigure()` optargs like `replicas`, `shards`, etc. are not allowed.\r\n\r\nNote that it's illegal to run `r.db(...).reconfigure({emergencyRepair: ...})`; the user must run it on each table one at a time, instead of running it on the entire database at once.\r\n\r\nSpecifying `emergencyRepair: null` is equivalent to not specifying `emergencyRepair` at all. For example, `table.reconfigure({emergencyRepair: null, shards: 2, replicas: 2})` is a perfectly valid command that's equivalent to `table.reconfigure({shards: 2, replicas: 2})`. This is mostly just for consistency; we don't expect people to use it very often.\r\n\r\nThe return value has the same format as for a regular `reconfigure()` command.\r\n\r\n##### Behavior\r\n\r\nThe emergency repair command examines each shard of the table, and puts it into one of three categories:\r\n1. More than half of the shard's voting replicas are still available; the shard is healthy.\r\n2. The shard is missing half or more of its voting replicas, but there is still at least one voting or non-voting replica available. So the shard can be repaired.\r\n3. The shard is missing all of its voting and non-voting replicas, so it is beyond repair.\r\n\r\nIf all of the shards are healthy, the emergency repair command raises an error, reporting that the table doesn't need to be repaired.\r\n\r\nFor each shard that can be repaired, the emergency repair command forcibly converts any unavailable voting replicas into non-voting replicas. Then, if all of the voting replicas were removed, it forcibly converts an arbitrary available non-voting replica into a voting replica. Normally, it's impossible to change the set of voting replicas unless a quorum of voting replicas is available; but the emergency repair command bypasses this safeguard.\r\n\r\nIf the `emergencyRepair` parameter was set to `\"unsafe_rollback\"`, then any shards that were beyond repair will be left alone. If all of the shards are either healthy or beyond repair, so there is nothing to do, the command will raise an error. If the `emergencyRepair` parameter was set to `\"unsafe_rollback_or_erase\"`, then the shards that are beyond repair will be destroyed and re-created from scratch. Any data that was on those shards before will be permanently lost; but the shards will be available for writing again.\r\n\r\nIf `dryRun` is set to `true`, then the table will not actually be changed, but the emergency repair command will report what would have been done via the `config_changes` field of the return value.\r\n\r\n##### Warnings\r\n\r\nThe emergency repair command is extremely dangerous. It should only be used as a last resort. It works by bypassing the normal safeguards that prevent people from losing or corrupting their data. It invalidates the linearizability guarantees that RethinkDB normally provides. In `unsafe_rollback_or_erase` mode, it can easily lose large chunks of data. Even in `unsafe_rollback` mode it can lose data; for example, if it converts a non-voting replica to a voting replica, that replica might not have a complete copy of the table's data yet."}}