{"action":"created","issue":{"url":"https://api.github.com/repos/neovim/neovim/issues/2905","labels_url":"https://api.github.com/repos/neovim/neovim/issues/2905/labels{/name}","comments_url":"https://api.github.com/repos/neovim/neovim/issues/2905/comments","events_url":"https://api.github.com/repos/neovim/neovim/issues/2905/events","html_url":"https://github.com/neovim/neovim/pull/2905","id":91233180,"number":2905,"title":"[WIP] Only allow encoding=utf-8 ?","user":{"login":"bfredl","id":1363104,"avatar_url":"https://avatars.githubusercontent.com/u/1363104?v=3","gravatar_id":"","url":"https://api.github.com/users/bfredl","html_url":"https://github.com/bfredl","followers_url":"https://api.github.com/users/bfredl/followers","following_url":"https://api.github.com/users/bfredl/following{/other_user}","gists_url":"https://api.github.com/users/bfredl/gists{/gist_id}","starred_url":"https://api.github.com/users/bfredl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bfredl/subscriptions","organizations_url":"https://api.github.com/users/bfredl/orgs","repos_url":"https://api.github.com/users/bfredl/repos","events_url":"https://api.github.com/users/bfredl/events{/privacy}","received_events_url":"https://api.github.com/users/bfredl/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/neovim/neovim/labels/decision","name":"decision","color":"006b75"},{"url":"https://api.github.com/repos/neovim/neovim/labels/encoding","name":"encoding","color":"c7def8"},{"url":"https://api.github.com/repos/neovim/neovim/labels/WIP","name":"WIP","color":"dfdfdf"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/neovim/neovim/milestones/10","html_url":"https://github.com/neovim/neovim/milestones/0.2","labels_url":"https://api.github.com/repos/neovim/neovim/milestones/10/labels","id":1183808,"number":10,"title":"0.2","description":"beta release, codename: \"keanu\"","creator":{"login":"justinmk","id":1359421,"avatar_url":"https://avatars.githubusercontent.com/u/1359421?v=3","gravatar_id":"","url":"https://api.github.com/users/justinmk","html_url":"https://github.com/justinmk","followers_url":"https://api.github.com/users/justinmk/followers","following_url":"https://api.github.com/users/justinmk/following{/other_user}","gists_url":"https://api.github.com/users/justinmk/gists{/gist_id}","starred_url":"https://api.github.com/users/justinmk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/justinmk/subscriptions","organizations_url":"https://api.github.com/users/justinmk/orgs","repos_url":"https://api.github.com/users/justinmk/repos","events_url":"https://api.github.com/users/justinmk/events{/privacy}","received_events_url":"https://api.github.com/users/justinmk/received_events","type":"User","site_admin":false},"open_issues":75,"closed_issues":5,"state":"open","created_at":"2015-06-27T19:05:24Z","updated_at":"2015-08-26T01:42:34Z","due_on":null,"closed_at":null},"comments":14,"created_at":"2015-06-26T11:58:11Z","updated_at":"2015-08-28T23:50:52Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/neovim/neovim/pulls/2905","html_url":"https://github.com/neovim/neovim/pull/2905","diff_url":"https://github.com/neovim/neovim/pull/2905.diff","patch_url":"https://github.com/neovim/neovim/pull/2905.patch"},"body":"During #2469 the possibilty to only allow `encoding=utf-8` was discussed, which would enable simplifications both for nvim core, and for guis/plugins, which then always can assume the encoding to be utf-8.\r\n\r\n`&encoding` has several effects, like being the default for `setglobal fileencoding` (encoding of new files), and in nvim the TUI (currently) always uses it. The internal effect is that switches between three different modes in which text processing is done: 1-byte \"latin\", 2-byte \"dbcs\", and unicode \"utf8\", by setting flags and reassigning a number of function pointers. A quick grep shows that these flags (`enc_utf8|has_mbyte|enc_dbcs|enc_latin1like`) are used in 600 places throughout the codebase. (Though it would be unrealistic to remove them all in one PR, and if done it should be done with some care, as they do serve as documentation for what assumtions the code is relying on)\r\n\r\nAFAIK nowadays almost everyone is using `encoding=utf-8` and I believe that any text processing in latin and dbcs mode could just as well be done by converting to utf-8 and using utf-8 mode, and whatever performance difference  should be pretty small in the large context. Please correct me if this isn't true.\r\n\r\nWhat this PR does so far is always set `p_enc` to \"utf-8\" at start up. `set encoding=utf-8` (and synonyms) are allowed (as it is very common in vimrc:s), but any other value is an error. I've started cleaning up `mbyte.c`.\r\n\r\nWhat might should be done is use `enc_locale()` as a default for `setglobal fenc`, to preserve the effect locale has on encoding for new files. Or perhaps just leave the default with `utf-8`, not sure what's best.\r\nPerhaps also should implement conversion from/to locale encoding (if not utf-8) in the TUI. This might not be widely used, but it should not be too much work to support at least 1-byte encodings. (no buffering of leftover bytes and such).\r\n\r\nThere is more cleanup to be done (not to mention documentation) though it would be best to discuss if we really want to do this before doing more mork.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/neovim/neovim/issues/comments/135916428","html_url":"https://github.com/neovim/neovim/pull/2905#issuecomment-135916428","issue_url":"https://api.github.com/repos/neovim/neovim/issues/2905","id":135916428,"user":{"login":"justinmk","id":1359421,"avatar_url":"https://avatars.githubusercontent.com/u/1359421?v=3","gravatar_id":"","url":"https://api.github.com/users/justinmk","html_url":"https://github.com/justinmk","followers_url":"https://api.github.com/users/justinmk/followers","following_url":"https://api.github.com/users/justinmk/following{/other_user}","gists_url":"https://api.github.com/users/justinmk/gists{/gist_id}","starred_url":"https://api.github.com/users/justinmk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/justinmk/subscriptions","organizations_url":"https://api.github.com/users/justinmk/orgs","repos_url":"https://api.github.com/users/justinmk/repos","events_url":"https://api.github.com/users/justinmk/events{/privacy}","received_events_url":"https://api.github.com/users/justinmk/received_events","type":"User","site_admin":false},"created_at":"2015-08-28T23:50:52Z","updated_at":"2015-08-28T23:50:52Z","body":"@bfredl I am speaking specifically of this:\r\n\r\n> If there's no results, then you might not be using a UTF-8 locale\r\n\r\nMy thought was: user's locale should not matter since we would no longer initialize `'encoding'` based on the locale. \r\n\r\n>  It should at least eliminate the need for the workarounds scriptencoding utf-8 in vimrc and override LC_ALL in a buggy locale environment\r\n\r\nThose are good things to note."}}