{"action":"created","issue":{"url":"https://api.github.com/repos/ashtuchkin/iconv-lite/issues/97","labels_url":"https://api.github.com/repos/ashtuchkin/iconv-lite/issues/97/labels{/name}","comments_url":"https://api.github.com/repos/ashtuchkin/iconv-lite/issues/97/comments","events_url":"https://api.github.com/repos/ashtuchkin/iconv-lite/issues/97/events","html_url":"https://github.com/ashtuchkin/iconv-lite/issues/97","id":78904194,"number":97,"title":"Handle BOMs properly when changing encoding","user":{"login":"bpasero","id":900690,"avatar_url":"https://avatars.githubusercontent.com/u/900690?v=3","gravatar_id":"","url":"https://api.github.com/users/bpasero","html_url":"https://github.com/bpasero","followers_url":"https://api.github.com/users/bpasero/followers","following_url":"https://api.github.com/users/bpasero/following{/other_user}","gists_url":"https://api.github.com/users/bpasero/gists{/gist_id}","starred_url":"https://api.github.com/users/bpasero/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bpasero/subscriptions","organizations_url":"https://api.github.com/users/bpasero/orgs","repos_url":"https://api.github.com/users/bpasero/repos","events_url":"https://api.github.com/users/bpasero/events{/privacy}","received_events_url":"https://api.github.com/users/bpasero/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-05-21T09:11:39Z","updated_at":"2015-05-22T06:34:56Z","closed_at":null,"body":"First of all, kudos for this lib, its very nice!\r\n\r\nI think handling of BOMs should be part of the library though. Imagine you have a file with UTF-8 BOM and you save it as cp1252, the BOM should be removed automatically. In addition, if a BOM is found (UTF 8, UTF 16 LE or BE) and the file is saved with one of those encodings, the BOM should be changed to the proper one. In other words:\r\n\r\n- if a file has no BOM, do nothing special\r\n- if a file has a BOM, remove it if the encoding is !== UTF 8, 16 LE, 16 BE\r\n- if a file has a BOM that is !== the encoding of [UTF 8, 16 LE, 16 BE] change it accordingly\r\n- if a file has a BOM and the same encoding is used, keep the BOM"},"comment":{"url":"https://api.github.com/repos/ashtuchkin/iconv-lite/issues/comments/104537421","html_url":"https://github.com/ashtuchkin/iconv-lite/issues/97#issuecomment-104537421","issue_url":"https://api.github.com/repos/ashtuchkin/iconv-lite/issues/97","id":104537421,"user":{"login":"ashtuchkin","id":627997,"avatar_url":"https://avatars.githubusercontent.com/u/627997?v=3","gravatar_id":"","url":"https://api.github.com/users/ashtuchkin","html_url":"https://github.com/ashtuchkin","followers_url":"https://api.github.com/users/ashtuchkin/followers","following_url":"https://api.github.com/users/ashtuchkin/following{/other_user}","gists_url":"https://api.github.com/users/ashtuchkin/gists{/gist_id}","starred_url":"https://api.github.com/users/ashtuchkin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ashtuchkin/subscriptions","organizations_url":"https://api.github.com/users/ashtuchkin/orgs","repos_url":"https://api.github.com/users/ashtuchkin/repos","events_url":"https://api.github.com/users/ashtuchkin/events{/privacy}","received_events_url":"https://api.github.com/users/ashtuchkin/received_events","type":"User","site_admin":false},"created_at":"2015-05-22T06:34:56Z","updated_at":"2015-05-22T06:34:56Z","body":"Thanks, that makes sense. So basically decode will always remove BOM and all decoded strings would not have it, plus encode would add BOM if required by encoding name, right? I'm also thinking about adding a flag instead of adding encoding names, what do you think about that?\r\n\r\nAlso, can you describe your use case a bit more, so that I could understand context, if it's not secret of course? Are you working on a text editor?"}}