{"action":"created","issue":{"url":"https://api.github.com/repos/YorickPeterse/oga/issues/90","labels_url":"https://api.github.com/repos/YorickPeterse/oga/issues/90/labels{/name}","comments_url":"https://api.github.com/repos/YorickPeterse/oga/issues/90/comments","events_url":"https://api.github.com/repos/YorickPeterse/oga/issues/90/events","html_url":"https://github.com/YorickPeterse/oga/issues/90","id":66361144,"number":90,"title":"What is &nbsp; replaced with?","user":{"login":"lham","id":3420315,"avatar_url":"https://avatars.githubusercontent.com/u/3420315?v=3","gravatar_id":"","url":"https://api.github.com/users/lham","html_url":"https://github.com/lham","followers_url":"https://api.github.com/users/lham/followers","following_url":"https://api.github.com/users/lham/following{/other_user}","gists_url":"https://api.github.com/users/lham/gists{/gist_id}","starred_url":"https://api.github.com/users/lham/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lham/subscriptions","organizations_url":"https://api.github.com/users/lham/orgs","repos_url":"https://api.github.com/users/lham/repos","events_url":"https://api.github.com/users/lham/events{/privacy}","received_events_url":"https://api.github.com/users/lham/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2015-04-04T20:10:59Z","updated_at":"2015-04-05T21:13:30Z","closed_at":"2015-04-04T20:29:58Z","body":"I'm running into a weird problem when parsing elements containing text followed by some `&nbsp;` within my rails project. If I run \r\n\r\n    require 'oga'\r\n    \r\n    doc = Oga.parse_html('<p>hello&nbsp;&nbsp;</p>');\r\n    str = doc.css('p').text;\r\n    \r\n    puts str.inspect\r\n    puts str.strip.inspect\r\n    puts str[/^\\s*(.*?)\\s*$/, 1].inspect\r\n\r\nit gives me the following output\r\n\r\n    \"hello  \"\r\n    \"hello  \"\r\n    \"hello  \"\r\n\r\nFor some reason I'm not able to remove the trailing whitespace using `strip()` or regex. To me it seems like `&nbsp;` is being replaced by something else than whitespace. Not sure if this is due to the gem or if I'm doing something else wrong here..."},"comment":{"url":"https://api.github.com/repos/YorickPeterse/oga/issues/comments/89853246","html_url":"https://github.com/YorickPeterse/oga/issues/90#issuecomment-89853246","issue_url":"https://api.github.com/repos/YorickPeterse/oga/issues/90","id":89853246,"user":{"login":"YorickPeterse","id":86065,"avatar_url":"https://avatars.githubusercontent.com/u/86065?v=3","gravatar_id":"","url":"https://api.github.com/users/YorickPeterse","html_url":"https://github.com/YorickPeterse","followers_url":"https://api.github.com/users/YorickPeterse/followers","following_url":"https://api.github.com/users/YorickPeterse/following{/other_user}","gists_url":"https://api.github.com/users/YorickPeterse/gists{/gist_id}","starred_url":"https://api.github.com/users/YorickPeterse/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/YorickPeterse/subscriptions","organizations_url":"https://api.github.com/users/YorickPeterse/orgs","repos_url":"https://api.github.com/users/YorickPeterse/repos","events_url":"https://api.github.com/users/YorickPeterse/events{/privacy}","received_events_url":"https://api.github.com/users/YorickPeterse/received_events","type":"User","site_admin":false},"created_at":"2015-04-05T21:13:30Z","updated_at":"2015-04-05T21:13:30Z","body":"> Also the decoders behave slightly differently, so I assumed there would be two encoders - probably doesn't matter but from a usability perspective it makes sense I think:\r\n\r\nIn the first line you're decoding a non existing XML entity, in the second line you're decoding an HTML entity. The entity `&nbsp;` is only defined in HTML, not XML, thus you only get an unbreakable space when using `Oga::HTML::Entities`. There's currently no `Oga::HTML::Entities.encode` because Oga itself has no use case for such a method.\r\n\r\n> Note these two are different, the latter is the correct conversion for &cups;. You can't see the difference because I turned off ASCII encoding...\r\n\r\nThe difference you're seeing here is because while Oga uses `[8746, 65024].pack('U')` this doesn't actually produce a string that uses the exact same input codepoints:\r\n\r\n    [8746, 65024].pack('U').codepoints # => [8746]\r\n\r\nAs a result `Oga::HTML::Entities::DECODE_MAPPING['&cups;']` will not be the same as `\"\\u222A\\uFE00\"`. I'm not sure why this is the case, but it's not something I can do much about other than replacing all `Array#pack` calls (which I'd rather not do unless absolutely necessary)."}}