{"action":"created","issue":{"url":"https://api.github.com/repos/gjtorikian/commonmarker/issues/9","labels_url":"https://api.github.com/repos/gjtorikian/commonmarker/issues/9/labels{/name}","comments_url":"https://api.github.com/repos/gjtorikian/commonmarker/issues/9/comments","events_url":"https://api.github.com/repos/gjtorikian/commonmarker/issues/9/events","html_url":"https://github.com/gjtorikian/commonmarker/issues/9","id":93406248,"number":9,"title":"Error \"incompatible character encodings: UTF-8 and ASCII-8BIT\" when combined with a rails app","user":{"login":"oboxodo","id":8324,"avatar_url":"https://avatars.githubusercontent.com/u/8324?v=3","gravatar_id":"","url":"https://api.github.com/users/oboxodo","html_url":"https://github.com/oboxodo","followers_url":"https://api.github.com/users/oboxodo/followers","following_url":"https://api.github.com/users/oboxodo/following{/other_user}","gists_url":"https://api.github.com/users/oboxodo/gists{/gist_id}","starred_url":"https://api.github.com/users/oboxodo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/oboxodo/subscriptions","organizations_url":"https://api.github.com/users/oboxodo/orgs","repos_url":"https://api.github.com/users/oboxodo/repos","events_url":"https://api.github.com/users/oboxodo/events{/privacy}","received_events_url":"https://api.github.com/users/oboxodo/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-07-07T00:18:44Z","updated_at":"2015-07-07T03:56:06Z","closed_at":null,"body":"I think this might not be a `commonmarker` problem, BUT the error is **not** raised when using `pandoc-ruby` nor `redcarpet`, so it has something to do with `commonmarker`.\r\n\r\nHere you can see a test run from the command line with both `cmark` and `commonmarker` and there's no problem:\r\n\r\n```\r\n$ cat test-curly-quotes.md\r\nThis curly quote “makes commonmarker throw an exception”.\r\n\r\n$ cmark --version\r\ncmark 0.20.0 - CommonMark converter\r\n(C) 2014, 2015 John MacFarlane\r\n\r\n$ cmark test-curly-quotes.md\r\n<p>This curly quote “makes commonmarker throw an exception”.</p>\r\n\r\n$ gem list --local commonmarker\r\n\r\n*** LOCAL GEMS ***\r\n\r\ncommonmarker (0.2.0)\r\n\r\n$ cat test-curly-quotes.md | ruby -r commonmarker -e \"puts CommonMarker.render_html(gets)\"\r\n<p>This curly quote “makes commonmarker throw an exception”.</p>\r\n```\r\n\r\nThat said, I'm testing different markdown parsers/renderers for our rails 4.1.12 (ruby 2.2.2) based app and I'm getting the following error:\r\n\r\n```\r\nActionView::Template::Error (incompatible character encodings: UTF-8 and ASCII-8BIT):\r\n    12:       - if user_signed_in?\r\n    13:         .outline-content\r\n    14:           = commonmarker_markdown(@quimbee_outline.source)\r\n  app/views/outlines/show.html.slim:15:in `_app_views_outlines_show_html_slim___3317075370232322437_70158621096300'\r\n\r\n\r\n  Rendered /Users/oboxodo/.rbenv/versions/2.2.2/lib/ruby/gems/2.2.0/gems/actionpack-4.1.12/lib/action_dispatch/middleware/templates/rescues/_trace.html.erb (2.9ms)\r\n  Rendered /Users/oboxodo/.rbenv/versions/2.2.2/lib/ruby/gems/2.2.0/gems/actionpack-4.1.12/lib/action_dispatch/middleware/templates/rescues/_request_and_response.html.erb (1.7ms)\r\n  Rendered /Users/oboxodo/.rbenv/versions/2.2.2/lib/ruby/gems/2.2.0/gems/actionpack-4.1.12/lib/action_dispatch/middleware/templates/rescues/template_error.html.erb within rescues/layout (69.1ms)\r\n```\r\n\r\nI have these helpers:\r\n\r\n```ruby\r\n# encoding: UTF-8\r\nmodule ApplicationHelper\r\n  def commonmarker_markdown(text)\r\n    CommonMarker.render_html(text, :smart).html_safe\r\n  end\r\n\r\n  def pandoc_markdown(text)\r\n    converter = PandocRuby.new(text, from: :markdown, to: :html)\r\n    converter.convert.html_safe\r\n  end\r\n\r\n  def redcarpet_markdown(text)\r\n    # ...\r\n  end\r\nend\r\n```\r\n\r\nChanging the call to `commonmarker_markdown` to either `pandoc_markdown` or `redcarpet_markdown` renders the expected result with no errors.\r\n\r\nIt's not a DB (postgresql) encoding problem either as hardcoding the test phrase in place of the `text` variable (no DB involved) causes the same problem.\r\n\r\nAny ideas about what could be happening?\r\n"},"comment":{"url":"https://api.github.com/repos/gjtorikian/commonmarker/issues/comments/119066314","html_url":"https://github.com/gjtorikian/commonmarker/issues/9#issuecomment-119066314","issue_url":"https://api.github.com/repos/gjtorikian/commonmarker/issues/9","id":119066314,"user":{"login":"gjtorikian","id":64050,"avatar_url":"https://avatars.githubusercontent.com/u/64050?v=3","gravatar_id":"","url":"https://api.github.com/users/gjtorikian","html_url":"https://github.com/gjtorikian","followers_url":"https://api.github.com/users/gjtorikian/followers","following_url":"https://api.github.com/users/gjtorikian/following{/other_user}","gists_url":"https://api.github.com/users/gjtorikian/gists{/gist_id}","starred_url":"https://api.github.com/users/gjtorikian/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gjtorikian/subscriptions","organizations_url":"https://api.github.com/users/gjtorikian/orgs","repos_url":"https://api.github.com/users/gjtorikian/repos","events_url":"https://api.github.com/users/gjtorikian/events{/privacy}","received_events_url":"https://api.github.com/users/gjtorikian/received_events","type":"User","site_admin":true},"created_at":"2015-07-07T03:56:06Z","updated_at":"2015-07-07T03:56:06Z","body":"> But that seems wrong/unfair/not the responsibility of the consumer.\r\n\r\nTo finish my thought: probably this library should do the `force_encoding`. Could you verify that `force_encoding` fixes the problem for you? If so I'll do a patch release for this."}}