{"action":"created","issue":{"url":"https://api.github.com/repos/ztellman/aleph/issues/196","labels_url":"https://api.github.com/repos/ztellman/aleph/issues/196/labels{/name}","comments_url":"https://api.github.com/repos/ztellman/aleph/issues/196/comments","events_url":"https://api.github.com/repos/ztellman/aleph/issues/196/events","html_url":"https://github.com/ztellman/aleph/issues/196","id":120262007,"number":196,"title":"unicode, freebsd, openjdk8 and websockets: unicode chars being whacked?","user":{"login":"zentrope","id":95732,"avatar_url":"https://avatars.githubusercontent.com/u/95732?v=3","gravatar_id":"","url":"https://api.github.com/users/zentrope","html_url":"https://github.com/zentrope","followers_url":"https://api.github.com/users/zentrope/followers","following_url":"https://api.github.com/users/zentrope/following{/other_user}","gists_url":"https://api.github.com/users/zentrope/gists{/gist_id}","starred_url":"https://api.github.com/users/zentrope/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zentrope/subscriptions","organizations_url":"https://api.github.com/users/zentrope/orgs","repos_url":"https://api.github.com/users/zentrope/repos","events_url":"https://api.github.com/users/zentrope/events{/privacy}","received_events_url":"https://api.github.com/users/zentrope/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-12-03T20:26:04Z","updated_at":"2015-12-13T01:29:37Z","closed_at":"2015-12-13T01:29:37Z","body":"I'm using aleph's websocket-connection and transit and I have a question about the raw-stream? flag for the web-socket-connect. \r\n\r\nBut first, a weird thing:\r\n\r\nWhen I deploy the server on `FreeBSD` (using `OpenJDK8`), unicode characters such as ø typed into a web form, encoded via transit, sent via web socket, come out as \"?\" on the Aleph side even before I send the string to transit for decoding.\r\n\r\nIn other words, (stream/take! ws-stream), already produces the ? rather than the ø.\r\n\r\nThe byte sequence (-61 -72) becomes 63 (a \"?\") and eveyrthing else remains the same.\r\n\r\nEverything works fine when the server runs on OSX (or maybe it's Oracle's JDK8), remote or localhost.\r\n\r\nUsing EDN instead of transit works fine. Huh? Typing international characters into the web form, doing a (pr-str value) and sending that via the websocket, then taking it from the stream and applying clojure.edn/read-string works just fine.\r\n\r\nHow can that be? Either one of these things is just a raw string when on the web-socket wire!\r\n\r\nSO, what I tried to do is add :raw-stream? true to the websocket-connection-options in hopes that whatever manifold or netty or the JVM is doing is minimized, but I still get a string-type back out when I do a stream/take. How can I get at the raw bytes before the web-socket payload is converted to a string?\r\n\r\nMy set up:\r\n\r\n```clojure\r\n        options {:raw-stream? true\r\n                 :headers (-> (select-keys headers [\"sec-websocket-protocol\"\r\n                                                    \"sec-websocket-version\"])\r\n                              (assoc \"content-type\" \"application/octet-stream\"))}\r\n        stream @(http/websocket-connection req options)\r\n```\r\n\r\nUsed as:\r\n\r\n```clojure\r\n(decode @(stream/take! stream))\r\n```\r\n\r\nAnd decode:\r\n\r\n```clojure\r\n(defn- decode\r\n  [payload]\r\n  (println \"DECODE:\" (type payload) payload)\r\n  (when payload\r\n    (with-open [in (ByteArrayInputStream. (bs/to-byte-array payload))]\r\n      (let [reader (transit/reader in :json)]\r\n        (transit/read reader)))))\r\n```\r\n\r\nThe `(type payload)` is still a string and payload contains the \"?\" character before I ever send it to the transit stuff. I use the to-byte-array function because I expected payload to be some sort of stream (otherwise .getBytes works).\r\n\r\nAny tips for getting to the underlying byte buffers?\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/ztellman/aleph/issues/comments/164209034","html_url":"https://github.com/ztellman/aleph/issues/196#issuecomment-164209034","issue_url":"https://api.github.com/repos/ztellman/aleph/issues/196","id":164209034,"user":{"login":"zentrope","id":95732,"avatar_url":"https://avatars.githubusercontent.com/u/95732?v=3","gravatar_id":"","url":"https://api.github.com/users/zentrope","html_url":"https://github.com/zentrope","followers_url":"https://api.github.com/users/zentrope/followers","following_url":"https://api.github.com/users/zentrope/following{/other_user}","gists_url":"https://api.github.com/users/zentrope/gists{/gist_id}","starred_url":"https://api.github.com/users/zentrope/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zentrope/subscriptions","organizations_url":"https://api.github.com/users/zentrope/orgs","repos_url":"https://api.github.com/users/zentrope/repos","events_url":"https://api.github.com/users/zentrope/events{/privacy}","received_events_url":"https://api.github.com/users/zentrope/received_events","type":"User","site_admin":false},"created_at":"2015-12-13T01:29:37Z","updated_at":"2015-12-13T01:29:37Z","body":"Alas, I just don't have any time to figure this out and it's pointless that this remains an open issue here. Reverted to using Clojure EDN rather than transit and will investigate again if \"speed\" is of the essence."}}