{"action":"created","issue":{"url":"https://api.github.com/repos/msgpack/msgpack-python/issues/162","labels_url":"https://api.github.com/repos/msgpack/msgpack-python/issues/162/labels{/name}","comments_url":"https://api.github.com/repos/msgpack/msgpack-python/issues/162/comments","events_url":"https://api.github.com/repos/msgpack/msgpack-python/issues/162/events","html_url":"https://github.com/msgpack/msgpack-python/issues/162","id":120820512,"number":162,"title":"String serialization not always using smallest representation","user":{"login":"kyleburnett","id":6369365,"avatar_url":"https://avatars.githubusercontent.com/u/6369365?v=3","gravatar_id":"","url":"https://api.github.com/users/kyleburnett","html_url":"https://github.com/kyleburnett","followers_url":"https://api.github.com/users/kyleburnett/followers","following_url":"https://api.github.com/users/kyleburnett/following{/other_user}","gists_url":"https://api.github.com/users/kyleburnett/gists{/gist_id}","starred_url":"https://api.github.com/users/kyleburnett/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kyleburnett/subscriptions","organizations_url":"https://api.github.com/users/kyleburnett/orgs","repos_url":"https://api.github.com/users/kyleburnett/repos","events_url":"https://api.github.com/users/kyleburnett/events{/privacy}","received_events_url":"https://api.github.com/users/kyleburnett/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-12-07T17:01:07Z","updated_at":"2015-12-08T12:46:38Z","closed_at":null,"body":"According to the [MessagePack spec](https://github.com/msgpack/msgpack/blob/master/spec.md#serialization-type-to-format-conversion):\r\n\r\n> If an object can be represented in multiple possible output formats, serializers SHOULD use the format which represents the data in the smallest number of bytes.\r\n\r\nStrings of length 32 to 255 should be encoded using the `str 8` format, but this library encodes them using `str 16`.\r\n\r\nHere's the script I was running to test encodings at the various string format thresholds using Python 2.7.6:\r\n\r\n```\r\nimport msgpack\r\n\r\ndef output_bytes(l):\r\n    packed = msgpack.packb(l)\r\n    print \"Buffer size:\", len(packed)\r\n    arr = [p.encode('hex') for p in packed]\r\n    if len(arr) < 10:\r\n        print ' '.join(arr)\r\n    else:\r\n        arr = arr[:10]\r\n        print ' '.join(arr), '...'\r\n\r\n############################################################\r\n# Testing string thresholds\r\n############################################################\r\n\r\nsingle = \"x\"\r\n\r\n# Fix strings\r\n\r\ndata = single * 1\r\noutput_bytes(data)\r\nprint \"Header should be 0xa1\"\r\n\r\ndata = single * 31\r\noutput_bytes(data)\r\nprint \"Header should be 0xbf\"\r\n\r\n# Str 8\r\n\r\ndata = single * 32\r\noutput_bytes(data)\r\nprint \"Header should be 0xd9\"\r\n\r\ndata = single * 255\r\noutput_bytes(data)\r\nprint \"Header should be 0xd9\"\r\n\r\n# Str 16\r\n\r\ndata = single * 256\r\noutput_bytes(data)\r\nprint \"Header should be 0xda\"\r\n\r\ndata = single * 65535\r\noutput_bytes(data)\r\nprint \"Header should be 0xda\"\r\n\r\n# Str 32\r\n\r\ndata = single * 65536\r\noutput_bytes(data)\r\nprint \"Header should be 0xdb\"\r\n```\r\n\r\nThis outputs the following:\r\n\r\n```\r\nBuffer size: 2\r\na1 78\r\nHeader should be 0xa1\r\nBuffer size: 32\r\nbf 78 78 78 78 78 78 78 78 78 ...\r\nHeader should be 0xbf\r\nBuffer size: 35\r\nda 00 20 78 78 78 78 78 78 78 ...\r\nHeader should be 0xd9\r\nBuffer size: 258\r\nda 00 ff 78 78 78 78 78 78 78 ...\r\nHeader should be 0xd9\r\nBuffer size: 259\r\nda 01 00 78 78 78 78 78 78 78 ...\r\nHeader should be 0xda\r\nBuffer size: 65538\r\nda ff ff 78 78 78 78 78 78 78 ...\r\nHeader should be 0xda\r\nBuffer size: 65541\r\ndb 00 01 00 00 78 78 78 78 78 ...\r\nHeader should be 0xdb\r\n```"},"comment":{"url":"https://api.github.com/repos/msgpack/msgpack-python/issues/comments/162869495","html_url":"https://github.com/msgpack/msgpack-python/issues/162#issuecomment-162869495","issue_url":"https://api.github.com/repos/msgpack/msgpack-python/issues/162","id":162869495,"user":{"login":"methane","id":199592,"avatar_url":"https://avatars.githubusercontent.com/u/199592?v=3","gravatar_id":"","url":"https://api.github.com/users/methane","html_url":"https://github.com/methane","followers_url":"https://api.github.com/users/methane/followers","following_url":"https://api.github.com/users/methane/following{/other_user}","gists_url":"https://api.github.com/users/methane/gists{/gist_id}","starred_url":"https://api.github.com/users/methane/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/methane/subscriptions","organizations_url":"https://api.github.com/users/methane/orgs","repos_url":"https://api.github.com/users/methane/repos","events_url":"https://api.github.com/users/methane/events{/privacy}","received_events_url":"https://api.github.com/users/methane/received_events","type":"User","site_admin":false},"created_at":"2015-12-08T12:46:38Z","updated_at":"2015-12-08T12:46:38Z","body":"I know it and it's intended behavior.\r\nAt first, there are no bin types and str8 type. They were introduced later.\r\nUsing bin types and str8 type cause compatibility problem with old decoders.\r\n\r\nstr8 is used when you use `use_bin_type=True` option and encoding unicode string."}}