{"action":"created","comment":{"url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/pulls/comments/25644903","id":25644903,"diff_hunk":"@@ -1 +1,102 @@\n-Placeholder for network protocol.\n+Status: this is a rough draft, intended to get some ideas out there. \n+\n+Missing: how to run on top of HTTP/2. Possibly the structure or semantics of publisher name strings.\n+Might be removed: extension support.\n+Need to choose serialization method (protobuf, msgpack, other?)\n+\n+## Transport assumptions\n+\n+A supporting transport must be similar to a streaming socket:\n+\n+1. Bidirectional and full-duplex\n+2. An octet stream, i.e. all octet values may be sent unencoded\n+3. Ordered delivery: an implementation may map protocol messages to some features of the underlying transport (e.g. [Ømq](http://zeromq.org/) messages), but the messages must arrive in the same order as they were sent\n+\n+Definitely supported transports include TCP, TLS (over TCP), WebSockets, and most socket-like objects (e.g. pipes). HTTP/2 will be supported, but may require a dedicated specification for implementing this protocol.\n+\n+## Message framing\n+\n+An existing serialization format should be used. Current candidates are [Protocol Buffers](https://github.com/google/protobuf/) (which is slightly less space efficient), [MessagePack](http://msgpack.org/) (whose Scala implementation may not be as good as the others), and possibly [Thrift](https://thrift.apache.org/) (with which I'm less familiar). \n+\n+The serialization format should be fast and space-efficient. It does not need to be self-describing, since the message types and their structures are fully specified in the protocol. It needs to have the types boolean, string / byte array (length-prefixed), and varint (an integer encoded using 1 or more bytes depending on its value).\n+\n+The full complexity of these formats may not be needed today, but future protocol extensions might benefit. Also, an implementation might encode the published elements using the same format and decode both the framing and the messages using the same parser.\n+\n+Each message (frame) begins with a message type, which is a varint, followed by its contents. Messages are self-delimiting, because their structure is known from their type, and all fields are either of fixed size, self-delimited varints, or length-prefixed strings or byte arrays.","path":"NETWORK_PROTOCOL.md","position":26,"original_position":26,"commit_id":"aa9c93564cc71360f1df0c62999a28dbabc1b97c","original_commit_id":"aa9c93564cc71360f1df0c62999a28dbabc1b97c","user":{"login":"danarmak","id":527616,"avatar_url":"https://avatars.githubusercontent.com/u/527616?v=3","gravatar_id":"","url":"https://api.github.com/users/danarmak","html_url":"https://github.com/danarmak","followers_url":"https://api.github.com/users/danarmak/followers","following_url":"https://api.github.com/users/danarmak/following{/other_user}","gists_url":"https://api.github.com/users/danarmak/gists{/gist_id}","starred_url":"https://api.github.com/users/danarmak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/danarmak/subscriptions","organizations_url":"https://api.github.com/users/danarmak/orgs","repos_url":"https://api.github.com/users/danarmak/repos","events_url":"https://api.github.com/users/danarmak/events{/privacy}","received_events_url":"https://api.github.com/users/danarmak/received_events","type":"User","site_admin":false},"body":"@experquisite The message type could be a single byte. Even if we run out of values, types could be defined where the second byte onwards specifies a subtype. I think this is probably a good change to make regardless of varint parsing efficiency, and I'll make it.\r\n\r\nThe varints used in length-prefixed arrays could be replaced with regular 32bit ints. This would use on average 2, maybe 3 more bytes per message. My intuition was to optimize for size. But I really don't know if there would be a noticeable performance hit.\r\n\r\nIf you have the time, you can run a performance test scenario and find out...","created_at":"2015-03-02T22:24:16Z","updated_at":"2015-03-02T22:24:16Z","html_url":"https://github.com/reactive-streams/reactive-streams-io/pull/4#discussion_r25644903","pull_request_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/pulls/4","_links":{"self":{"href":"https://api.github.com/repos/reactive-streams/reactive-streams-io/pulls/comments/25644903"},"html":{"href":"https://github.com/reactive-streams/reactive-streams-io/pull/4#discussion_r25644903"},"pull_request":{"href":"https://api.github.com/repos/reactive-streams/reactive-streams-io/pulls/4"}}},"pull_request":{"url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/pulls/4","id":30129888,"html_url":"https://github.com/reactive-streams/reactive-streams-io/pull/4","diff_url":"https://github.com/reactive-streams/reactive-streams-io/pull/4.diff","patch_url":"https://github.com/reactive-streams/reactive-streams-io/pull/4.patch","issue_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/issues/4","number":4,"state":"open","locked":false,"title":"Protocol draft - for discussion only","user":{"login":"danarmak","id":527616,"avatar_url":"https://avatars.githubusercontent.com/u/527616?v=3","gravatar_id":"","url":"https://api.github.com/users/danarmak","html_url":"https://github.com/danarmak","followers_url":"https://api.github.com/users/danarmak/followers","following_url":"https://api.github.com/users/danarmak/following{/other_user}","gists_url":"https://api.github.com/users/danarmak/gists{/gist_id}","starred_url":"https://api.github.com/users/danarmak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/danarmak/subscriptions","organizations_url":"https://api.github.com/users/danarmak/orgs","repos_url":"https://api.github.com/users/danarmak/repos","events_url":"https://api.github.com/users/danarmak/events{/privacy}","received_events_url":"https://api.github.com/users/danarmak/received_events","type":"User","site_admin":false},"body":"I found it interesting to try to write a network protocol specification. This is a first draft and it might well be completely unsuitable. I'm submitting the PR only to generate discussion, not for merging.","created_at":"2015-02-27T00:00:53Z","updated_at":"2015-03-02T22:24:16Z","closed_at":null,"merged_at":null,"merge_commit_sha":"8dcc8f3a0fb8024818d7949dcd99e9e6446b940b","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/pulls/4/commits","review_comments_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/pulls/4/comments","review_comment_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/pulls/comments{/number}","comments_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/issues/4/comments","statuses_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/statuses/aa9c93564cc71360f1df0c62999a28dbabc1b97c","head":{"label":"danarmak:protocol-draft","ref":"protocol-draft","sha":"aa9c93564cc71360f1df0c62999a28dbabc1b97c","user":{"login":"danarmak","id":527616,"avatar_url":"https://avatars.githubusercontent.com/u/527616?v=3","gravatar_id":"","url":"https://api.github.com/users/danarmak","html_url":"https://github.com/danarmak","followers_url":"https://api.github.com/users/danarmak/followers","following_url":"https://api.github.com/users/danarmak/following{/other_user}","gists_url":"https://api.github.com/users/danarmak/gists{/gist_id}","starred_url":"https://api.github.com/users/danarmak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/danarmak/subscriptions","organizations_url":"https://api.github.com/users/danarmak/orgs","repos_url":"https://api.github.com/users/danarmak/repos","events_url":"https://api.github.com/users/danarmak/events{/privacy}","received_events_url":"https://api.github.com/users/danarmak/received_events","type":"User","site_admin":false},"repo":{"id":31387657,"name":"reactive-streams-io","full_name":"danarmak/reactive-streams-io","owner":{"login":"danarmak","id":527616,"avatar_url":"https://avatars.githubusercontent.com/u/527616?v=3","gravatar_id":"","url":"https://api.github.com/users/danarmak","html_url":"https://github.com/danarmak","followers_url":"https://api.github.com/users/danarmak/followers","following_url":"https://api.github.com/users/danarmak/following{/other_user}","gists_url":"https://api.github.com/users/danarmak/gists{/gist_id}","starred_url":"https://api.github.com/users/danarmak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/danarmak/subscriptions","organizations_url":"https://api.github.com/users/danarmak/orgs","repos_url":"https://api.github.com/users/danarmak/repos","events_url":"https://api.github.com/users/danarmak/events{/privacy}","received_events_url":"https://api.github.com/users/danarmak/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/danarmak/reactive-streams-io","description":"Reactive Streams standardization for network protocols","fork":true,"url":"https://api.github.com/repos/danarmak/reactive-streams-io","forks_url":"https://api.github.com/repos/danarmak/reactive-streams-io/forks","keys_url":"https://api.github.com/repos/danarmak/reactive-streams-io/keys{/key_id}","collaborators_url":"https://api.github.com/repos/danarmak/reactive-streams-io/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/danarmak/reactive-streams-io/teams","hooks_url":"https://api.github.com/repos/danarmak/reactive-streams-io/hooks","issue_events_url":"https://api.github.com/repos/danarmak/reactive-streams-io/issues/events{/number}","events_url":"https://api.github.com/repos/danarmak/reactive-streams-io/events","assignees_url":"https://api.github.com/repos/danarmak/reactive-streams-io/assignees{/user}","branches_url":"https://api.github.com/repos/danarmak/reactive-streams-io/branches{/branch}","tags_url":"https://api.github.com/repos/danarmak/reactive-streams-io/tags","blobs_url":"https://api.github.com/repos/danarmak/reactive-streams-io/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/danarmak/reactive-streams-io/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/danarmak/reactive-streams-io/git/refs{/sha}","trees_url":"https://api.github.com/repos/danarmak/reactive-streams-io/git/trees{/sha}","statuses_url":"https://api.github.com/repos/danarmak/reactive-streams-io/statuses/{sha}","languages_url":"https://api.github.com/repos/danarmak/reactive-streams-io/languages","stargazers_url":"https://api.github.com/repos/danarmak/reactive-streams-io/stargazers","contributors_url":"https://api.github.com/repos/danarmak/reactive-streams-io/contributors","subscribers_url":"https://api.github.com/repos/danarmak/reactive-streams-io/subscribers","subscription_url":"https://api.github.com/repos/danarmak/reactive-streams-io/subscription","commits_url":"https://api.github.com/repos/danarmak/reactive-streams-io/commits{/sha}","git_commits_url":"https://api.github.com/repos/danarmak/reactive-streams-io/git/commits{/sha}","comments_url":"https://api.github.com/repos/danarmak/reactive-streams-io/comments{/number}","issue_comment_url":"https://api.github.com/repos/danarmak/reactive-streams-io/issues/comments{/number}","contents_url":"https://api.github.com/repos/danarmak/reactive-streams-io/contents/{+path}","compare_url":"https://api.github.com/repos/danarmak/reactive-streams-io/compare/{base}...{head}","merges_url":"https://api.github.com/repos/danarmak/reactive-streams-io/merges","archive_url":"https://api.github.com/repos/danarmak/reactive-streams-io/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/danarmak/reactive-streams-io/downloads","issues_url":"https://api.github.com/repos/danarmak/reactive-streams-io/issues{/number}","pulls_url":"https://api.github.com/repos/danarmak/reactive-streams-io/pulls{/number}","milestones_url":"https://api.github.com/repos/danarmak/reactive-streams-io/milestones{/number}","notifications_url":"https://api.github.com/repos/danarmak/reactive-streams-io/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/danarmak/reactive-streams-io/labels{/name}","releases_url":"https://api.github.com/repos/danarmak/reactive-streams-io/releases{/id}","created_at":"2015-02-26T20:38:44Z","updated_at":"2015-02-26T19:29:54Z","pushed_at":"2015-02-27T16:12:38Z","git_url":"git://github.com/danarmak/reactive-streams-io.git","ssh_url":"git@github.com:danarmak/reactive-streams-io.git","clone_url":"https://github.com/danarmak/reactive-streams-io.git","svn_url":"https://github.com/danarmak/reactive-streams-io","homepage":null,"size":0,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"reactive-streams:master","ref":"master","sha":"0a0362ea2296ef8fb138f1300bcc1d933c71952b","user":{"login":"reactive-streams","id":7080130,"avatar_url":"https://avatars.githubusercontent.com/u/7080130?v=3","gravatar_id":"","url":"https://api.github.com/users/reactive-streams","html_url":"https://github.com/reactive-streams","followers_url":"https://api.github.com/users/reactive-streams/followers","following_url":"https://api.github.com/users/reactive-streams/following{/other_user}","gists_url":"https://api.github.com/users/reactive-streams/gists{/gist_id}","starred_url":"https://api.github.com/users/reactive-streams/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/reactive-streams/subscriptions","organizations_url":"https://api.github.com/users/reactive-streams/orgs","repos_url":"https://api.github.com/users/reactive-streams/repos","events_url":"https://api.github.com/users/reactive-streams/events{/privacy}","received_events_url":"https://api.github.com/users/reactive-streams/received_events","type":"Organization","site_admin":false},"repo":{"id":31283667,"name":"reactive-streams-io","full_name":"reactive-streams/reactive-streams-io","owner":{"login":"reactive-streams","id":7080130,"avatar_url":"https://avatars.githubusercontent.com/u/7080130?v=3","gravatar_id":"","url":"https://api.github.com/users/reactive-streams","html_url":"https://github.com/reactive-streams","followers_url":"https://api.github.com/users/reactive-streams/followers","following_url":"https://api.github.com/users/reactive-streams/following{/other_user}","gists_url":"https://api.github.com/users/reactive-streams/gists{/gist_id}","starred_url":"https://api.github.com/users/reactive-streams/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/reactive-streams/subscriptions","organizations_url":"https://api.github.com/users/reactive-streams/orgs","repos_url":"https://api.github.com/users/reactive-streams/repos","events_url":"https://api.github.com/users/reactive-streams/events{/privacy}","received_events_url":"https://api.github.com/users/reactive-streams/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/reactive-streams/reactive-streams-io","description":"Reactive Streams Network Protocol","fork":false,"url":"https://api.github.com/repos/reactive-streams/reactive-streams-io","forks_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/forks","keys_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/keys{/key_id}","collaborators_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/teams","hooks_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/hooks","issue_events_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/issues/events{/number}","events_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/events","assignees_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/assignees{/user}","branches_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/branches{/branch}","tags_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/tags","blobs_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/git/refs{/sha}","trees_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/git/trees{/sha}","statuses_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/statuses/{sha}","languages_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/languages","stargazers_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/stargazers","contributors_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/contributors","subscribers_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/subscribers","subscription_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/subscription","commits_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/commits{/sha}","git_commits_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/git/commits{/sha}","comments_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/comments{/number}","issue_comment_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/issues/comments{/number}","contents_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/contents/{+path}","compare_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/compare/{base}...{head}","merges_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/merges","archive_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/downloads","issues_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/issues{/number}","pulls_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/pulls{/number}","milestones_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/milestones{/number}","notifications_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/labels{/name}","releases_url":"https://api.github.com/repos/reactive-streams/reactive-streams-io/releases{/id}","created_at":"2015-02-24T21:47:12Z","updated_at":"2015-03-02T19:04:33Z","pushed_at":"2015-02-26T20:38:57Z","git_url":"git://github.com/reactive-streams/reactive-streams-io.git","ssh_url":"git@github.com:reactive-streams/reactive-streams-io.git","clone_url":"https://github.com/reactive-streams/reactive-streams-io.git","svn_url":"https://github.com/reactive-streams/reactive-streams-io","homepage":"reactive-streams.io","size":0,"stargazers_count":19,"watchers_count":19,"language":null,"has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":4,"mirror_url":null,"open_issues_count":7,"forks":4,"open_issues":7,"watchers":19,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/reactive-streams/reactive-streams-io/pulls/4"},"html":{"href":"https://github.com/reactive-streams/reactive-streams-io/pull/4"},"issue":{"href":"https://api.github.com/repos/reactive-streams/reactive-streams-io/issues/4"},"comments":{"href":"https://api.github.com/repos/reactive-streams/reactive-streams-io/issues/4/comments"},"review_comments":{"href":"https://api.github.com/repos/reactive-streams/reactive-streams-io/pulls/4/comments"},"review_comment":{"href":"https://api.github.com/repos/reactive-streams/reactive-streams-io/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/reactive-streams/reactive-streams-io/pulls/4/commits"},"statuses":{"href":"https://api.github.com/repos/reactive-streams/reactive-streams-io/statuses/aa9c93564cc71360f1df0c62999a28dbabc1b97c"}}}}