{"action":"created","comment":{"url":"https://api.github.com/repos/WeAreWizards/blog/pulls/comments/30905917","id":30905917,"diff_hunk":"@@ -0,0 +1,107 @@\n+Title: Using Protobuf instead of JSON to communicate with a frontend\n+Date: 2015-05-22\n+Short_summary: Protobuf (or others tools like Thrift) are very often used to communicate between services. Let's see how it would work to use it between a server and a javascript frontend\n+Category: Dev\n+Authors: Vincent\n+\n+*Reading time: ~15 minutes.*  \n+\n+Protobuf (or other binary serialization formats like Thrift) are widely used to communicate between services. JSON is overwhelmingly used for backend <-> frontend communication and for APIs.  \n+Let's see how a client/server using Protobuf would look like.\n+<!-- PELICAN_END_SUMMARY --> \n+\n+## What is Protobuf\n+[Protobuf](https://developers.google.com/protocol-buffers/docs/overview) is a binary serialization format from Google meant to serialize structured data. It has clients in most languages, including Python and Javascript which we are going to use in our toy app.  \n+\n+Tools like Protobuf have a few advantages over JSON:\n+\n+- smaller in size\n+- typed\n+- offer a common interface for all your services, you can just update your .proto or .thrift and share those with the services using it, as long as there is a client for that language\n+\n+[Thrift](https://thrift.apache.org/) does more as it also allows you to define methods as well as structs.\n+\n+## Example app\n+As an example we are going to make a small address book app. Coincidentally that's the example that the Python Protobuf doc is using as well but that wasn't on purpose I swear! The data is generated randomly on startup of the flask app.  \n+The repo is on our [github](https://github.com/WeAreWizards/protojson) if you want to try it yourself.  \n+As mentioned on the README, the Python client for Protobuf does not support Python3 (yet) so you will have to make a Python2 virtualenv if you decide to install the server.  \n+The server will deliver the content in JSON or Protobuf based on the `Accept` header.\n+\n+### Server-side\n+Let's have a quick look at the server, it might not be new if you have already used Protobuf before but it is helpful for the rest (like me, I only used Thrift before).  \n+First we need to define our Protobuf schema, which is done in the [addressbook.proto](https://github.com/WeAreWizards/protojson/blob/master/addressbook.proto) file. Here is one of the model as example:\n+\n+```protobuf\n+message Contact {\n+  required string first_name = 1;\n+  required string last_name = 2;\n+  optional Address address = 3;\n+  repeated Phone phone_numbers = 4;\n+};\n+```\n+In this model, `first_name` and `last_name` are required (controversial, I'll come back to that later), the address model is optional and we can have 0, 1 or n `phone_numbers`.  \n+Now, Python doesn't know about .proto files so we need to use the [python client](https://pypi.python.org/pypi/protobuf) to generate code that we can use. This is the [addressbook_pb2.py](https://github.com/WeAreWizards/protojson/blob/master/addressbook_pb2.py) file. You can then import that in your Python code and access the models defined in your .proto file.  \n+Have a look on the [data.py](https://github.com/WeAreWizards/protojson/blob/master/data.py) client to see an example. You can use those models as you would with a class and using kwargs for every attributes.  \n+You also need to serialize them before sending, using the `SerializeToString()` method in our app.  \n+\n+### Client-side\n+The client is written in Angular purely as an example and because ng-repeat was exactly what I wanted here. The code is a quick prototype so don't expect it to be idiomatic.  \n+It uses [ProtoBuf.js](https://github.com/dcodeIO/ProtoBuf.js) as a Protobuf client.  \n+The first thing we do is a basic GET to get all our contacts which is trivial since you can just dump the response data into our scope variable. On the other hand, we need to handle Protobuf deserialization since looking at our network tab we receive binary data looking like:\n+\n+```\n+.\n+\u0006Marlon\u0012\u0006Sheets\u001A\u001C\n+\u00137771 Eastern Avenue\u0012\u000511239\n+,\n+\u0006Harley\u0012\u0006Malloy\u001A\u0007\u0012\u000560139\"\u0011\b\u0001\u0012\n+(712)614-9303\n+Y\n+\u0004Mack\u0012  Hendricks\u001A3\n+\u00129573 Kimridge Cove\n+\u00167490 Orchard Hill Cove\u0012\u000511239\"\u0011\b\u0002\u0012\n+(802)466-7004\n+\u001A\n+\bMadeline\u0012\u0005Bowen\u001A\u0007\u0012\u000587023\n+0\n+\u0005Willa\u0012\u0006Sadler\u001A\u001F\n+\u0016693 Burlington Parkway\u0012\u000543824\n+T\n+```\n+By default, this is more compact than JSON by quite a bit but once I turned on GZIP, the difference became much smaller: 851B for ProtoBuf and 942B for JSON.  \n+\n+In the example I copied the .proto file in the static directory and created our [models](https://github.com/WeAreWizards/protojson/blob/master/static/main.js#L12-L15) following the ProtoBuf.js tutorial.  ","path":"content/articles/protobuf-instead-of-json-in-frontend.md","position":73,"original_position":73,"commit_id":"43a3e34b46b0649dff36454c479008109c201781","original_commit_id":"43a3e34b46b0649dff36454c479008109c201781","user":{"login":"teh","id":139251,"avatar_url":"https://avatars.githubusercontent.com/u/139251?v=3","gravatar_id":"","url":"https://api.github.com/users/teh","html_url":"https://github.com/teh","followers_url":"https://api.github.com/users/teh/followers","following_url":"https://api.github.com/users/teh/following{/other_user}","gists_url":"https://api.github.com/users/teh/gists{/gist_id}","starred_url":"https://api.github.com/users/teh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/teh/subscriptions","organizations_url":"https://api.github.com/users/teh/orgs","repos_url":"https://api.github.com/users/teh/repos","events_url":"https://api.github.com/users/teh/events{/privacy}","received_events_url":"https://api.github.com/users/teh/received_events","type":"User","site_admin":false},"body":"\"into the static directory\"","created_at":"2015-05-22T15:40:48Z","updated_at":"2015-05-22T15:40:48Z","html_url":"https://github.com/WeAreWizards/blog/pull/22#discussion_r30905917","pull_request_url":"https://api.github.com/repos/WeAreWizards/blog/pulls/22","_links":{"self":{"href":"https://api.github.com/repos/WeAreWizards/blog/pulls/comments/30905917"},"html":{"href":"https://github.com/WeAreWizards/blog/pull/22#discussion_r30905917"},"pull_request":{"href":"https://api.github.com/repos/WeAreWizards/blog/pulls/22"}}},"pull_request":{"url":"https://api.github.com/repos/WeAreWizards/blog/pulls/22","id":36035272,"html_url":"https://github.com/WeAreWizards/blog/pull/22","diff_url":"https://github.com/WeAreWizards/blog/pull/22.diff","patch_url":"https://github.com/WeAreWizards/blog/pull/22.patch","issue_url":"https://api.github.com/repos/WeAreWizards/blog/issues/22","number":22,"state":"open","locked":false,"title":"Protojson","user":{"login":"Keats","id":680355,"avatar_url":"https://avatars.githubusercontent.com/u/680355?v=3","gravatar_id":"","url":"https://api.github.com/users/Keats","html_url":"https://github.com/Keats","followers_url":"https://api.github.com/users/Keats/followers","following_url":"https://api.github.com/users/Keats/following{/other_user}","gists_url":"https://api.github.com/users/Keats/gists{/gist_id}","starred_url":"https://api.github.com/users/Keats/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Keats/subscriptions","organizations_url":"https://api.github.com/users/Keats/orgs","repos_url":"https://api.github.com/users/Keats/repos","events_url":"https://api.github.com/users/Keats/events{/privacy}","received_events_url":"https://api.github.com/users/Keats/received_events","type":"User","site_admin":false},"body":"[rendered](https://github.com/WeAreWizards/blog/blob/protojson/content/articles/protobuf-instead-of-json-in-frontend.md)","created_at":"2015-05-22T14:21:18Z","updated_at":"2015-05-22T15:40:48Z","closed_at":null,"merged_at":null,"merge_commit_sha":"326932a27c76179fc7021de7b632b0975b595b4c","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/WeAreWizards/blog/pulls/22/commits","review_comments_url":"https://api.github.com/repos/WeAreWizards/blog/pulls/22/comments","review_comment_url":"https://api.github.com/repos/WeAreWizards/blog/pulls/comments{/number}","comments_url":"https://api.github.com/repos/WeAreWizards/blog/issues/22/comments","statuses_url":"https://api.github.com/repos/WeAreWizards/blog/statuses/43a3e34b46b0649dff36454c479008109c201781","head":{"label":"WeAreWizards:protojson","ref":"protojson","sha":"43a3e34b46b0649dff36454c479008109c201781","user":{"login":"WeAreWizards","id":8532150,"avatar_url":"https://avatars.githubusercontent.com/u/8532150?v=3","gravatar_id":"","url":"https://api.github.com/users/WeAreWizards","html_url":"https://github.com/WeAreWizards","followers_url":"https://api.github.com/users/WeAreWizards/followers","following_url":"https://api.github.com/users/WeAreWizards/following{/other_user}","gists_url":"https://api.github.com/users/WeAreWizards/gists{/gist_id}","starred_url":"https://api.github.com/users/WeAreWizards/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/WeAreWizards/subscriptions","organizations_url":"https://api.github.com/users/WeAreWizards/orgs","repos_url":"https://api.github.com/users/WeAreWizards/repos","events_url":"https://api.github.com/users/WeAreWizards/events{/privacy}","received_events_url":"https://api.github.com/users/WeAreWizards/received_events","type":"Organization","site_admin":false},"repo":{"id":24109771,"name":"blog","full_name":"WeAreWizards/blog","owner":{"login":"WeAreWizards","id":8532150,"avatar_url":"https://avatars.githubusercontent.com/u/8532150?v=3","gravatar_id":"","url":"https://api.github.com/users/WeAreWizards","html_url":"https://github.com/WeAreWizards","followers_url":"https://api.github.com/users/WeAreWizards/followers","following_url":"https://api.github.com/users/WeAreWizards/following{/other_user}","gists_url":"https://api.github.com/users/WeAreWizards/gists{/gist_id}","starred_url":"https://api.github.com/users/WeAreWizards/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/WeAreWizards/subscriptions","organizations_url":"https://api.github.com/users/WeAreWizards/orgs","repos_url":"https://api.github.com/users/WeAreWizards/repos","events_url":"https://api.github.com/users/WeAreWizards/events{/privacy}","received_events_url":"https://api.github.com/users/WeAreWizards/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/WeAreWizards/blog","description":"We Are Wizards Blog","fork":false,"url":"https://api.github.com/repos/WeAreWizards/blog","forks_url":"https://api.github.com/repos/WeAreWizards/blog/forks","keys_url":"https://api.github.com/repos/WeAreWizards/blog/keys{/key_id}","collaborators_url":"https://api.github.com/repos/WeAreWizards/blog/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/WeAreWizards/blog/teams","hooks_url":"https://api.github.com/repos/WeAreWizards/blog/hooks","issue_events_url":"https://api.github.com/repos/WeAreWizards/blog/issues/events{/number}","events_url":"https://api.github.com/repos/WeAreWizards/blog/events","assignees_url":"https://api.github.com/repos/WeAreWizards/blog/assignees{/user}","branches_url":"https://api.github.com/repos/WeAreWizards/blog/branches{/branch}","tags_url":"https://api.github.com/repos/WeAreWizards/blog/tags","blobs_url":"https://api.github.com/repos/WeAreWizards/blog/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/WeAreWizards/blog/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/WeAreWizards/blog/git/refs{/sha}","trees_url":"https://api.github.com/repos/WeAreWizards/blog/git/trees{/sha}","statuses_url":"https://api.github.com/repos/WeAreWizards/blog/statuses/{sha}","languages_url":"https://api.github.com/repos/WeAreWizards/blog/languages","stargazers_url":"https://api.github.com/repos/WeAreWizards/blog/stargazers","contributors_url":"https://api.github.com/repos/WeAreWizards/blog/contributors","subscribers_url":"https://api.github.com/repos/WeAreWizards/blog/subscribers","subscription_url":"https://api.github.com/repos/WeAreWizards/blog/subscription","commits_url":"https://api.github.com/repos/WeAreWizards/blog/commits{/sha}","git_commits_url":"https://api.github.com/repos/WeAreWizards/blog/git/commits{/sha}","comments_url":"https://api.github.com/repos/WeAreWizards/blog/comments{/number}","issue_comment_url":"https://api.github.com/repos/WeAreWizards/blog/issues/comments{/number}","contents_url":"https://api.github.com/repos/WeAreWizards/blog/contents/{+path}","compare_url":"https://api.github.com/repos/WeAreWizards/blog/compare/{base}...{head}","merges_url":"https://api.github.com/repos/WeAreWizards/blog/merges","archive_url":"https://api.github.com/repos/WeAreWizards/blog/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/WeAreWizards/blog/downloads","issues_url":"https://api.github.com/repos/WeAreWizards/blog/issues{/number}","pulls_url":"https://api.github.com/repos/WeAreWizards/blog/pulls{/number}","milestones_url":"https://api.github.com/repos/WeAreWizards/blog/milestones{/number}","notifications_url":"https://api.github.com/repos/WeAreWizards/blog/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/WeAreWizards/blog/labels{/name}","releases_url":"https://api.github.com/repos/WeAreWizards/blog/releases{/id}","created_at":"2014-09-16T17:23:36Z","updated_at":"2015-05-07T12:06:38Z","pushed_at":"2015-05-22T14:21:18Z","git_url":"git://github.com/WeAreWizards/blog.git","ssh_url":"git@github.com:WeAreWizards/blog.git","clone_url":"https://github.com/WeAreWizards/blog.git","svn_url":"https://github.com/WeAreWizards/blog","homepage":"https://blog.wearewizards.io/","size":2497,"stargazers_count":0,"watchers_count":0,"language":"HTML","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"open_issues_count":3,"forks":1,"open_issues":3,"watchers":0,"default_branch":"master"}},"base":{"label":"WeAreWizards:master","ref":"master","sha":"dc45e7ede6e0403d5cf42e4544a9a2d860c80341","user":{"login":"WeAreWizards","id":8532150,"avatar_url":"https://avatars.githubusercontent.com/u/8532150?v=3","gravatar_id":"","url":"https://api.github.com/users/WeAreWizards","html_url":"https://github.com/WeAreWizards","followers_url":"https://api.github.com/users/WeAreWizards/followers","following_url":"https://api.github.com/users/WeAreWizards/following{/other_user}","gists_url":"https://api.github.com/users/WeAreWizards/gists{/gist_id}","starred_url":"https://api.github.com/users/WeAreWizards/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/WeAreWizards/subscriptions","organizations_url":"https://api.github.com/users/WeAreWizards/orgs","repos_url":"https://api.github.com/users/WeAreWizards/repos","events_url":"https://api.github.com/users/WeAreWizards/events{/privacy}","received_events_url":"https://api.github.com/users/WeAreWizards/received_events","type":"Organization","site_admin":false},"repo":{"id":24109771,"name":"blog","full_name":"WeAreWizards/blog","owner":{"login":"WeAreWizards","id":8532150,"avatar_url":"https://avatars.githubusercontent.com/u/8532150?v=3","gravatar_id":"","url":"https://api.github.com/users/WeAreWizards","html_url":"https://github.com/WeAreWizards","followers_url":"https://api.github.com/users/WeAreWizards/followers","following_url":"https://api.github.com/users/WeAreWizards/following{/other_user}","gists_url":"https://api.github.com/users/WeAreWizards/gists{/gist_id}","starred_url":"https://api.github.com/users/WeAreWizards/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/WeAreWizards/subscriptions","organizations_url":"https://api.github.com/users/WeAreWizards/orgs","repos_url":"https://api.github.com/users/WeAreWizards/repos","events_url":"https://api.github.com/users/WeAreWizards/events{/privacy}","received_events_url":"https://api.github.com/users/WeAreWizards/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/WeAreWizards/blog","description":"We Are Wizards Blog","fork":false,"url":"https://api.github.com/repos/WeAreWizards/blog","forks_url":"https://api.github.com/repos/WeAreWizards/blog/forks","keys_url":"https://api.github.com/repos/WeAreWizards/blog/keys{/key_id}","collaborators_url":"https://api.github.com/repos/WeAreWizards/blog/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/WeAreWizards/blog/teams","hooks_url":"https://api.github.com/repos/WeAreWizards/blog/hooks","issue_events_url":"https://api.github.com/repos/WeAreWizards/blog/issues/events{/number}","events_url":"https://api.github.com/repos/WeAreWizards/blog/events","assignees_url":"https://api.github.com/repos/WeAreWizards/blog/assignees{/user}","branches_url":"https://api.github.com/repos/WeAreWizards/blog/branches{/branch}","tags_url":"https://api.github.com/repos/WeAreWizards/blog/tags","blobs_url":"https://api.github.com/repos/WeAreWizards/blog/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/WeAreWizards/blog/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/WeAreWizards/blog/git/refs{/sha}","trees_url":"https://api.github.com/repos/WeAreWizards/blog/git/trees{/sha}","statuses_url":"https://api.github.com/repos/WeAreWizards/blog/statuses/{sha}","languages_url":"https://api.github.com/repos/WeAreWizards/blog/languages","stargazers_url":"https://api.github.com/repos/WeAreWizards/blog/stargazers","contributors_url":"https://api.github.com/repos/WeAreWizards/blog/contributors","subscribers_url":"https://api.github.com/repos/WeAreWizards/blog/subscribers","subscription_url":"https://api.github.com/repos/WeAreWizards/blog/subscription","commits_url":"https://api.github.com/repos/WeAreWizards/blog/commits{/sha}","git_commits_url":"https://api.github.com/repos/WeAreWizards/blog/git/commits{/sha}","comments_url":"https://api.github.com/repos/WeAreWizards/blog/comments{/number}","issue_comment_url":"https://api.github.com/repos/WeAreWizards/blog/issues/comments{/number}","contents_url":"https://api.github.com/repos/WeAreWizards/blog/contents/{+path}","compare_url":"https://api.github.com/repos/WeAreWizards/blog/compare/{base}...{head}","merges_url":"https://api.github.com/repos/WeAreWizards/blog/merges","archive_url":"https://api.github.com/repos/WeAreWizards/blog/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/WeAreWizards/blog/downloads","issues_url":"https://api.github.com/repos/WeAreWizards/blog/issues{/number}","pulls_url":"https://api.github.com/repos/WeAreWizards/blog/pulls{/number}","milestones_url":"https://api.github.com/repos/WeAreWizards/blog/milestones{/number}","notifications_url":"https://api.github.com/repos/WeAreWizards/blog/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/WeAreWizards/blog/labels{/name}","releases_url":"https://api.github.com/repos/WeAreWizards/blog/releases{/id}","created_at":"2014-09-16T17:23:36Z","updated_at":"2015-05-07T12:06:38Z","pushed_at":"2015-05-22T14:21:18Z","git_url":"git://github.com/WeAreWizards/blog.git","ssh_url":"git@github.com:WeAreWizards/blog.git","clone_url":"https://github.com/WeAreWizards/blog.git","svn_url":"https://github.com/WeAreWizards/blog","homepage":"https://blog.wearewizards.io/","size":2497,"stargazers_count":0,"watchers_count":0,"language":"HTML","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"open_issues_count":3,"forks":1,"open_issues":3,"watchers":0,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/WeAreWizards/blog/pulls/22"},"html":{"href":"https://github.com/WeAreWizards/blog/pull/22"},"issue":{"href":"https://api.github.com/repos/WeAreWizards/blog/issues/22"},"comments":{"href":"https://api.github.com/repos/WeAreWizards/blog/issues/22/comments"},"review_comments":{"href":"https://api.github.com/repos/WeAreWizards/blog/pulls/22/comments"},"review_comment":{"href":"https://api.github.com/repos/WeAreWizards/blog/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/WeAreWizards/blog/pulls/22/commits"},"statuses":{"href":"https://api.github.com/repos/WeAreWizards/blog/statuses/43a3e34b46b0649dff36454c479008109c201781"}}}}