{"action":"created","comment":{"url":"https://api.github.com/repos/mozilla-services/heka/pulls/comments/22761385","id":22761385,"diff_hunk":"@@ -0,0 +1,204 @@\n+/***** BEGIN LICENSE BLOCK *****\n+# This Source Code Form is subject to the terms of the Mozilla Public\n+# License, v. 2.0. If a copy of the MPL was not distributed with this file,\n+# You can obtain one at http://mozilla.org/MPL/2.0/.\n+#\n+# The Initial Developer of the Original Code is the Mozilla Foundation.\n+# Portions created by the Initial Developer are Copyright (C) 2014\n+# the Initial Developer. All Rights Reserved.\n+#\n+# This plugin is based on the HttpListenPlugin.\n+#\n+# Contributor(s):\n+#   Anton Lindström (carlantonlindstrom@gmail.com)\n+#\n+# ***** END LICENSE BLOCK *****/\n+\n+package heroku\n+\n+import (\n+\t\"code.google.com/p/go-uuid/uuid\"\n+\t\"fmt\"\n+\t\"github.com/mozilla-services/heka/message\"\n+\t\"github.com/mozilla-services/heka/pipeline\"\n+\t\"io/ioutil\"\n+\t\"net\"\n+\t\"net/http\"\n+\t\"regexp\"\n+\t\"strconv\"\n+\t\"strings\"\n+\t\"time\"\n+)\n+\n+type HerokuLogInput struct {\n+\tconf        *HerokuLogInputConfig\n+\tlistener    net.Listener\n+\tstopChan    chan bool\n+\tir          pipeline.InputRunner\n+\tserver      *http.Server\n+\tstarterFunc func(hli *HerokuLogInput) error\n+\thostname    string\n+}\n+\n+const LOGPLEX_CONTENT_TYPE = \"application/logplex-1\"\n+\n+// Heroku Log Input config struct\n+type HerokuLogInputConfig struct {\n+\t// TCP Address to listen to\n+\t// Defaults to \"0.0.0.0:8325\".\n+\tAddress string","path":"plugins/heroku/heroku_log_input.go","position":49,"original_position":49,"commit_id":"349f4a1c0fb410f41e7e470ce51acd9b13df2887","original_commit_id":"349f4a1c0fb410f41e7e470ce51acd9b13df2887","user":{"login":"carlanton","id":342897,"avatar_url":"https://avatars.githubusercontent.com/u/342897?v=3","gravatar_id":"","url":"https://api.github.com/users/carlanton","html_url":"https://github.com/carlanton","followers_url":"https://api.github.com/users/carlanton/followers","following_url":"https://api.github.com/users/carlanton/following{/other_user}","gists_url":"https://api.github.com/users/carlanton/gists{/gist_id}","starred_url":"https://api.github.com/users/carlanton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/carlanton/subscriptions","organizations_url":"https://api.github.com/users/carlanton/orgs","repos_url":"https://api.github.com/users/carlanton/repos","events_url":"https://api.github.com/users/carlanton/events{/privacy}","received_events_url":"https://api.github.com/users/carlanton/received_events","type":"User","site_admin":false},"body":"@4r9h You don't need the tag thing for \"simple\" names.","created_at":"2015-01-10T19:39:52Z","updated_at":"2015-01-10T19:39:52Z","html_url":"https://github.com/mozilla-services/heka/pull/1267#discussion_r22761385","pull_request_url":"https://api.github.com/repos/mozilla-services/heka/pulls/1267","_links":{"self":{"href":"https://api.github.com/repos/mozilla-services/heka/pulls/comments/22761385"},"html":{"href":"https://github.com/mozilla-services/heka/pull/1267#discussion_r22761385"},"pull_request":{"href":"https://api.github.com/repos/mozilla-services/heka/pulls/1267"}}},"pull_request":{"url":"https://api.github.com/repos/mozilla-services/heka/pulls/1267","id":27098918,"html_url":"https://github.com/mozilla-services/heka/pull/1267","diff_url":"https://github.com/mozilla-services/heka/pull/1267.diff","patch_url":"https://github.com/mozilla-services/heka/pull/1267.patch","issue_url":"https://api.github.com/repos/mozilla-services/heka/issues/1267","number":1267,"state":"open","locked":false,"title":"[WIP] Add Heroku log input plugin","user":{"login":"carlanton","id":342897,"avatar_url":"https://avatars.githubusercontent.com/u/342897?v=3","gravatar_id":"","url":"https://api.github.com/users/carlanton","html_url":"https://github.com/carlanton","followers_url":"https://api.github.com/users/carlanton/followers","following_url":"https://api.github.com/users/carlanton/following{/other_user}","gists_url":"https://api.github.com/users/carlanton/gists{/gist_id}","starred_url":"https://api.github.com/users/carlanton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/carlanton/subscriptions","organizations_url":"https://api.github.com/users/carlanton/orgs","repos_url":"https://api.github.com/users/carlanton/repos","events_url":"https://api.github.com/users/carlanton/events{/privacy}","received_events_url":"https://api.github.com/users/carlanton/received_events","type":"User","site_admin":false},"body":"In #1241 we discussed how to receive log messages from Heroku using HttpListenInput and a decoder or filter.\r\nThis pr instead adds a modified version of the HttpListenInput with input validation and log parsing.\r\n\r\nWith this approach we can:\r\n - Reject messages (we can require that incoming requests have certain headers etc.)\r\n - Do the basic Heroku syslog parsing (parsing of the actual message payload is application specific and can be done with sandbox decoder or something)\r\n - Attach metadata to the messages from the http headers (just DrainToken for now)\r\n\r\nIs this something that would be useful to have in Heka core? :octocat: ","created_at":"2015-01-09T13:14:12Z","updated_at":"2015-01-10T19:39:52Z","closed_at":null,"merged_at":null,"merge_commit_sha":"e2584853c2e1815c672045f3a03b5ecc611b5e99","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/mozilla-services/heka/pulls/1267/commits","review_comments_url":"https://api.github.com/repos/mozilla-services/heka/pulls/1267/comments","review_comment_url":"https://api.github.com/repos/mozilla-services/heka/pulls/comments/{number}","comments_url":"https://api.github.com/repos/mozilla-services/heka/issues/1267/comments","statuses_url":"https://api.github.com/repos/mozilla-services/heka/statuses/349f4a1c0fb410f41e7e470ce51acd9b13df2887","head":{"label":"carlanton:heroku-log-input","ref":"heroku-log-input","sha":"349f4a1c0fb410f41e7e470ce51acd9b13df2887","user":{"login":"carlanton","id":342897,"avatar_url":"https://avatars.githubusercontent.com/u/342897?v=3","gravatar_id":"","url":"https://api.github.com/users/carlanton","html_url":"https://github.com/carlanton","followers_url":"https://api.github.com/users/carlanton/followers","following_url":"https://api.github.com/users/carlanton/following{/other_user}","gists_url":"https://api.github.com/users/carlanton/gists{/gist_id}","starred_url":"https://api.github.com/users/carlanton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/carlanton/subscriptions","organizations_url":"https://api.github.com/users/carlanton/orgs","repos_url":"https://api.github.com/users/carlanton/repos","events_url":"https://api.github.com/users/carlanton/events{/privacy}","received_events_url":"https://api.github.com/users/carlanton/received_events","type":"User","site_admin":false},"repo":{"id":24492022,"name":"heka","full_name":"carlanton/heka","owner":{"login":"carlanton","id":342897,"avatar_url":"https://avatars.githubusercontent.com/u/342897?v=3","gravatar_id":"","url":"https://api.github.com/users/carlanton","html_url":"https://github.com/carlanton","followers_url":"https://api.github.com/users/carlanton/followers","following_url":"https://api.github.com/users/carlanton/following{/other_user}","gists_url":"https://api.github.com/users/carlanton/gists{/gist_id}","starred_url":"https://api.github.com/users/carlanton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/carlanton/subscriptions","organizations_url":"https://api.github.com/users/carlanton/orgs","repos_url":"https://api.github.com/users/carlanton/repos","events_url":"https://api.github.com/users/carlanton/events{/privacy}","received_events_url":"https://api.github.com/users/carlanton/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/carlanton/heka","description":"Data collection and processing made easy.","fork":true,"url":"https://api.github.com/repos/carlanton/heka","forks_url":"https://api.github.com/repos/carlanton/heka/forks","keys_url":"https://api.github.com/repos/carlanton/heka/keys{/key_id}","collaborators_url":"https://api.github.com/repos/carlanton/heka/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/carlanton/heka/teams","hooks_url":"https://api.github.com/repos/carlanton/heka/hooks","issue_events_url":"https://api.github.com/repos/carlanton/heka/issues/events{/number}","events_url":"https://api.github.com/repos/carlanton/heka/events","assignees_url":"https://api.github.com/repos/carlanton/heka/assignees{/user}","branches_url":"https://api.github.com/repos/carlanton/heka/branches{/branch}","tags_url":"https://api.github.com/repos/carlanton/heka/tags","blobs_url":"https://api.github.com/repos/carlanton/heka/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/carlanton/heka/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/carlanton/heka/git/refs{/sha}","trees_url":"https://api.github.com/repos/carlanton/heka/git/trees{/sha}","statuses_url":"https://api.github.com/repos/carlanton/heka/statuses/{sha}","languages_url":"https://api.github.com/repos/carlanton/heka/languages","stargazers_url":"https://api.github.com/repos/carlanton/heka/stargazers","contributors_url":"https://api.github.com/repos/carlanton/heka/contributors","subscribers_url":"https://api.github.com/repos/carlanton/heka/subscribers","subscription_url":"https://api.github.com/repos/carlanton/heka/subscription","commits_url":"https://api.github.com/repos/carlanton/heka/commits{/sha}","git_commits_url":"https://api.github.com/repos/carlanton/heka/git/commits{/sha}","comments_url":"https://api.github.com/repos/carlanton/heka/comments{/number}","issue_comment_url":"https://api.github.com/repos/carlanton/heka/issues/comments/{number}","contents_url":"https://api.github.com/repos/carlanton/heka/contents/{+path}","compare_url":"https://api.github.com/repos/carlanton/heka/compare/{base}...{head}","merges_url":"https://api.github.com/repos/carlanton/heka/merges","archive_url":"https://api.github.com/repos/carlanton/heka/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/carlanton/heka/downloads","issues_url":"https://api.github.com/repos/carlanton/heka/issues{/number}","pulls_url":"https://api.github.com/repos/carlanton/heka/pulls{/number}","milestones_url":"https://api.github.com/repos/carlanton/heka/milestones{/number}","notifications_url":"https://api.github.com/repos/carlanton/heka/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/carlanton/heka/labels{/name}","releases_url":"https://api.github.com/repos/carlanton/heka/releases{/id}","created_at":"2014-09-26T08:24:31Z","updated_at":"2014-09-26T13:30:14Z","pushed_at":"2015-01-10T15:07:55Z","git_url":"git://github.com/carlanton/heka.git","ssh_url":"git@github.com:carlanton/heka.git","clone_url":"https://github.com/carlanton/heka.git","svn_url":"https://github.com/carlanton/heka","homepage":"http://hekad.readthedocs.org/","size":9943,"stargazers_count":0,"watchers_count":0,"language":"Go","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"dev"}},"base":{"label":"mozilla-services:dev","ref":"dev","sha":"28ffad108ea83f534d464138bd97c9a28752685c","user":{"login":"mozilla-services","id":1066228,"avatar_url":"https://avatars.githubusercontent.com/u/1066228?v=3","gravatar_id":"","url":"https://api.github.com/users/mozilla-services","html_url":"https://github.com/mozilla-services","followers_url":"https://api.github.com/users/mozilla-services/followers","following_url":"https://api.github.com/users/mozilla-services/following{/other_user}","gists_url":"https://api.github.com/users/mozilla-services/gists{/gist_id}","starred_url":"https://api.github.com/users/mozilla-services/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mozilla-services/subscriptions","organizations_url":"https://api.github.com/users/mozilla-services/orgs","repos_url":"https://api.github.com/users/mozilla-services/repos","events_url":"https://api.github.com/users/mozilla-services/events{/privacy}","received_events_url":"https://api.github.com/users/mozilla-services/received_events","type":"Organization","site_admin":false},"repo":{"id":6248212,"name":"heka","full_name":"mozilla-services/heka","owner":{"login":"mozilla-services","id":1066228,"avatar_url":"https://avatars.githubusercontent.com/u/1066228?v=3","gravatar_id":"","url":"https://api.github.com/users/mozilla-services","html_url":"https://github.com/mozilla-services","followers_url":"https://api.github.com/users/mozilla-services/followers","following_url":"https://api.github.com/users/mozilla-services/following{/other_user}","gists_url":"https://api.github.com/users/mozilla-services/gists{/gist_id}","starred_url":"https://api.github.com/users/mozilla-services/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mozilla-services/subscriptions","organizations_url":"https://api.github.com/users/mozilla-services/orgs","repos_url":"https://api.github.com/users/mozilla-services/repos","events_url":"https://api.github.com/users/mozilla-services/events{/privacy}","received_events_url":"https://api.github.com/users/mozilla-services/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/mozilla-services/heka","description":"Data collection and processing made easy.","fork":false,"url":"https://api.github.com/repos/mozilla-services/heka","forks_url":"https://api.github.com/repos/mozilla-services/heka/forks","keys_url":"https://api.github.com/repos/mozilla-services/heka/keys{/key_id}","collaborators_url":"https://api.github.com/repos/mozilla-services/heka/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/mozilla-services/heka/teams","hooks_url":"https://api.github.com/repos/mozilla-services/heka/hooks","issue_events_url":"https://api.github.com/repos/mozilla-services/heka/issues/events{/number}","events_url":"https://api.github.com/repos/mozilla-services/heka/events","assignees_url":"https://api.github.com/repos/mozilla-services/heka/assignees{/user}","branches_url":"https://api.github.com/repos/mozilla-services/heka/branches{/branch}","tags_url":"https://api.github.com/repos/mozilla-services/heka/tags","blobs_url":"https://api.github.com/repos/mozilla-services/heka/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/mozilla-services/heka/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/mozilla-services/heka/git/refs{/sha}","trees_url":"https://api.github.com/repos/mozilla-services/heka/git/trees{/sha}","statuses_url":"https://api.github.com/repos/mozilla-services/heka/statuses/{sha}","languages_url":"https://api.github.com/repos/mozilla-services/heka/languages","stargazers_url":"https://api.github.com/repos/mozilla-services/heka/stargazers","contributors_url":"https://api.github.com/repos/mozilla-services/heka/contributors","subscribers_url":"https://api.github.com/repos/mozilla-services/heka/subscribers","subscription_url":"https://api.github.com/repos/mozilla-services/heka/subscription","commits_url":"https://api.github.com/repos/mozilla-services/heka/commits{/sha}","git_commits_url":"https://api.github.com/repos/mozilla-services/heka/git/commits{/sha}","comments_url":"https://api.github.com/repos/mozilla-services/heka/comments{/number}","issue_comment_url":"https://api.github.com/repos/mozilla-services/heka/issues/comments/{number}","contents_url":"https://api.github.com/repos/mozilla-services/heka/contents/{+path}","compare_url":"https://api.github.com/repos/mozilla-services/heka/compare/{base}...{head}","merges_url":"https://api.github.com/repos/mozilla-services/heka/merges","archive_url":"https://api.github.com/repos/mozilla-services/heka/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/mozilla-services/heka/downloads","issues_url":"https://api.github.com/repos/mozilla-services/heka/issues{/number}","pulls_url":"https://api.github.com/repos/mozilla-services/heka/pulls{/number}","milestones_url":"https://api.github.com/repos/mozilla-services/heka/milestones{/number}","notifications_url":"https://api.github.com/repos/mozilla-services/heka/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/mozilla-services/heka/labels{/name}","releases_url":"https://api.github.com/repos/mozilla-services/heka/releases{/id}","created_at":"2012-10-16T17:20:51Z","updated_at":"2015-01-10T16:38:56Z","pushed_at":"2015-01-10T00:45:37Z","git_url":"git://github.com/mozilla-services/heka.git","ssh_url":"git@github.com:mozilla-services/heka.git","clone_url":"https://github.com/mozilla-services/heka.git","svn_url":"https://github.com/mozilla-services/heka","homepage":"http://hekad.readthedocs.org/","size":17916,"stargazers_count":1817,"watchers_count":1817,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":224,"mirror_url":null,"open_issues_count":112,"forks":224,"open_issues":112,"watchers":1817,"default_branch":"dev"}},"_links":{"self":{"href":"https://api.github.com/repos/mozilla-services/heka/pulls/1267"},"html":{"href":"https://github.com/mozilla-services/heka/pull/1267"},"issue":{"href":"https://api.github.com/repos/mozilla-services/heka/issues/1267"},"comments":{"href":"https://api.github.com/repos/mozilla-services/heka/issues/1267/comments"},"review_comments":{"href":"https://api.github.com/repos/mozilla-services/heka/pulls/1267/comments"},"review_comment":{"href":"https://api.github.com/repos/mozilla-services/heka/pulls/comments/{number}"},"commits":{"href":"https://api.github.com/repos/mozilla-services/heka/pulls/1267/commits"},"statuses":{"href":"https://api.github.com/repos/mozilla-services/heka/statuses/349f4a1c0fb410f41e7e470ce51acd9b13df2887"}}}}