{"action":"created","issue":{"url":"https://api.github.com/repos/googlegenomics/utils-java/issues/40","labels_url":"https://api.github.com/repos/googlegenomics/utils-java/issues/40/labels{/name}","comments_url":"https://api.github.com/repos/googlegenomics/utils-java/issues/40/comments","events_url":"https://api.github.com/repos/googlegenomics/utils-java/issues/40/events","html_url":"https://github.com/googlegenomics/utils-java/pull/40","id":94979808,"number":40,"title":"From logs to graphs and summaries","user":{"login":"jean-philippe-martin","id":10458949,"avatar_url":"https://avatars.githubusercontent.com/u/10458949?v=3","gravatar_id":"","url":"https://api.github.com/users/jean-philippe-martin","html_url":"https://github.com/jean-philippe-martin","followers_url":"https://api.github.com/users/jean-philippe-martin/followers","following_url":"https://api.github.com/users/jean-philippe-martin/following{/other_user}","gists_url":"https://api.github.com/users/jean-philippe-martin/gists{/gist_id}","starred_url":"https://api.github.com/users/jean-philippe-martin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jean-philippe-martin/subscriptions","organizations_url":"https://api.github.com/users/jean-philippe-martin/orgs","repos_url":"https://api.github.com/users/jean-philippe-martin/repos","events_url":"https://api.github.com/users/jean-philippe-martin/events{/privacy}","received_events_url":"https://api.github.com/users/jean-philippe-martin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-07-14T16:06:05Z","updated_at":"2015-07-16T15:42:10Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/googlegenomics/utils-java/pulls/40","html_url":"https://github.com/googlegenomics/utils-java/pull/40","diff_url":"https://github.com/googlegenomics/utils-java/pull/40.diff","patch_url":"https://github.com/googlegenomics/utils-java/pull/40.patch"},"body":"This is a little piece of code I'm using to help get the big picture from Dataflow logs. I've found it very useful. It relies on the user to add a few logging statements in the \"bunny\" format, namely:\r\n```\r\n=[**]= START <id> <name>\r\n=[**]= STEPEND <id> <step_name>\r\n=[**]= END <id>\r\n```\r\n\r\nThis will get you a little bar from start to end time, labeled with the name you chose. Intermediate steps, if any, show up as \"bubbles\" inside the bar. The summary will show how many times this name started, the aggregate time, and span time (from first start to last end).\r\n"},"comment":{"url":"https://api.github.com/repos/googlegenomics/utils-java/issues/comments/121997189","html_url":"https://github.com/googlegenomics/utils-java/pull/40#issuecomment-121997189","issue_url":"https://api.github.com/repos/googlegenomics/utils-java/issues/40","id":121997189,"user":{"login":"deflaux","id":1395500,"avatar_url":"https://avatars.githubusercontent.com/u/1395500?v=3","gravatar_id":"","url":"https://api.github.com/users/deflaux","html_url":"https://github.com/deflaux","followers_url":"https://api.github.com/users/deflaux/followers","following_url":"https://api.github.com/users/deflaux/following{/other_user}","gists_url":"https://api.github.com/users/deflaux/gists{/gist_id}","starred_url":"https://api.github.com/users/deflaux/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/deflaux/subscriptions","organizations_url":"https://api.github.com/users/deflaux/orgs","repos_url":"https://api.github.com/users/deflaux/repos","events_url":"https://api.github.com/users/deflaux/events{/privacy}","received_events_url":"https://api.github.com/users/deflaux/received_events","type":"User","site_admin":false},"created_at":"2015-07-16T15:42:10Z","updated_at":"2015-07-16T15:42:10Z","body":"This looks like a handy utility.  Could you please describe a little more how one uses this?  I'm guessing its something like:\r\n\r\n (1) update your DoFn, startBundle, etc. to emit bunny-format to the log\r\n (2) run your pipeline\r\n (3) retrieve worker logs\r\n (4) run logsummarizer.Main on one log at a time\r\n"}}