{"action":"created","issue":{"url":"https://api.github.com/repos/kaizenplatform/fluent-plugin-bigquery/issues/37","labels_url":"https://api.github.com/repos/kaizenplatform/fluent-plugin-bigquery/issues/37/labels{/name}","comments_url":"https://api.github.com/repos/kaizenplatform/fluent-plugin-bigquery/issues/37/comments","events_url":"https://api.github.com/repos/kaizenplatform/fluent-plugin-bigquery/issues/37/events","html_url":"https://github.com/kaizenplatform/fluent-plugin-bigquery/pull/37","id":55003333,"number":37,"title":"Implement BigQuery load job","user":{"login":"shojikai","id":6204023,"avatar_url":"https://avatars.githubusercontent.com/u/6204023?v=3","gravatar_id":"","url":"https://api.github.com/users/shojikai","html_url":"https://github.com/shojikai","followers_url":"https://api.github.com/users/shojikai/followers","following_url":"https://api.github.com/users/shojikai/following{/other_user}","gists_url":"https://api.github.com/users/shojikai/gists{/gist_id}","starred_url":"https://api.github.com/users/shojikai/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shojikai/subscriptions","organizations_url":"https://api.github.com/users/shojikai/orgs","repos_url":"https://api.github.com/users/shojikai/repos","events_url":"https://api.github.com/users/shojikai/events{/privacy}","received_events_url":"https://api.github.com/users/shojikai/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-01-21T10:47:21Z","updated_at":"2015-11-11T11:35:32Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/kaizenplatform/fluent-plugin-bigquery/pulls/37","html_url":"https://github.com/kaizenplatform/fluent-plugin-bigquery/pull/37","diff_url":"https://github.com/kaizenplatform/fluent-plugin-bigquery/pull/37.diff","patch_url":"https://github.com/kaizenplatform/fluent-plugin-bigquery/pull/37.patch"},"body":"Hi,\r\n\r\nI've implemented BigQuery load job.\r\n(Because streaming inserts cost $0.01 per 100,000 rows from 2015.)\r\n\r\nSo, please check it. Implementation points are below.\r\n\r\nImplementation points:\r\n- Suported buffer_type is \"file\" only.\r\n- A buffer file format is NEWLINED_DELIMITED_JSON. (https://cloud.google.com/bigquery/preparing-data-for-bigquery#dataformats)\r\n- You can select sync/async mode using sync_load_job option.\r\n  - sync mode (default): Don't dequeue from buffer_queue until a load job state is done. A load job state is checked at retry interval of fluentd.\r\n  - async mode: Dequeue from buffer_queue as soon as a load job is registered in BigQuery. A load job state is not checked.\r\n\r\nIf you have any opinions, please let me know.\r\n\r\nBest regards."},"comment":{"url":"https://api.github.com/repos/kaizenplatform/fluent-plugin-bigquery/issues/comments/155742789","html_url":"https://github.com/kaizenplatform/fluent-plugin-bigquery/pull/37#issuecomment-155742789","issue_url":"https://api.github.com/repos/kaizenplatform/fluent-plugin-bigquery/issues/37","id":155742789,"user":{"login":"Yanson","id":6860163,"avatar_url":"https://avatars.githubusercontent.com/u/6860163?v=3","gravatar_id":"","url":"https://api.github.com/users/Yanson","html_url":"https://github.com/Yanson","followers_url":"https://api.github.com/users/Yanson/followers","following_url":"https://api.github.com/users/Yanson/following{/other_user}","gists_url":"https://api.github.com/users/Yanson/gists{/gist_id}","starred_url":"https://api.github.com/users/Yanson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Yanson/subscriptions","organizations_url":"https://api.github.com/users/Yanson/orgs","repos_url":"https://api.github.com/users/Yanson/repos","events_url":"https://api.github.com/users/Yanson/events{/privacy}","received_events_url":"https://api.github.com/users/Yanson/received_events","type":"User","site_admin":false},"created_at":"2015-11-11T11:35:32Z","updated_at":"2015-11-11T11:35:32Z","body":"Could you clarify how to do this?\r\nI've created a Gemfile with the contents:\r\n```\r\nsource \"https://rubygems.org\"\r\ngem 'fluent-plugin-bigquery', :git => 'https://github.com/shojikai/fluent-plugin-bigquery', :branch => 'work'\r\n```\r\nand run `/opt/td-agent/embedded/bin/bundle install`.\r\nI see it installed in `/opt/td-agent/embedded/lib/ruby/gems/2.1.0/bundler/gems/fluent-plugin-bigquery-aa6a15dc3282` but it's not picked up by fluentd.\r\n\r\nHave I missed a step?"}}