{"action":"opened","issue":{"url":"https://api.github.com/repos/raintank/raintank-probe/issues/4","labels_url":"https://api.github.com/repos/raintank/raintank-probe/issues/4/labels{/name}","comments_url":"https://api.github.com/repos/raintank/raintank-probe/issues/4/comments","events_url":"https://api.github.com/repos/raintank/raintank-probe/issues/4/events","html_url":"https://github.com/raintank/raintank-probe/issues/4","id":103689902,"number":4,"title":"use heka as a base?","user":{"login":"Dieterbe","id":20774,"avatar_url":"https://avatars.githubusercontent.com/u/20774?v=3","gravatar_id":"","url":"https://api.github.com/users/Dieterbe","html_url":"https://github.com/Dieterbe","followers_url":"https://api.github.com/users/Dieterbe/followers","following_url":"https://api.github.com/users/Dieterbe/following{/other_user}","gists_url":"https://api.github.com/users/Dieterbe/gists{/gist_id}","starred_url":"https://api.github.com/users/Dieterbe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Dieterbe/subscriptions","organizations_url":"https://api.github.com/users/Dieterbe/orgs","repos_url":"https://api.github.com/users/Dieterbe/repos","events_url":"https://api.github.com/users/Dieterbe/events{/privacy}","received_events_url":"https://api.github.com/users/Dieterbe/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-08-28T10:03:05Z","updated_at":"2015-08-28T10:03:05Z","closed_at":null,"body":"this came up as part of https://github.com/raintank/ops/issues/97\r\nI think it may make sense to use heka as a base platform for the new probe.\r\nwhy?\r\n* it has in-memory and on-disk buffering built-in (no strong sync() semantics, but still)\r\n* it is very extensible, in case we want to add more plugins later to provide other kinds of metrics\r\n* it has ways to graph data in flight , do thresholding/anomaly detection in realtime as data flows through, etc\r\n\r\ncons:\r\n* it comes with overhead, it is a very flexible tool so it does a bunch of stuff we don't need yet or may never need. (like the internal flexible routing system with matchers and protobuf encoding/decoding)\r\n* it uses non-go code, like the lua sandbox which is C, so it's harder to build.\r\n\r\n\r\nquote @woodsaj \r\n> I hadn't considered using Heka as the core of the collector process, but it does sound interesting. I would assume that you would use an input plugin for scheduling and executing checks, and an output plugin for sending results. This would mean that we would need 2 connections to the grafana-server. 1 for the input plugin to receive check add/update/delete commands and one for the output plugin to send results. Which would complicate tracking whether a collector is online or not. I would much rather just have one persistent authenticated connection with bi-directional communication.\r\n\r\nwe wouldn't need 2 connections if both plugins import a separate package that has a connection singleton. (instantiated by only 1 of the plugins, but used by both)\r\n\r\npersonally I think the in-mem and on-disk buffering alone is not enough to make this a good choice.\r\nwe want to keep this lightweight so it runs well on rbpi's. so unless we see a clear need for any of the other features, i say we just write it custom for ourselves, possibly reusing the nsqd diskqueue.go\r\n\r\nI like the raintank-metric development model where we keep it simple at first and iterate on it, and once we hit clear roadblocks and need to re-architect things, only then do we make an impactful decision like adopting a tool like heka, porting the code is usually not too much work."}}