{"action":"created","issue":{"url":"https://api.github.com/repos/DataDog/dd-agent/issues/1949","labels_url":"https://api.github.com/repos/DataDog/dd-agent/issues/1949/labels{/name}","comments_url":"https://api.github.com/repos/DataDog/dd-agent/issues/1949/comments","events_url":"https://api.github.com/repos/DataDog/dd-agent/issues/1949/events","html_url":"https://github.com/DataDog/dd-agent/pull/1949","id":108357696,"number":1949,"title":"Add a command \"predicate\" to allow runtime blackholing of metrics.","user":{"login":"gphat","id":45159,"avatar_url":"https://avatars.githubusercontent.com/u/45159?v=3","gravatar_id":"","url":"https://api.github.com/users/gphat","html_url":"https://github.com/gphat","followers_url":"https://api.github.com/users/gphat/followers","following_url":"https://api.github.com/users/gphat/following{/other_user}","gists_url":"https://api.github.com/users/gphat/gists{/gist_id}","starred_url":"https://api.github.com/users/gphat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gphat/subscriptions","organizations_url":"https://api.github.com/users/gphat/orgs","repos_url":"https://api.github.com/users/gphat/repos","events_url":"https://api.github.com/users/gphat/events{/privacy}","received_events_url":"https://api.github.com/users/gphat/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-09-25T16:22:30Z","updated_at":"2015-09-28T09:34:31Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/DataDog/dd-agent/pulls/1949","html_url":"https://github.com/DataDog/dd-agent/pull/1949","diff_url":"https://github.com/DataDog/dd-agent/pull/1949.diff","patch_url":"https://github.com/DataDog/dd-agent/pull/1949.patch"},"body":"# What's this PR do?\r\n\r\nAdds a configuration option `flush_metrics_predicate` that controls if the forwarder flushes or \"blackholes\" metrics, dropping them permanently.\r\n\r\n# Motivation\r\n\r\nOur hosts have an on-box sentinel that controls if they are \"live\" or not. This liveness is used for health checks and other things. The semantics are such that if the command returns a non-zero status, we don't want to collect metrics for this host.  Assuming that the monitors in question do not alert on \"No Data\" then we've effectively muted the host and removed it's weirdo metrics.\r\n\r\n# Notes\r\n\r\n* I had to add a few methods to the `TransactionManager` to facilitate testing.\r\n* It's possible that our liveness utility may occasionally want to send metrics anyway in some circumstances and that this won't ultimately be what we use but it felt like a somewhat useful feature.\r\n* The semantics of this option and the methods (false versus true, non-zero output, etc) are totally changeable if you folks see it another way.\r\n* Since the execution of the check is a blocking operation for the `TransactionManager` I made notes in the config file warning of slow checks.\r\n* I'm not a Pythonista so improvements to the execution code are welcome if I've done poorly. :)"},"comment":{"url":"https://api.github.com/repos/DataDog/dd-agent/issues/comments/143691119","html_url":"https://github.com/DataDog/dd-agent/pull/1949#issuecomment-143691119","issue_url":"https://api.github.com/repos/DataDog/dd-agent/issues/1949","id":143691119,"user":{"login":"remh","id":1311477,"avatar_url":"https://avatars.githubusercontent.com/u/1311477?v=3","gravatar_id":"","url":"https://api.github.com/users/remh","html_url":"https://github.com/remh","followers_url":"https://api.github.com/users/remh/followers","following_url":"https://api.github.com/users/remh/following{/other_user}","gists_url":"https://api.github.com/users/remh/gists{/gist_id}","starred_url":"https://api.github.com/users/remh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/remh/subscriptions","organizations_url":"https://api.github.com/users/remh/orgs","repos_url":"https://api.github.com/users/remh/repos","events_url":"https://api.github.com/users/remh/events{/privacy}","received_events_url":"https://api.github.com/users/remh/received_events","type":"User","site_admin":false},"created_at":"2015-09-28T09:34:31Z","updated_at":"2015-09-28T09:34:31Z","body":"Thanks a lot @gphat . As @alq666 , i love the details you brought to this PR. :clap: \r\n\r\nRegarding the idea itself, i understand the use case, but would it make sense to use a cron job or maybe a long living script that would control (stop / start ) the agent depending on the liveness check ? \r\n\r\nAnother benefit of that approach is that the agent would effectively be stopped and won't consume any additional resources.\r\n"}}