{"action":"created","issue":{"url":"https://api.github.com/repos/bosun-monitor/bosun/issues/727","labels_url":"https://api.github.com/repos/bosun-monitor/bosun/issues/727/labels{/name}","comments_url":"https://api.github.com/repos/bosun-monitor/bosun/issues/727/comments","events_url":"https://api.github.com/repos/bosun-monitor/bosun/issues/727/events","html_url":"https://github.com/bosun-monitor/bosun/issues/727","id":57963079,"number":727,"title":"Kill scollector if the handle count exceed given threshold","user":{"login":"gbrayut","id":304401,"avatar_url":"https://avatars.githubusercontent.com/u/304401?v=3","gravatar_id":"","url":"https://api.github.com/users/gbrayut","html_url":"https://github.com/gbrayut","followers_url":"https://api.github.com/users/gbrayut/followers","following_url":"https://api.github.com/users/gbrayut/following{/other_user}","gists_url":"https://api.github.com/users/gbrayut/gists{/gist_id}","starred_url":"https://api.github.com/users/gbrayut/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gbrayut/subscriptions","organizations_url":"https://api.github.com/users/gbrayut/orgs","repos_url":"https://api.github.com/users/gbrayut/repos","events_url":"https://api.github.com/users/gbrayut/events{/privacy}","received_events_url":"https://api.github.com/users/gbrayut/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-02-17T18:37:01Z","updated_at":"2015-02-18T17:24:47Z","closed_at":null,"body":"Today we had an issue where scollector exhausted the UDP ports on the local server. It looks like the handle count for the process was over 16,000 by the time we found the root cause, where as normally it operates in the 500 range.\r\n\r\n![image](https://cloud.githubusercontent.com/assets/304401/6234584/33c8eb96-b699-11e4-9b9a-874d73ea624b.png)\r\n\r\n\r\nWe should add a kill switch to scollector if it ever goes above 5,000 or 10,000 similar to the [memory usage panic](https://github.com/bosun-monitor/bosun/blob/master/cmd/scollector/main.go#L260)\r\n"},"comment":{"url":"https://api.github.com/repos/bosun-monitor/bosun/issues/comments/74906631","html_url":"https://github.com/bosun-monitor/bosun/issues/727#issuecomment-74906631","issue_url":"https://api.github.com/repos/bosun-monitor/bosun/issues/727","id":74906631,"user":{"login":"kylebrandt","id":1692624,"avatar_url":"https://avatars.githubusercontent.com/u/1692624?v=3","gravatar_id":"","url":"https://api.github.com/users/kylebrandt","html_url":"https://github.com/kylebrandt","followers_url":"https://api.github.com/users/kylebrandt/followers","following_url":"https://api.github.com/users/kylebrandt/following{/other_user}","gists_url":"https://api.github.com/users/kylebrandt/gists{/gist_id}","starred_url":"https://api.github.com/users/kylebrandt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kylebrandt/subscriptions","organizations_url":"https://api.github.com/users/kylebrandt/orgs","repos_url":"https://api.github.com/users/kylebrandt/repos","events_url":"https://api.github.com/users/kylebrandt/events{/privacy}","received_events_url":"https://api.github.com/users/kylebrandt/received_events","type":"User","site_admin":false},"created_at":"2015-02-18T17:24:47Z","updated_at":"2015-02-18T17:24:47Z","body":"Something to keep in mind with this is that if we tie this into the\ncollector code, people can disable those collectors and probably wouldn't\nrealize that they no longer get threshold protection. So I think these\nmaybe all needs to go into kill_windows.go and kill_linux.go etc or\nsomething to be independent.\n\nOn Wed, Feb 18, 2015 at 12:03 PM, Greg Bray <notifications@github.com>\nwrote:\n\n> On Windows, the handle count is already monitored via win.proc.handle_count\n> On Linux, linux.proc.num_fds is similar to the handle count but may need a\n> different threshold\n> On Mac ?\n>\n> —\n> Reply to this email directly or view it on GitHub\n> <https://github.com/bosun-monitor/bosun/issues/727#issuecomment-74902475>.\n>\n"}}