{"action":"opened","issue":{"url":"https://api.github.com/repos/sensu/sensu/issues/1052","labels_url":"https://api.github.com/repos/sensu/sensu/issues/1052/labels{/name}","comments_url":"https://api.github.com/repos/sensu/sensu/issues/1052/comments","events_url":"https://api.github.com/repos/sensu/sensu/issues/1052/events","html_url":"https://github.com/sensu/sensu/issues/1052","id":106764133,"number":1052,"title":"sensu-client fails to restart when a check is not finished yet","user":{"login":"stefanandres","id":2674469,"avatar_url":"https://avatars.githubusercontent.com/u/2674469?v=3","gravatar_id":"","url":"https://api.github.com/users/stefanandres","html_url":"https://github.com/stefanandres","followers_url":"https://api.github.com/users/stefanandres/followers","following_url":"https://api.github.com/users/stefanandres/following{/other_user}","gists_url":"https://api.github.com/users/stefanandres/gists{/gist_id}","starred_url":"https://api.github.com/users/stefanandres/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stefanandres/subscriptions","organizations_url":"https://api.github.com/users/stefanandres/orgs","repos_url":"https://api.github.com/users/stefanandres/repos","events_url":"https://api.github.com/users/stefanandres/events{/privacy}","received_events_url":"https://api.github.com/users/stefanandres/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-09-16T12:33:20Z","updated_at":"2015-09-16T12:33:20Z","closed_at":null,"body":"When sensu-client is running local checks which might take longer (or never end, because of a bug) the sensu-client cannot restart:\r\n\r\nStopping the client fails:\r\n```\r\nroot@xxx- /etc/sensu/conf.d/checks(controller-slave) $ /etc/init.d/sensu-client restart\r\n * Stopping sensu-client                                                                                                                                                        * Timed out waiting for sensu-client to stop\r\n                                                                                                                                                                        [fail]\r\nroot@xxx- /etc/sensu/conf.d/checks(controller-slave) $ ps auxw|grep sensu-cl\r\nsensu     5099  0.2  0.0 1440020 27052 ?       Sl   13:31   0:06 /opt/sensu/embedded/bin/ruby /opt/sensu/bin/sensu-client -b -c /etc/sensu/config.json -d /etc/sensu/conf.d -e /etc/sensu/extensions -p /var/run/sensu/sensu-client.pid -l /var/log/sensu/sensu-client.log -L info\r\n```\r\n\r\nKilling the sensu-client only works with -9. When doing so, it cannot start properly again:\r\n```\r\nroot@xxx- /etc/sensu/conf.d/checks(controller-slave) $ killall -9 sensu-client\r\nsensu-client: no process found\r\nroot@xxx- /etc/sensu/conf.d/checks(controller-slave) $ /etc/init.d/sensu-client start\r\n * Starting sensu-client                                                                                                                                                [fail]\r\n```\r\n\r\nThe Log-File only says\r\n```\r\n/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:859:in `open_udp_socket': no datagram socket (RuntimeError)\r\n        from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:859:in `open_datagram_socket'\r\n        from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.20.3/lib/sensu/client/process.rb:355:in `setup_sockets'\r\n        from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.20.3/lib/sensu/client/process.rb:413:in `start'\r\n        from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.20.3/lib/sensu/client/process.rb:19:in `block in run'\r\n        from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in `call'\r\n        from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in `run_machine'\r\n        from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in `run'\r\n        from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.20.3/lib/sensu/client/process.rb:18:in `run'\r\n        from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.20.3/bin/sensu-client:10:in `<top (required)>'\r\n        from /opt/sensu/bin/sensu-client:23:in `load'\r\n        from /opt/sensu/bin/sensu-client:23:in `<main>'\r\n```\r\n\r\nWhen adding debug Output, it says port already in use for 3030/UDP\r\n```\r\nroot@xxx- /etc/sensu/conf.d/checks(controller-slave) $ netstat -ntulp|grep 3030\r\nudp        0      0 127.0.0.1:3030          0.0.0.0:*                           602/sh\r\nroot@xxx- /etc/sensu/conf.d/checks(controller-slave) $ kill -9^C\r\nroot@xxx- /etc/sensu/conf.d/checks(controller-slave) $ ps auxw|grep 602\r\nsensu      602  0.0  0.0   4448   748 ?        S    13:42   0:00 sh -c sudo /usr/lib/nagios/plugins/check_yyy 3\r\nroot@xxx- /etc/sensu/conf.d/checks(controller-slave) $ kill -9 602\r\nroot@xxx- /etc/sensu/conf.d/checks(controller-slave) $ netstat -ntulp|grep 3030\r\nudp        0      0 127.0.0.1:3030          0.0.0.0:*                           603/sudo\r\nroot@xxx- /etc/sensu/conf.d/checks(controller-slave) $ kill -9 603\r\nroot@xxx- /etc/sensu/conf.d/checks(controller-slave) $ netstat -ntulp|grep 3030\r\nroot@xxx- /etc/sensu/conf.d/checks(controller-slave) $ /etc/init.d/sensu-client start\r\n * Starting sensu-client                                                                                                                                                [ OK ]\r\nroot@xxx- /etc/sensu/conf.d/checks(controller-slave) $\r\n```\r\n\r\nSo killing all childprocesses is the only way. I consider that a bug, I would think it should a) abandon the childs or b) kill the childs c) give a proper log message\r\n"}}