{"action":"created","issue":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/14870","labels_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/14870/labels{/name}","comments_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/14870/comments","events_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/14870/events","html_url":"https://github.com/kubernetes/kubernetes/issues/14870","id":109175776,"number":14870,"title":"kube-proxy keeps adding duplicate iptables rules","user":{"login":"AdrianAbraham","id":13833219,"avatar_url":"https://avatars.githubusercontent.com/u/13833219?v=3","gravatar_id":"","url":"https://api.github.com/users/AdrianAbraham","html_url":"https://github.com/AdrianAbraham","followers_url":"https://api.github.com/users/AdrianAbraham/followers","following_url":"https://api.github.com/users/AdrianAbraham/following{/other_user}","gists_url":"https://api.github.com/users/AdrianAbraham/gists{/gist_id}","starred_url":"https://api.github.com/users/AdrianAbraham/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AdrianAbraham/subscriptions","organizations_url":"https://api.github.com/users/AdrianAbraham/orgs","repos_url":"https://api.github.com/users/AdrianAbraham/repos","events_url":"https://api.github.com/users/AdrianAbraham/events{/privacy}","received_events_url":"https://api.github.com/users/AdrianAbraham/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/team/cluster","name":"team/cluster","color":"d2b48c"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-09-30T20:45:04Z","updated_at":"2015-10-12T02:37:20Z","closed_at":null,"body":"I noticed that while connections between pods were usually nearly instant, sometimes they took a few seconds and they sometimes hung altogether, so I started poking around. I quickly found that `kube-proxy` is taking up between 80% - 100% of a CPU on each system that it's running on. The logs weren't very interesting, but then I checked the iptables rules:\r\n```\r\n# sudo iptables -L -t nat | wc -l\r\n164757\r\n```\r\n\r\nThe rules themselves:\r\n```\r\n# sudo iptables -L -t nat\r\nChain PREROUTING (policy ACCEPT 0 packets, 0 bytes)\r\n pkts bytes target     prot opt in     out     source               destination         \r\n    0     0 KUBE-PORTALS-CONTAINER  all  --  any    any     anywhere             anywhere            /* handle ClusterIPs; NOTE: this must be before the NodePort rules */ \r\n    1    60 KUBE-PORTALS-CONTAINER  all  --  any    any     anywhere             anywhere            /* handle ClusterIPs; NOTE: this must be before the NodePort rules */ \r\n    4   258 KUBE-PORTALS-CONTAINER  all  --  any    any     anywhere             anywhere            /* handle ClusterIPs; NOTE: this must be before the NodePort rules */ \r\n    6   547 KUBE-PORTALS-CONTAINER  all  --  any    any     anywhere             anywhere            /* handle ClusterIPs; NOTE: this must be before the NodePort rules */ \r\n   12  1632 KUBE-PORTALS-CONTAINER  all  --  any    any     anywhere             anywhere            /* handle ClusterIPs; NOTE: this must be before the NodePort rules */ \r\n...\r\n```\r\n\r\nThe `OUTPUT` chain is just as bad as the `PREROUTING` chain. The `KUBE-*` chains are all sane, having one rule per service port.\r\n\r\nFor now, I've set up a cron job to periodically flush `PREROUTING` and `OUTPUT` (and recreate the proper 4 rules to hopefully avoid any downtime), but this is obviously a poor workaround.\r\n\r\nI'm running release 1.0.6 on CentOS 6."},"comment":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/comments/147276368","html_url":"https://github.com/kubernetes/kubernetes/issues/14870#issuecomment-147276368","issue_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/14870","id":147276368,"user":{"login":"cnzhangquan","id":5462876,"avatar_url":"https://avatars.githubusercontent.com/u/5462876?v=3","gravatar_id":"","url":"https://api.github.com/users/cnzhangquan","html_url":"https://github.com/cnzhangquan","followers_url":"https://api.github.com/users/cnzhangquan/followers","following_url":"https://api.github.com/users/cnzhangquan/following{/other_user}","gists_url":"https://api.github.com/users/cnzhangquan/gists{/gist_id}","starred_url":"https://api.github.com/users/cnzhangquan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cnzhangquan/subscriptions","organizations_url":"https://api.github.com/users/cnzhangquan/orgs","repos_url":"https://api.github.com/users/cnzhangquan/repos","events_url":"https://api.github.com/users/cnzhangquan/events{/privacy}","received_events_url":"https://api.github.com/users/cnzhangquan/received_events","type":"User","site_admin":false},"created_at":"2015-10-12T02:37:20Z","updated_at":"2015-10-12T02:37:20Z","body":"looks i am suffering in the same issue"}}