{"action":"created","issue":{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/9995","labels_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/9995/labels{/name}","comments_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/9995/comments","events_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/9995/events","html_url":"https://github.com/GoogleCloudPlatform/kubernetes/issues/9995","id":89164777,"number":9995,"title":"Allow Service NodePorts on low ports","user":{"login":"thockin","id":5595220,"avatar_url":"https://avatars.githubusercontent.com/u/5595220?v=3","gravatar_id":"","url":"https://api.github.com/users/thockin","html_url":"https://github.com/thockin","followers_url":"https://api.github.com/users/thockin/followers","following_url":"https://api.github.com/users/thockin/following{/other_user}","gists_url":"https://api.github.com/users/thockin/gists{/gist_id}","starred_url":"https://api.github.com/users/thockin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/thockin/subscriptions","organizations_url":"https://api.github.com/users/thockin/orgs","repos_url":"https://api.github.com/users/thockin/repos","events_url":"https://api.github.com/users/thockin/events{/privacy}","received_events_url":"https://api.github.com/users/thockin/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/labels/kind/design","name":"kind/design","color":"c7def8"},{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/labels/priority/P2","name":"priority/P2","color":"fbca04"},{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/labels/team/cluster","name":"team/cluster","color":"d2b48c"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":12,"created_at":"2015-06-18T00:48:21Z","updated_at":"2015-06-19T00:13:46Z","closed_at":null,"body":"To capture some of the discussion.\r\n\r\nGripe: Service NodePorts can only be in a \"special\" port range (apiserver flag).  This is annoying for people who want to expose a service on port 80 (for example) but instead have to use something like 30986.\r\n\r\nRationale for this behavior:\r\n  1. We don't want service node ports to tromp on real ports used by the node (e.g. 22). \r\n  2. We don't want service node ports to tromp on pod host ports.\r\n  3. We don't want to randomly allocate someone port 80 or 443 or 22.\r\n\r\nProposed compromise:  Allow NodePorts on low ports with a bunch of caveats.\r\n\r\nTo address rationale points (1) and (2): Make kube-proxy open (and hold open) ports that it is using.  This will prevent it from using port 22 (for example) for a service node port.\r\n\r\nTo address rationale point (3): Use the flag-configured range for random allocations, but allow users to request any port they want.\r\n\r\nCaveats and analysis:\r\n\r\n1) Error reporting is not good.  We do not know about what non-container stuff is using ports on the host and we do not have an easy way for the API server to allocate ports between pods and services.  Not easy to resolve.  The implication of this is that port-in-use errors will only be detected by kube-proxy when it tries to open the port on the node.  Sending events for this is possible, but not great (one event from every node) and no other node-level error gets a kube-proxy event.  Net result: user asked for a service on node port 22, and it just doesn't work and they have no idea why.\r\n\r\n2) Doing dual-ranges (use the flag range for allocations but allow the whole range to be used) is non-trivial and has to be plumbed through more code than I am comfortable with at this point.  The implication of not doing this is that sometimes people will get allocated a port that happens to be 22 and can never work.  Combined with caveat (1) this is really unpleasant.  We could do some REALLY hacky things like just retry the random allocation if it is not in the flagged range.  This avoids plumbing the dual-range logic down, but is embarassingly ugly.  I am ashamed to have suggested it.\r\n\r\n3) Holding ports open is pretty easy and we should do this anyway.  The error reporting properties are still not good.\r\n\r\n\r\nSummary: I am unconvinced that this is worthwhile to rush into v1.  We still have the option of baking a container that receives traffic on a HostPort and redirects it to a service.  I have a demo of using socat to do this (simulating a pod manually):\r\n\r\n```\r\nroot@kubernetes-minion-nnx6:/home/thockin# docker run -d --dns=10.0.0.10 --dns-search=default.svc.cluster.local -p 8080:93 thockin/hostport-to-service 93 hostnames2:80\r\n1dcc1e94c30834290ae243ac298c6699b2a3348fc014b4b77ae34c13ead44854\r\n\r\nroot@kubernetes-minion-nnx6:/home/thockin# curl localhost:8080\r\nhostnames2-gxt4f\r\n```\r\n\r\n@brendandburns @justinsb @eparis"},"comment":{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/comments/113323427","html_url":"https://github.com/GoogleCloudPlatform/kubernetes/issues/9995#issuecomment-113323427","issue_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/9995","id":113323427,"user":{"login":"eparis","id":8093535,"avatar_url":"https://avatars.githubusercontent.com/u/8093535?v=3","gravatar_id":"","url":"https://api.github.com/users/eparis","html_url":"https://github.com/eparis","followers_url":"https://api.github.com/users/eparis/followers","following_url":"https://api.github.com/users/eparis/following{/other_user}","gists_url":"https://api.github.com/users/eparis/gists{/gist_id}","starred_url":"https://api.github.com/users/eparis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eparis/subscriptions","organizations_url":"https://api.github.com/users/eparis/orgs","repos_url":"https://api.github.com/users/eparis/repos","events_url":"https://api.github.com/users/eparis/events{/privacy}","received_events_url":"https://api.github.com/users/eparis/received_events","type":"User","site_admin":false},"created_at":"2015-06-19T00:13:46Z","updated_at":"2015-06-19T00:13:46Z","body":"@thockin some people do have an external load balancer to terminate ssl and load balance to pods, which operates with access to the overlay network. Those people have better scalability for sure. But we provide a simple 'load balancer', running on ${some_node_ips} :80 :443 (nodePort would mean $all_node_ips).\r\n\r\nWe have no controller for nodePort yet, but yes, the plan is to outright reject a service if a 'normal' user specified anything other than 0.\r\n\r\nTo be honest, it does make me wonder what we do today if someone asks for hostPort=22.  I think we just outright reject hostPort, but I don't know for sure..."}}