{"action":"created","issue":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/17456","labels_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/17456/labels{/name}","comments_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/17456/comments","events_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/17456/events","html_url":"https://github.com/kubernetes/kubernetes/issues/17456","id":117675603,"number":17456,"title":"Deploying applications to Kube should take into account security and the permissions necessary to accomplish their goals","user":{"login":"smarterclayton","id":1163175,"avatar_url":"https://avatars.githubusercontent.com/u/1163175?v=3","gravatar_id":"","url":"https://api.github.com/users/smarterclayton","html_url":"https://github.com/smarterclayton","followers_url":"https://api.github.com/users/smarterclayton/followers","following_url":"https://api.github.com/users/smarterclayton/following{/other_user}","gists_url":"https://api.github.com/users/smarterclayton/gists{/gist_id}","starred_url":"https://api.github.com/users/smarterclayton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/smarterclayton/subscriptions","organizations_url":"https://api.github.com/users/smarterclayton/orgs","repos_url":"https://api.github.com/users/smarterclayton/repos","events_url":"https://api.github.com/users/smarterclayton/events{/privacy}","received_events_url":"https://api.github.com/users/smarterclayton/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/area/app-config-deployment","name":"area/app-config-deployment","color":"fb6420"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/area/security","name":"area/security","color":"d4c5f9"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/team/redhat","name":"team/redhat","color":"d2b48c"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":22,"created_at":"2015-11-18T20:34:44Z","updated_at":"2015-11-19T22:48:49Z","closed_at":null,"body":"We are starting to discuss formalizing / defining how an application is deployed onto Kube (via config files, examples, deployment-manager, Helm, and OpenShift templates).  Many of those applications are going to be isolated with no external dependencies.  That's the easy problem.\r\n\r\nThe next problem is how those applications (or Kube extensions running as extensions) are:\r\n\r\n1. Authorized to talk to the Kube API or extension API to:\r\n  1. ... read resources in their namespace (cassandra example reading service endpoints)\r\n  2. ... change resources in their namespace (pod that updates status of resources, like a local controller)\r\n  3. ... read resources across all namespaces (heapster and logging integrations need to read pods)\r\n  4. ... change some resources across all namespaces (controllers running in pods)\r\n2. Authorized to talk to other namespaces\r\n  1. ... via the network\r\n  2. ... via a proxy endpoint\r\n3. Are given secrets that allow them to securely connect to other services in Kube, in other clusters, or outside of clusters\r\n  1. We can inject secrets, but we don't make it easy to tie secrets and config to services and access them via apps in a way that is not \"copy your secrets to every other place\"\r\n4. Are given secrets that allow them to securely serve for other services to connect to\r\n  1. ... it is hard to run an app that protects itself via SSL on the cluster (requires certs that are generated, but the certs need to be signed by the proper authority, but the cluster controls internal DNS, but the external DNS is not visible to the cluster today for ingress or load balancers)\r\n5. Are properly authorized to do dangerous things on the cluster, including but not limited to:\r\n  1. ... accessing hostPath (root access on node)\r\n  2. ... accessing pid / net / ipc / privileged flags (root access on node in some cases)\r\n  3. ... defining arbitrary volumes in their pod spec which can mount any AWS / GCE / Azure host in the cluster (because the node has permissions to attach disks to itself, and the kubelet accepts the pod, the kubelet is a confused deputy)\r\n  4. ... running images that other containers have access to (by specifying PullIfNotPresent and guessing the other images name) or that come from arbitrary sources (unscreened code)\r\n\r\nCurrently, Kube has limited protection / solutions for any of the above.  A few are being added today as a result of work done in OpenShift or in the community:\r\n\r\n* PodSecurity policy protects clusters from random pods getting root on boxes #7893\r\n* Authn/z in the cluster would define a way for applications to request 1 #17440\r\n* Network isolation is being discussed in #15465\r\n* Service accounts are in the code, which allow roles to be assigned to pods to enable 1, but you can't assign roles\r\n  * Service accounts can control what secrets are visible to containers via an optional admission controller\r\n* Discussion of generating secrets for apps automatically\r\n\r\nTo properly deploy applications to Kube, we will need to separate the \"what\" of the thing being installed cluster with the \"what is it allowed to do\".  Running random code from an upstream repo on your infrastructure that also has access to your internal network generally leads to bad results.  Extension components to Kube also need this clarity - it is insufficient for them to expect to run as root on the cluster.\r\n\r\nWe should formalize:\r\n\r\n* How applications request and are granted the permissions they need to operate on the cluster\r\n* How applications request and are granted the access to other applications on the cluster\r\n* How extension applications (logging, metrics, extension API groups) differ from regular applications\r\n* How our clients know which actions are escalation actions (make kubectl confused deputy) and which are not\r\n* How simple applications can be given complex permissions without requiring the details of each cluster to be defined\r\n\r\nWe should make all of the above items easy to layer *on top* of application definitions.\r\n\r\n@bgrant0607 @erictune this is my tracking issue for \"it's not enough to just deploy apps, they have to not take root on your cluster when they do it\"."},"comment":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/comments/158225137","html_url":"https://github.com/kubernetes/kubernetes/issues/17456#issuecomment-158225137","issue_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/17456","id":158225137,"user":{"login":"smarterclayton","id":1163175,"avatar_url":"https://avatars.githubusercontent.com/u/1163175?v=3","gravatar_id":"","url":"https://api.github.com/users/smarterclayton","html_url":"https://github.com/smarterclayton","followers_url":"https://api.github.com/users/smarterclayton/followers","following_url":"https://api.github.com/users/smarterclayton/following{/other_user}","gists_url":"https://api.github.com/users/smarterclayton/gists{/gist_id}","starred_url":"https://api.github.com/users/smarterclayton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/smarterclayton/subscriptions","organizations_url":"https://api.github.com/users/smarterclayton/orgs","repos_url":"https://api.github.com/users/smarterclayton/repos","events_url":"https://api.github.com/users/smarterclayton/events{/privacy}","received_events_url":"https://api.github.com/users/smarterclayton/received_events","type":"User","site_admin":false},"created_at":"2015-11-19T22:48:49Z","updated_at":"2015-11-19T22:48:49Z","body":"> Do we think the normal pattern will be to create a new namespace to install an application into, or to install an application into an existing namespace?\r\n\r\nI don't think there's one or the other so - namespaces are an organizational concept, so different people will organize differently.  However, we can use your PROs and CONs to guide recommendations about how to organize.  Online actually doesn't assume namespace corresponds to account - we always want to enable multiple namespaces for users.\r\n\r\nUsing short names is good - having short names represent \"the things you care about\" is best.  The best packages fit neatly into a namespace in their intended function.  Generally, you don't need 2 cassandra clusters from the same app, or they may not be managed or accessed by the same applications.  That means the service name might be \"cassandra\" much of the time.  Occasionally, you'd have \"cassandra-logs\" and \"cassandra-customer\" - in those cases, someone has to take the package and parameterize it (hence why OpenShift templates are often flexible to allowing name to be parameterized).  \r\n\r\nWorth remembering, cassandra / redis / mysql aren't really applications - they're tools that are used to build applications.  Applications are often composites of these pieces, sometimes under the same team, sometimes not."}}