{"action":"created","issue":{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/6848","labels_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/6848/labels{/name}","comments_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/6848/comments","events_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/6848/events","html_url":"https://github.com/GoogleCloudPlatform/kubernetes/issues/6848","id":68568415,"number":6848,"title":"Running kubelet in a container: mounts","user":{"login":"pmorie","id":366488,"avatar_url":"https://avatars.githubusercontent.com/u/366488?v=3","gravatar_id":"","url":"https://api.github.com/users/pmorie","html_url":"https://github.com/pmorie","followers_url":"https://api.github.com/users/pmorie/followers","following_url":"https://api.github.com/users/pmorie/following{/other_user}","gists_url":"https://api.github.com/users/pmorie/gists{/gist_id}","starred_url":"https://api.github.com/users/pmorie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pmorie/subscriptions","organizations_url":"https://api.github.com/users/pmorie/orgs","repos_url":"https://api.github.com/users/pmorie/repos","events_url":"https://api.github.com/users/pmorie/events{/privacy}","received_events_url":"https://api.github.com/users/pmorie/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/labels/priority/design","name":"priority/design","color":"c7def8"},{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/labels/team/node","name":"team/node","color":"d2b48c"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":18,"created_at":"2015-04-15T04:10:45Z","updated_at":"2015-04-15T21:29:26Z","closed_at":null,"body":"Currently when the kubelet is run in a container, the mounts that the kubelet performs are not visible to the containers in pods because the kubelet runs in its own mount namespace with a private shared-subtree mode.  In order to run the kubelet in a container, we must find a way to have the kubelet perform mounts in the host's mount namespace.\r\n\r\n#### How to do the mount\r\n\r\nThe mount must be performed from the root mount namespace.  Ideally, it would be possible to run a container in the host's root mount namespace with something like `docker run --mnt='host'`.  However, this is currently not possible, although it has been [requested](https://github.com/docker/docker/issues/10088).  That being the case, one option is the [super privileged container](http://developerblog.redhat.com/2014/11/06/introducing-a-super-privileged-container-concept/) concept.  The basic formula for a super privileged container to execute a command in the host's mount namespace is:\r\n\r\n1.  The container's filesystem must have the `nsenter` bits\r\n2.  Bind mount the host's filesystem into the container wholesale, viz: `docker run -v /:/host`\r\n3.  Read the value of the host's mount namespace from `/proc`:\r\n    `host_mnt_ns=$(</host/proc/1/ns/mnt)`\r\n4.  Execute a command in the host's mount namespace by execing a subprocess:\r\n    `nsenter --mount=$host_mnt_ns <some command>`\r\n5.  A caveat: once you enter the host's mount namespace, you lose what's mounted in the container's mount namespace; the command nsenter execs will be relative to the host's filesystem.\r\n\r\n#### Wrinkle: mount helpers\r\n\r\nWhen `mount -t <fstype>` is invoked, it looks for a mount helper named `mount.<fstype>` to delegate the mount operation to. In order for a containerized kubelet to be able to mount all filesystem types in the manner described here, the mount helpers would need to be installed on the host.\r\n\r\n#### Factoring kubernetes\r\n\r\nCurrently the volume plugins use the `mount.Interface` interface to perform mounts, which serves our purposes for keeping volume plugin code orthogonal from what is performing the mount.  We can make an implementation of this interface which handles execing a subprocess that nsenter's the host's mount namespace to perform the mount without requiring any changes to the interface itself.\r\n\r\nCurrently the volume plugins create the instance of `mount.Interface` directly using the `mount.New()` method.  In order to facilitate injecting an `nsenter`ing mounter, we could instead provide volume plugins with a new `MounterFactory` (via the `Host` interface) which they can use to get a mounter:\r\n\r\n```go\r\npackage mount\r\n\r\ntype MounterFactory interface {\r\n  New() mount.Interface\r\n}\r\n```\r\n\r\n```go\r\npackage volume\r\n\r\ntype Host interface {\r\n  // other methods omitted\r\n  MounterFactory() mount.Factory\r\n}\r\n```\r\n\r\nThe `MounterFactory` should be an exported field on the Kubelet so that the creator of the Kubelet can provide whatever `MounterFactory` implementation they want.  This will facilitate using alternate implementations in downstream projects, integration tests, etc."},"comment":{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/comments/93574669","html_url":"https://github.com/GoogleCloudPlatform/kubernetes/issues/6848#issuecomment-93574669","issue_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/6848","id":93574669,"user":{"login":"eparis","id":8093535,"avatar_url":"https://avatars.githubusercontent.com/u/8093535?v=3","gravatar_id":"","url":"https://api.github.com/users/eparis","html_url":"https://github.com/eparis","followers_url":"https://api.github.com/users/eparis/followers","following_url":"https://api.github.com/users/eparis/following{/other_user}","gists_url":"https://api.github.com/users/eparis/gists{/gist_id}","starred_url":"https://api.github.com/users/eparis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eparis/subscriptions","organizations_url":"https://api.github.com/users/eparis/orgs","repos_url":"https://api.github.com/users/eparis/repos","events_url":"https://api.github.com/users/eparis/events{/privacy}","received_events_url":"https://api.github.com/users/eparis/received_events","type":"User","site_admin":false},"created_at":"2015-04-15T21:29:26Z","updated_at":"2015-04-15T21:29:26Z","body":"but isn't updating such a container easier than updating binaries in your host/vm (which also have to track some upstream source)?"}}