{"action":"created","issue":{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/3585","labels_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/3585/labels{/name}","comments_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/3585/comments","events_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/3585/events","html_url":"https://github.com/GoogleCloudPlatform/kubernetes/issues/3585","id":54638798,"number":3585,"title":"Places for hooks","user":{"login":"erictune","id":8105590,"avatar_url":"https://avatars.githubusercontent.com/u/8105590?v=3","gravatar_id":"","url":"https://api.github.com/users/erictune","html_url":"https://github.com/erictune","followers_url":"https://api.github.com/users/erictune/followers","following_url":"https://api.github.com/users/erictune/following{/other_user}","gists_url":"https://api.github.com/users/erictune/gists{/gist_id}","starred_url":"https://api.github.com/users/erictune/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/erictune/subscriptions","organizations_url":"https://api.github.com/users/erictune/orgs","repos_url":"https://api.github.com/users/erictune/repos","events_url":"https://api.github.com/users/erictune/events{/privacy}","received_events_url":"https://api.github.com/users/erictune/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-01-16T23:34:30Z","updated_at":"2015-01-17T00:56:12Z","closed_at":null,"body":"There are lots of reasons why the \"system\" wants to look at an object, like a pod, and modify and/or act on it.  There are also a number of places where we can put in \"hooks\" for these actions.  We'll end up with a better system in the long run, I think, if we put some thought into what hooks to use in what situations.  \r\n\r\n### Places for hooks\r\n 1. kubectl, other clients\r\n 1. proxy in front of apiserver\r\n 1. apiserver \r\n 1. post-apiserver\r\n\r\nWithin apiserver, there are both hardcoded actions on objects, and extensible ones (such as what @derekwaynecarr has implemented with \"admission control\", #3472 and #3319).\r\n\r\nThe last one is a somewhat new concept, so it deserves a bit of explanation.  After a pod is POST'ed to the apiserver, and persisted to etcd, some other component (a different process than apiserver), which is watching for new objects, will see it, and act on it and/or modify it.  It should not start running on a Kubelet until all the things that need to act on it have done so.  @smarterclayton called this a \"finalizer\".  The scheduler can be viewed as a finalizer that takes pods that have everything except a HostIP set, and it sets the HostIP. \r\n\r\n### Pros/cons of each hook location\r\n - kubectl, other clients\r\n    - hooks won't be mandatory.  users could just use a different client.  \r\n    - updating all clients to change behavior is very difficult.\r\n    - can interact with users local directory, templates, etc...\r\n - proxy\r\n   - can be made manadatory, \r\n   - may be a pain to setup?\r\n - apiserver\r\n    - can prevent object from ever being persisted to storage\r\n    - currently only place where atomic read-then-write is possible.\r\n    - adding an action requires modifying apiserver binary, \r\n    - adding an action requires checking in changes to kubernetes project or maintaining a branch.\r\n - post-apiserver (finalizer)\r\n  - easy for users to customize by maintaining their own component, in separate repository if necessary\r\n  - can't \"reject\" an object.  hard for user to understand if the POST succeeds but the pod never runs.\r\n\r\n### Use cases for hooked actions\r\n  1. Set default resource limits on pods \r\n  1. Reject pods with resource limits above or below sane levels. #3057 \r\n  1. Limit aggregate amount of resources used by a tenant, or by objects matching some selector. #3057 \r\n  1. Reject Pods with resource limits which are difficult shapes for the system to schedule (e.g. lots of ram and very little CPU or vice versa.)\r\n  1. Prevent creation of too many objects of any kind, either because it is obviously user error, or because it will hurt the system. #3057\r\n  1. Schedule pods to nodes (the existing scheduler)\r\n  1. setup network routes for pods (@pmorie working on something like this for OpenShift, I think)\r\n  1. custom allocator for IP addresses, vlans, etc (#3350) \r\n  1. something that distributes secrets to nodes via some side channel for the pods to use.\r\n  1. pod limit auto-adjuster\r\n\r\n### Which hooks to use for what\r\n\r\nSuggested guidelines for what hooks to use for what types of actions\r\n  - Prefer in apiserver if need to prevent some object from ever \"executing\"\r\n    - Prefer to say no as early as possible for debugging.\r\n    - e.g. resource quotas in apiserver\r\n  - Put it in the apiserver if the act of persisting of the object could be harmful\r\n    - protect apiserver storage space\r\n    - e.g. object quotas in apiserver\r\n  - Prefer finalizer (outside apiserver) otherwise. \r\n    - No need to recompile apiserver or commit to github.  \r\n    - Separation of responsibilities\r\n\r\n### Next steps\r\n\r\n  - debate the above proposal\r\n"},"comment":{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/comments/70346432","html_url":"https://github.com/GoogleCloudPlatform/kubernetes/issues/3585#issuecomment-70346432","issue_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/3585","id":70346432,"user":{"login":"bgrant0607","id":7725777,"avatar_url":"https://avatars.githubusercontent.com/u/7725777?v=3","gravatar_id":"","url":"https://api.github.com/users/bgrant0607","html_url":"https://github.com/bgrant0607","followers_url":"https://api.github.com/users/bgrant0607/followers","following_url":"https://api.github.com/users/bgrant0607/following{/other_user}","gists_url":"https://api.github.com/users/bgrant0607/gists{/gist_id}","starred_url":"https://api.github.com/users/bgrant0607/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bgrant0607/subscriptions","organizations_url":"https://api.github.com/users/bgrant0607/orgs","repos_url":"https://api.github.com/users/bgrant0607/repos","events_url":"https://api.github.com/users/bgrant0607/events{/privacy}","received_events_url":"https://api.github.com/users/bgrant0607/received_events","type":"User","site_admin":false},"created_at":"2015-01-17T00:56:11Z","updated_at":"2015-01-17T00:56:11Z","body":"Another option: webhooks"}}