{"action":"created","issue":{"url":"https://api.github.com/repos/openshift/origin/issues/3333","labels_url":"https://api.github.com/repos/openshift/origin/issues/3333/labels{/name}","comments_url":"https://api.github.com/repos/openshift/origin/issues/3333/comments","events_url":"https://api.github.com/repos/openshift/origin/issues/3333/events","html_url":"https://github.com/openshift/origin/issues/3333","id":89604410,"number":3333,"title":"Registry pruning improvements","user":{"login":"ncdc","id":354,"avatar_url":"https://avatars.githubusercontent.com/u/354?v=3","gravatar_id":"","url":"https://api.github.com/users/ncdc","html_url":"https://github.com/ncdc","followers_url":"https://api.github.com/users/ncdc/followers","following_url":"https://api.github.com/users/ncdc/following{/other_user}","gists_url":"https://api.github.com/users/ncdc/gists{/gist_id}","starred_url":"https://api.github.com/users/ncdc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ncdc/subscriptions","organizations_url":"https://api.github.com/users/ncdc/orgs","repos_url":"https://api.github.com/users/ncdc/repos","events_url":"https://api.github.com/users/ncdc/events{/privacy}","received_events_url":"https://api.github.com/users/ncdc/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/openshift/origin/labels/component/imageregistry","name":"component/imageregistry","color":"0052cc"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-06-19T16:24:26Z","updated_at":"2015-06-30T15:31:45Z","closed_at":null,"body":"When pruning images and layers, the `oadm prune images` command:\r\n\r\n1. determines which images and layers can be pruned\r\n1. removes images and updates image streams in etcd\r\n1. makes multiple http requests to the registry to prune layers and signatures\r\n\r\nInstead of sending multiple http requests to the registry, it would be better to store information about which layers and signatures the registry needs to prune. The registry could periodically query OpenShift for that data and act on it.\r\n\r\nThis has multiple benefits:\r\n\r\n1. a potential significant reduction in the number of http requests needed to prune content from the registry\r\n1. less divergence from the upstream registry code base - we had to modify its internals pretty heavily to support deleting signatures and deleting layers via http requests\r\n\r\nHA note - if there are multiple replicas of the registry pod, it is probably a good idea to have the registry's pruning task scheduled a bit randomly. For example, replica 1 might run the job at 13 minutes after the hour, while replica 2 might run it at 27 minutes after the hour."},"comment":{"url":"https://api.github.com/repos/openshift/origin/issues/comments/117229753","html_url":"https://github.com/openshift/origin/issues/3333#issuecomment-117229753","issue_url":"https://api.github.com/repos/openshift/origin/issues/3333","id":117229753,"user":{"login":"smarterclayton","id":1163175,"avatar_url":"https://avatars.githubusercontent.com/u/1163175?v=3","gravatar_id":"","url":"https://api.github.com/users/smarterclayton","html_url":"https://github.com/smarterclayton","followers_url":"https://api.github.com/users/smarterclayton/followers","following_url":"https://api.github.com/users/smarterclayton/following{/other_user}","gists_url":"https://api.github.com/users/smarterclayton/gists{/gist_id}","starred_url":"https://api.github.com/users/smarterclayton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/smarterclayton/subscriptions","organizations_url":"https://api.github.com/users/smarterclayton/orgs","repos_url":"https://api.github.com/users/smarterclayton/repos","events_url":"https://api.github.com/users/smarterclayton/events{/privacy}","received_events_url":"https://api.github.com/users/smarterclayton/received_events","type":"User","site_admin":false},"created_at":"2015-06-30T15:31:45Z","updated_at":"2015-06-30T15:31:45Z","body":"On Jun 30, 2015, at 11:27 AM, Andy Goldstein <notifications@github.com>\nwrote:\n\nAs a counter argument - why not just periodically read all layers not\nreferenced by an image manifest or link as a separate call?\n\nIn the registry's code? I'd like to avoid walking the registry's storage if\npossible, since it may not be an efficient or cheap operation for storage\nbackends like S3.\n\nI am working on a patch to at least be able to delete the blobs, relieving\nthe storage burden. But it won't handle deleting the layer links.\n\nBut don't we eventually have to have this check?  What happens when a\ncustomer runs into this?  We're not going to write perfect storage code, so\nwhy not write the fundamental \"make this clean\".  And with S3, don't we\nhave at least basic sweep functionality?\n\n—\nReply to this email directly or view it on GitHub\n<https://github.com/openshift/origin/issues/3333#issuecomment-117228589>.\n"}}