{"action":"created","issue":{"url":"https://api.github.com/repos/toptal/chewy/issues/163","labels_url":"https://api.github.com/repos/toptal/chewy/issues/163/labels{/name}","comments_url":"https://api.github.com/repos/toptal/chewy/issues/163/comments","events_url":"https://api.github.com/repos/toptal/chewy/issues/163/events","html_url":"https://github.com/toptal/chewy/issues/163","id":71441755,"number":163,"title":"Updates during zero-downtime reindexing","user":{"login":"fpbouchard","id":102135,"avatar_url":"https://avatars.githubusercontent.com/u/102135?v=3","gravatar_id":"","url":"https://api.github.com/users/fpbouchard","html_url":"https://github.com/fpbouchard","followers_url":"https://api.github.com/users/fpbouchard/followers","following_url":"https://api.github.com/users/fpbouchard/following{/other_user}","gists_url":"https://api.github.com/users/fpbouchard/gists{/gist_id}","starred_url":"https://api.github.com/users/fpbouchard/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fpbouchard/subscriptions","organizations_url":"https://api.github.com/users/fpbouchard/orgs","repos_url":"https://api.github.com/users/fpbouchard/repos","events_url":"https://api.github.com/users/fpbouchard/events{/privacy}","received_events_url":"https://api.github.com/users/fpbouchard/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-04-28T00:55:28Z","updated_at":"2015-04-28T00:58:26Z","closed_at":null,"body":"Hi,\r\n\r\nI'm wondering how to manage the following situation: some of our indexes take a lot of time to reindex (hours), and while this is not really a downtime issue (thanks to zero-downtime reindexing), the issue is that our app is indeed not down during the reindexation, so we have some updates that are not being applied to the index being reindexed (for objects already reindexed). So once the app is back online, I still have to call a global `update` on the index so any updates that I've missed will be applied. That doubles the deployment time and load of reindexing.\r\n\r\nI've thought of a few tricks we could do, one using an additional sidekiq queue that I could pause during the reindexation and in which I'd only queue if a reindexation is ongoing. Then, once the data has been reindexed, and sidekiq is restarted, I would unpause this queue which would replay all updates that were queued during the reindexation. I would need to keep the index status _somewhere_ (I'm thinking of doing this in Redis).\r\n\r\nDo you have other ideas?\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/toptal/chewy/issues/comments/96861702","html_url":"https://github.com/toptal/chewy/issues/163#issuecomment-96861702","issue_url":"https://api.github.com/repos/toptal/chewy/issues/163","id":96861702,"user":{"login":"fpbouchard","id":102135,"avatar_url":"https://avatars.githubusercontent.com/u/102135?v=3","gravatar_id":"","url":"https://api.github.com/users/fpbouchard","html_url":"https://github.com/fpbouchard","followers_url":"https://api.github.com/users/fpbouchard/followers","following_url":"https://api.github.com/users/fpbouchard/following{/other_user}","gists_url":"https://api.github.com/users/fpbouchard/gists{/gist_id}","starred_url":"https://api.github.com/users/fpbouchard/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fpbouchard/subscriptions","organizations_url":"https://api.github.com/users/fpbouchard/orgs","repos_url":"https://api.github.com/users/fpbouchard/repos","events_url":"https://api.github.com/users/fpbouchard/events{/privacy}","received_events_url":"https://api.github.com/users/fpbouchard/received_events","type":"User","site_admin":false},"created_at":"2015-04-28T00:58:26Z","updated_at":"2015-04-28T00:58:26Z","body":"I think I need to add some details on our current deployment strategy. When we deloy, we start by reindexing using the new code, while the app is running using the old code. We do this so everything still works for our customers, there is indeed no downtime. Then, as soon as the reindexation is over, we restart the apps with the new code."}}