{"action":"created","issue":{"url":"https://api.github.com/repos/mozilla/fxa-auth-db-server/issues/114","labels_url":"https://api.github.com/repos/mozilla/fxa-auth-db-server/issues/114/labels{/name}","comments_url":"https://api.github.com/repos/mozilla/fxa-auth-db-server/issues/114/comments","events_url":"https://api.github.com/repos/mozilla/fxa-auth-db-server/issues/114/events","html_url":"https://github.com/mozilla/fxa-auth-db-server/issues/114","id":54971309,"number":114,"title":"Problem with current patch solution","user":{"login":"chilts","id":3048,"avatar_url":"https://avatars.githubusercontent.com/u/3048?v=3","gravatar_id":"","url":"https://api.github.com/users/chilts","html_url":"https://github.com/chilts","followers_url":"https://api.github.com/users/chilts/followers","following_url":"https://api.github.com/users/chilts/following{/other_user}","gists_url":"https://api.github.com/users/chilts/gists{/gist_id}","starred_url":"https://api.github.com/users/chilts/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chilts/subscriptions","organizations_url":"https://api.github.com/users/chilts/orgs","repos_url":"https://api.github.com/users/chilts/repos","events_url":"https://api.github.com/users/chilts/events{/privacy}","received_events_url":"https://api.github.com/users/chilts/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/mozilla/fxa-auth-db-server/labels/in%20progress","name":"in progress","color":"ededed"}],"state":"closed","locked":false,"assignee":{"login":"jrgm","id":758162,"avatar_url":"https://avatars.githubusercontent.com/u/758162?v=3","gravatar_id":"","url":"https://api.github.com/users/jrgm","html_url":"https://github.com/jrgm","followers_url":"https://api.github.com/users/jrgm/followers","following_url":"https://api.github.com/users/jrgm/following{/other_user}","gists_url":"https://api.github.com/users/jrgm/gists{/gist_id}","starred_url":"https://api.github.com/users/jrgm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jrgm/subscriptions","organizations_url":"https://api.github.com/users/jrgm/orgs","repos_url":"https://api.github.com/users/jrgm/repos","events_url":"https://api.github.com/users/jrgm/events{/privacy}","received_events_url":"https://api.github.com/users/jrgm/received_events","type":"User","site_admin":false},"milestone":null,"comments":5,"created_at":"2015-01-21T02:54:29Z","updated_at":"2015-02-03T02:42:29Z","closed_at":"2015-02-03T02:18:10Z","body":"Currently we run into problems when we release to production if we have more than one database patch.\r\n\r\nIf a release has only one patch, then both dev and prod are patched correctly and there is no problem.\r\n\r\nThe problem with two (or more) patches goes like this:\r\n\r\n1. when the DB server starts up, it checks that the DB patch level is either `n` or `n+1` (ie. the current patch or the one after). This allows DB patching to occur prior to the release to make things easier. If the patch level is not `(n, n+1)` then the DB refuses to start.\r\n2. in the case where we have two patches, we can't patch the database up front with both patches since that could cause problems for the current stack if one of the DB servers needs to restart. It would fail due to the current db patch level being `n+2`.\r\n\r\nOne solution would be to merge the patches into a single patch, but this doesn't work if dev has already been patched with the first one.\r\n\r\nSome general solutions to this could be:\r\n\r\n1. do not fail if the database patch level is not what we expect and instead just log it (loudly). This relies on other current procedures to make sure only particular stacks run against the current database (patched or not).\r\n2. (or) instead of failing if the patch level isn't `n` or `n+1`, only fail if the patch level is `<n`. That way if we patch the database twice or more in a release, then the current stack would be okay if any DB server needed to restart.\r\n\r\nA temporary solution to the current problem (for train 29):\r\n\r\n1. change the behaviour of the DB server so that a patch level mis-match is only logged\r\n2. patch the prod database with both patches but only increment the patch level by one.\r\n3. release the new stack. Both stacks. should be happy since they the former is fulfilled and the latter is just logged.\r\n4. once the new stack is proven and the old stack goes away, update the patch level in the database\r\n\r\nCurrently my thinking for the overall problem is to check for `<n` and fail on that, rather than fail on anything other than `n` or `n+1`.\r\n\r\nAny other thoughts?\r\n"},"comment":{"url":"https://api.github.com/repos/mozilla/fxa-auth-db-server/issues/comments/72582483","html_url":"https://github.com/mozilla/fxa-auth-db-server/issues/114#issuecomment-72582483","issue_url":"https://api.github.com/repos/mozilla/fxa-auth-db-server/issues/114","id":72582483,"user":{"login":"chilts","id":3048,"avatar_url":"https://avatars.githubusercontent.com/u/3048?v=3","gravatar_id":"","url":"https://api.github.com/users/chilts","html_url":"https://github.com/chilts","followers_url":"https://api.github.com/users/chilts/followers","following_url":"https://api.github.com/users/chilts/following{/other_user}","gists_url":"https://api.github.com/users/chilts/gists{/gist_id}","starred_url":"https://api.github.com/users/chilts/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chilts/subscriptions","organizations_url":"https://api.github.com/users/chilts/orgs","repos_url":"https://api.github.com/users/chilts/repos","events_url":"https://api.github.com/users/chilts/events{/privacy}","received_events_url":"https://api.github.com/users/chilts/received_events","type":"User","site_admin":false},"created_at":"2015-02-03T02:42:28Z","updated_at":"2015-02-03T02:42:28Z","body":"Added this commit on the reverse-with-tests branch : https://github.com/chilts/fxa-auth-db-mysql/commit/dbc10c9fbd76f98fe8190d59a90d5b171f1fa2ac\r\n\r\nThis does the '<n' check, and not just log what the status is.\r\n"}}