{"action":"opened","issue":{"url":"https://api.github.com/repos/bloomberg/chef-bcpc/issues/630","labels_url":"https://api.github.com/repos/bloomberg/chef-bcpc/issues/630/labels{/name}","comments_url":"https://api.github.com/repos/bloomberg/chef-bcpc/issues/630/comments","events_url":"https://api.github.com/repos/bloomberg/chef-bcpc/issues/630/events","html_url":"https://github.com/bloomberg/chef-bcpc/issues/630","id":87384414,"number":630,"title":"nova-database-sync sometimes fails with a MySQL integrity error","user":{"login":"erhudy","id":1065271,"avatar_url":"https://avatars.githubusercontent.com/u/1065271?v=3","gravatar_id":"","url":"https://api.github.com/users/erhudy","html_url":"https://github.com/erhudy","followers_url":"https://api.github.com/users/erhudy/followers","following_url":"https://api.github.com/users/erhudy/following{/other_user}","gists_url":"https://api.github.com/users/erhudy/gists{/gist_id}","starred_url":"https://api.github.com/users/erhudy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/erhudy/subscriptions","organizations_url":"https://api.github.com/users/erhudy/orgs","repos_url":"https://api.github.com/users/erhudy/repos","events_url":"https://api.github.com/users/erhudy/events{/privacy}","received_events_url":"https://api.github.com/users/erhudy/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/bloomberg/chef-bcpc/labels/bug","name":"bug","color":"fc2929"},{"url":"https://api.github.com/repos/bloomberg/chef-bcpc/labels/MySQL","name":"MySQL","color":"d4c5f9"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-06-11T15:27:06Z","updated_at":"2015-06-11T15:27:06Z","closed_at":null,"body":"This sometimes happens. Recheffing fixes it.\r\n```\r\n  * bash[nova-database-sync] action run\r\n    \r\n    ================================================================================\r\n    Error executing action `run` on resource 'bash[nova-database-sync]'\r\n    ================================================================================\r\n    \r\n    Mixlib::ShellOut::ShellCommandFailed\r\n    ------------------------------------\r\n    Expected process to exit with [0], but received '1'\r\n    ---- Begin output of \"bash\"  \"/tmp/chef-script20150611-5675-1oqqzwp\" ----\r\n    STDOUT: Command failed, please check log for more info\r\n    STDERR: No handlers could be found for logger \"oslo_config.cfg\"\r\n    2015-06-11 15:24:52.346 19380 CRITICAL nova [-] DBDuplicateEntry: (IntegrityError) (1062, \"Duplicate entry 'bcpc-vm1-conductor-0' for key 'uniq_services0host0topic0deleted'\") 'ALTER TABLE services ADD CONSTRAINT uniq_services0host0topic0deleted UNIQUE (host, topic, deleted)' ()\r\n    2015-06-11 15:24:52.346 19380 TRACE nova Traceback (most recent call last):\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/bin/nova-manage\", line 10, in <module>\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     sys.exit(main())\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/nova/cmd/manage.py\", line 1382, in main\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     ret = fn(*fn_args, **fn_kwargs)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/nova/cmd/manage.py\", line 912, in sync\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     return migration.db_sync(version)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/nova/db/migration.py\", line 26, in db_sync\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     return IMPL.db_sync(version=version, database=database)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/migration.py\", line 57, in db_sync\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     version)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/migrate/versioning/api.py\", line 186, in upgrade\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     return _migrate(url, repository, version, upgrade=True, err=err, **opts)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"<string>\", line 2, in _migrate\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/migrate/versioning/util/__init__.py\", line 160, in with_engine\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     return f(*a, **kw)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/migrate/versioning/api.py\", line 366, in _migrate\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     schema.runchange(ver, change, changeset.step)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/migrate/versioning/schema.py\", line 93, in runchange\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     change.run(self.engine, step)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/migrate/versioning/script/py.py\", line 148, in run\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     script_func(engine)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/migrate_repo/versions/216_havana.py\", line 1173, in upgrade\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     name='uniq_services0host0topic0deleted').create()\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/migrate/changeset/constraint.py\", line 44, in create\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     self.__do_imports('constraintgenerator', *a, **kw)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/migrate/changeset/constraint.py\", line 32, in __do_imports\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     run_single_visitor(engine, visitorcallable, self, *a, **kw)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/migrate/changeset/databases/visitor.py\", line 85, in run_single_visitor\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     fn(element, **kwargs)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/migrate/changeset/ansisql.py\", line 294, in visit_migrate_unique_constraint\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     self._visit_constraint(*p, **k)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/migrate/changeset/ansisql.py\", line 300, in _visit_constraint\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     self.execute()\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/migrate/changeset/ansisql.py\", line 44, in execute\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     return self.connection.execute(self.buffer.getvalue())\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 721, in execute\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     return self._execute_text(object, multiparams, params)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 870, in _execute_text\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     statement, parameters\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 958, in _execute_context\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     context)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/compat/handle_error.py\", line 261, in _handle_dbapi_exception\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     e, statement, parameters, cursor, context)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 1155, in _handle_dbapi_exception\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     util.raise_from_cause(newraise, exc_info)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py\", line 199, in raise_from_cause\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     reraise(type(exception), exception, tb=exc_tb)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 951, in _execute_context\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     context)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py\", line 436, in do_execute\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     cursor.execute(statement, parameters)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py\", line 174, in execute\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     self.errorhandler(self, exc, value)\r\n    2015-06-11 15:24:52.346 19380 TRACE nova   File \"/usr/lib/python2.7/dist-packages/MySQLdb/connections.py\", line 36, in defaulterrorhandler\r\n    2015-06-11 15:24:52.346 19380 TRACE nova     raise errorclass, errorvalue\r\n    2015-06-11 15:24:52.346 19380 TRACE nova DBDuplicateEntry: (IntegrityError) (1062, \"Duplicate entry 'bcpc-vm1-conductor-0' for key 'uniq_services0host0topic0deleted'\") 'ALTER TABLE services ADD CONSTRAINT uniq_services0host0topic0deleted UNIQUE (host, topic, deleted)' ()\r\n    2015-06-11 15:24:52.346 19380 TRACE nova\r\n    ---- End output of \"bash\"  \"/tmp/chef-script20150611-5675-1oqqzwp\" ----\r\n    Ran \"bash\"  \"/tmp/chef-script20150611-5675-1oqqzwp\" returned 1\r\n    \r\n    Resource Declaration:\r\n    ---------------------\r\n    # In /var/chef/cache/cookbooks/bcpc/recipes/nova-head.rb\r\n    \r\n     48: bash \"nova-database-sync\" do\r\n     49:     action :nothing\r\n     50:     user \"root\"\r\n     51:     code \"nova-manage db sync\"\r\n     52:     notifies :restart, \"service[nova-scheduler]\", :immediately\r\n     53:     notifies :restart, \"service[nova-cert]\", :immediately\r\n     54:     notifies :restart, \"service[nova-consoleauth]\", :immediately\r\n     55:     notifies :restart, \"service[nova-conductor]\", :immediately\r\n     56: end\r\n     57: \r\n    \r\n    Compiled Resource:\r\n    ------------------\r\n    # Declared in /var/chef/cache/cookbooks/bcpc/recipes/nova-head.rb:48:in `from_file'\r\n    \r\n    bash(\"nova-database-sync\") do\r\n      action [:nothing]\r\n      retries 0\r\n      retry_delay 2\r\n      default_guard_interpreter :default\r\n      command \"nova-database-sync\"\r\n      backup 5\r\n      returns 0\r\n      user \"root\"\r\n      code \"nova-manage db sync\"\r\n      interpreter \"bash\"\r\n      declared_type :bash\r\n      cookbook_name \"bcpc\"\r\n      recipe_name \"nova-head\"\r\n    end\r\n```"}}