{"action":"created","issue":{"url":"https://api.github.com/repos/getsentry/sentry/issues/1584","labels_url":"https://api.github.com/repos/getsentry/sentry/issues/1584/labels{/name}","comments_url":"https://api.github.com/repos/getsentry/sentry/issues/1584/comments","events_url":"https://api.github.com/repos/getsentry/sentry/issues/1584/events","html_url":"https://github.com/getsentry/sentry/issues/1584","id":92122708,"number":1584,"title":"Internal sentry error after upgrading from 7.5.4 to 7.6.0","user":{"login":"Nihn","id":7430132,"avatar_url":"https://avatars.githubusercontent.com/u/7430132?v=3","gravatar_id":"","url":"https://api.github.com/users/Nihn","html_url":"https://github.com/Nihn","followers_url":"https://api.github.com/users/Nihn/followers","following_url":"https://api.github.com/users/Nihn/following{/other_user}","gists_url":"https://api.github.com/users/Nihn/gists{/gist_id}","starred_url":"https://api.github.com/users/Nihn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Nihn/subscriptions","organizations_url":"https://api.github.com/users/Nihn/orgs","repos_url":"https://api.github.com/users/Nihn/repos","events_url":"https://api.github.com/users/Nihn/events{/privacy}","received_events_url":"https://api.github.com/users/Nihn/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-06-30T15:20:53Z","updated_at":"2015-07-01T06:28:58Z","closed_at":"2015-06-30T16:22:12Z","body":"After upgrading I receive such errors in rate ~2/m:\r\n\r\nRaw json data:\r\n```\r\n{  \r\n   \"id\":\"8275baade0064e8d9c2d4e0bef764b62\",\r\n   \"project\":22,\r\n   \"release\":null,\r\n   \"platform\":\"python\",\r\n   \"culprit\":\"celery.worker.consumer in on_task_received\",\r\n   \"message\":\"Received unregistered task of type 'sentry.tasks.check_alerts'.\\nThe message has been ignored and discarded.\\n\\nDid you remember to import the module containing this task?\\nOr maybe you are using relative imports?\\nPlease see http://bit.ly/gLye1c for more information.\\n\\nThe full contents of the message body was:\\n{'utc': True, 'chord': None, 'args': [], 'retries': 0, 'expires': '2015-06-30T15:19:21.337893+00:00', 'task': 'sentry.tasks.check_alerts', 'callbacks': None, 'errbacks': None, 'timelimit': (None, None), 'taskset': None, 'kwargs': {}, 'eta': None, 'id': '2037a9f5-5308-4e03-a7e5-61aa53c01628'} (256b)\\n\",\r\n   \"datetime\":\"2015-06-30T15:18:21.000000Z\",\r\n   \"time_spent\":null,\r\n   \"tags\":[  \r\n      [  \r\n         \"site\",\r\n         \"example.com\"\r\n      ],\r\n      [  \r\n         \"level\",\r\n         \"error\"\r\n      ],\r\n      [  \r\n         \"logger\",\r\n         \"celery.worker.consumer\"\r\n      ],\r\n      [  \r\n         \"server_name\",\r\n         \"errors\"\r\n      ]\r\n   ],\r\n   \"extra\":{  \r\n      \"thread\":140284792469248,\r\n      \"process\":27884,\r\n      \"sys.argv\":[  \r\n         \"'/var/www/sentry/bin/sentry'\",\r\n         \"'--config=/etc/sentry/conf.py'\",\r\n         \"'celery'\",\r\n         \"'worker'\",\r\n         \"'-B'\",\r\n         \"'-l'\",\r\n         \"'WARNING'\"\r\n      ],\r\n      \"threadName\":\"'MainThread'\",\r\n      \"filename\":\"'consumer.py'\",\r\n      \"processName\":\"'MainProcess'\",\r\n      \"pathname\":\"'/var/www/sentry/local/lib/python2.7/site-packages/celery/worker/consumer.py'\",\r\n      \"lineno\":427,\r\n      \"asctime\":\"'2015-06-30 15:18:21,340'\"\r\n   },\r\n   \"modules\":{  \r\n      \"south\":\"1.0.1\",\r\n      \"sentry\":\"7.6.0\",\r\n      \"gunicorn\":\"19.3.0\",\r\n      \"python\":\"2.7.3\",\r\n      \"sudo\":\"unknown\",\r\n      \"social_auth\":\"0.7.28\",\r\n      \"django\":\"1.6.11\",\r\n      \"kombu.transport.django\":\"1.0.0\",\r\n      \"kombu\":\"3.0.26\",\r\n      \"sentry_jira\":\"0.9.0\",\r\n      \"rest_framework\":\"2.3.14\",\r\n      \"crispy_forms\":\"1.4.0\",\r\n      \"raven\":\"5.3.1\"\r\n   },\r\n   \"sentry.interfaces.Exception\":{  \r\n      \"exc_omitted\":null,\r\n      \"values\":[  \r\n         {  \r\n            \"stacktrace\":{  \r\n               \"frames\":[  \r\n                  {  \r\n                     \"function\":\"on_task_received\",\r\n                     \"abs_path\":\"/var/www/sentry/local/lib/python2.7/site-packages/celery/worker/consumer.py\",\r\n                     \"pre_context\":[  \r\n                        \"                name = body['task']\",\r\n                        \"            except (KeyError, TypeError):\",\r\n                        \"                return on_unknown_message(body, message)\",\r\n                        \"\",\r\n                        \"            try:\"\r\n                     ],\r\n                     \"vars\":{  \r\n                        \"body\":{  \r\n                           \"'errbacks'\":null,\r\n                           \"'callbacks'\":null,\r\n                           \"'eta'\":null,\r\n                           \"'args'\":[  \r\n\r\n                           ],\r\n                           \"'taskset'\":null,\r\n                           \"'retries'\":0,\r\n                           \"'expires'\":\"'2015-06-30T15:19:21.337893+00:00'\",\r\n                           \"'task'\":\"'sentry.tasks.check_alerts'\",\r\n                           \"'id'\":\"'2037a9f5-5308-4e03-a7e5-61aa53c01628'\",\r\n                           \"'chord'\":null,\r\n                           \"'timelimit'\":[  \r\n                              null,\r\n                              null\r\n                           ],\r\n                           \"'kwargs'\":{  \r\n\r\n                           },\r\n                           \"'utc'\":true\r\n                        },\r\n                        \"on_unknown_task\":\"<bound method Consumer.on_unknown_task of <Consumer: celery@errors (running)>>\",\r\n                        \"exc\":\"KeyError('sentry.tasks.check_alerts',)\",\r\n                        \"on_unknown_message\":\"<bound method Consumer.on_unknown_message of <Consumer: celery@errors (running)>>\",\r\n                        \"callbacks\":[  \r\n\r\n                        ],\r\n                        \"strategies\":{  \r\n                           \"'celery.chunks'\":\"<function task_message_handler from celery.worker.strategy at 0x5504500>\",\r\n                           \"'sentry.tasks.options.sync_options'\":\"<function task_message_handler from celery.worker.strategy at 0x550d668>\",\r\n                           \"'sentry.tasks.process_buffer.process_incr'\":\"<function task_message_handler from celery.worker.strategy at 0x550d230>\",\r\n                           \"'sentry.tasks.index.index_event'\":\"<function task_message_handler from celery.worker.strategy at 0x5504668>\",\r\n                           \"'celery.starmap'\":\"<function task_message_handler from celery.worker.strategy at 0x550d938>\",\r\n                           \"'sentry.tasks.deletion.delete_tag_key'\":\"<function task_message_handler from celery.worker.strategy at 0x550dc08>\",\r\n                           \"'sentry.tasks.post_process.record_affected_user'\":\"<function task_message_handler from celery.worker.strategy at 0x5519938>\"\r\n                        },\r\n                        \"message\":\"<kombu.transport.virtual.Message object at 0x553a938>\",\r\n                        \"on_invalid_task\":\"<bound method Consumer.on_invalid_task of <Consumer: celery@errors (running)>>\",\r\n                        \"name\":\"'sentry.tasks.check_alerts'\"\r\n                     },\r\n                     \"module\":\"celery.worker.consumer\",\r\n                     \"filename\":\"celery/worker/consumer.py\",\r\n                     \"post_context\":[  \r\n                        \"                                 message.ack_log_error,\",\r\n                        \"                                 message.reject_log_error,\",\r\n                        \"                                 callbacks)\",\r\n                        \"            except KeyError as exc:\",\r\n                        \"                on_unknown_task(body, message, exc)\"\r\n                     ],\r\n                     \"context_line\":\"                strategies[name](message, body,\",\r\n                     \"lineno\":455\r\n                  }\r\n               ],\r\n               \"frames_omitted\":null\r\n            },\r\n            \"type\":\"KeyError\",\r\n            \"module\":\"exceptions\",\r\n            \"value\":\"'sentry.tasks.check_alerts'\"\r\n         }\r\n      ]\r\n   },\r\n   \"sentry.interfaces.Message\":{  \r\n      \"message\":\"Received unregistered task of type %s.\\nThe message has been ignored and discarded.\\n\\nDid you remember to import the module containing this task?\\nOr maybe you are using relative imports?\\nPlease see http://bit.ly/gLye1c for more information.\\n\\nThe full contents of the message body was:\\n%s\\n\",\r\n      \"params\":[  \r\n         \"KeyError('sentry.tasks.check_alerts',)\",\r\n         \"'{'utc': True, 'chord': None, 'args': [], 'retries': 0, 'expires': '2015-06-30T15:19:21.337893+00:00', 'task': 'sentry.tasks.check_alerts', 'callbacks': None, 'errbacks': None, 'timelimit': (None, None), 'taskset': None, 'kwargs': {}, 'eta': None, 'id': '2037a9f5-5308-4e03-a7e5-61aa53c01628'} (256b)'\"\r\n      ]\r\n   },\r\n   \"version\":\"5\"\r\n}\r\n```"},"comment":{"url":"https://api.github.com/repos/getsentry/sentry/issues/comments/117452250","html_url":"https://github.com/getsentry/sentry/issues/1584#issuecomment-117452250","issue_url":"https://api.github.com/repos/getsentry/sentry/issues/1584","id":117452250,"user":{"login":"dcramer","id":23610,"avatar_url":"https://avatars.githubusercontent.com/u/23610?v=3","gravatar_id":"","url":"https://api.github.com/users/dcramer","html_url":"https://github.com/dcramer","followers_url":"https://api.github.com/users/dcramer/followers","following_url":"https://api.github.com/users/dcramer/following{/other_user}","gists_url":"https://api.github.com/users/dcramer/gists{/gist_id}","starred_url":"https://api.github.com/users/dcramer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dcramer/subscriptions","organizations_url":"https://api.github.com/users/dcramer/orgs","repos_url":"https://api.github.com/users/dcramer/repos","events_url":"https://api.github.com/users/dcramer/events{/privacy}","received_events_url":"https://api.github.com/users/dcramer/received_events","type":"User","site_admin":false},"created_at":"2015-07-01T06:28:58Z","updated_at":"2015-07-01T06:28:58Z","body":"Can you determine the celerybeat schedule file and try wiping it? It's likely in /tmp unless you set the path manually. \n\n\n\n\n\n\nOn Tue, Jun 30, 2015 at 11:27 PM -0700, \"Mateusz Moneta\" <notifications@github.com> wrote:\n\n\n\n\n\n\n\n\n\n\n\n\n@dcramer I've restarted sentry uwsgi, sentry workers and redis just to be sure. Events still coming.\n\n\n\n—\nReply to this email directly or view it on GitHub."}}