{"action":"created","issue":{"url":"https://api.github.com/repos/kbase/assembly/issues/299","labels_url":"https://api.github.com/repos/kbase/assembly/issues/299/labels{/name}","comments_url":"https://api.github.com/repos/kbase/assembly/issues/299/comments","events_url":"https://api.github.com/repos/kbase/assembly/issues/299/events","html_url":"https://github.com/kbase/assembly/issues/299","id":58488533,"number":299,"title":"mongodb sort error","user":{"login":"levinas","id":1056733,"avatar_url":"https://avatars.githubusercontent.com/u/1056733?v=3","gravatar_id":"","url":"https://api.github.com/users/levinas","html_url":"https://github.com/levinas","followers_url":"https://api.github.com/users/levinas/followers","following_url":"https://api.github.com/users/levinas/following{/other_user}","gists_url":"https://api.github.com/users/levinas/gists{/gist_id}","starred_url":"https://api.github.com/users/levinas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/levinas/subscriptions","organizations_url":"https://api.github.com/users/levinas/orgs","repos_url":"https://api.github.com/users/levinas/repos","events_url":"https://api.github.com/users/levinas/events{/privacy}","received_events_url":"https://api.github.com/users/levinas/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/kbase/assembly/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/kbase/assembly/milestones/5","labels_url":"https://api.github.com/repos/kbase/assembly/milestones/5/labels","id":761796,"number":5,"title":"v0.6.1: more assemblers, feature collection","description":"","creator":{"login":"levinas","id":1056733,"avatar_url":"https://avatars.githubusercontent.com/u/1056733?v=3","gravatar_id":"","url":"https://api.github.com/users/levinas","html_url":"https://github.com/levinas","followers_url":"https://api.github.com/users/levinas/followers","following_url":"https://api.github.com/users/levinas/following{/other_user}","gists_url":"https://api.github.com/users/levinas/gists{/gist_id}","starred_url":"https://api.github.com/users/levinas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/levinas/subscriptions","organizations_url":"https://api.github.com/users/levinas/orgs","repos_url":"https://api.github.com/users/levinas/repos","events_url":"https://api.github.com/users/levinas/events{/privacy}","received_events_url":"https://api.github.com/users/levinas/received_events","type":"User","site_admin":false},"open_issues":9,"closed_issues":6,"state":"open","created_at":"2014-08-21T19:04:13Z","updated_at":"2015-02-22T03:30:17Z","due_on":"2015-02-11T06:00:00Z","closed_at":null},"comments":1,"created_at":"2015-02-22T03:30:17Z","updated_at":"2015-02-23T17:33:10Z","closed_at":null,"body":"The jenkins test user has submitted enough jobs (~2000) to trigger this limit in mongodb sort:\r\n\r\n```\r\n        <h2>500 Internal Server Error</h2>\r\n        <p>The server encountered an unexpected condition which prevented it from fulfilling the request.</p>\r\n        <pre id=\"traceback\">Traceback (most recent call last):\r\n  File \"/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py\", line 670, in respond\r\n    response.body = self.handler()\r\n  File \"/usr/local/lib/python2.7/dist-packages/cherrypy/lib/encoding.py\", line 217, in __call__\r\n    self.body = self.oldhandler(*args, **kwargs)\r\n  File \"/usr/local/lib/python2.7/dist-packages/cherrypy/_cpdispatch.py\", line 61, in __call__\r\n    return self.callable(*self.args, **self.kwargs)\r\n  File \"/home/ubuntu/assembly/lib/assembly/router.py\", line 504, in status\r\n    docs = [sanitize_doc(d) for d in metadata.list_jobs(userid)]\r\n  File \"/home/ubuntu/assembly/lib/assembly/metadata.py\", line 107, in list_jobs\r\n    for j in jobs.find({'ARASTUSER':user}).sort('job_id', 1):\r\n  File \"/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py\", line 1058, in next\r\n    if len(self.__data) or self._refresh():\r\n  File \"/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py\", line 1002, in _refresh\r\n    self.__uuid_subtype))\r\n  File \"/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py\", line 940, in __send_message\r\n    self.__compile_re)\r\n  File \"/usr/local/lib/python2.7/dist-packages/pymongo/helpers.py\", line 109, in _unpack_response\r\n    error_object)\r\nOperationFailure: database error: too much data for sort() with no index.  add an index or specify a smaller limit\r\n```\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/kbase/assembly/issues/comments/75590279","html_url":"https://github.com/kbase/assembly/issues/299#issuecomment-75590279","issue_url":"https://api.github.com/repos/kbase/assembly/issues/299","id":75590279,"user":{"login":"cbun","id":354825,"avatar_url":"https://avatars.githubusercontent.com/u/354825?v=3","gravatar_id":"","url":"https://api.github.com/users/cbun","html_url":"https://github.com/cbun","followers_url":"https://api.github.com/users/cbun/followers","following_url":"https://api.github.com/users/cbun/following{/other_user}","gists_url":"https://api.github.com/users/cbun/gists{/gist_id}","starred_url":"https://api.github.com/users/cbun/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cbun/subscriptions","organizations_url":"https://api.github.com/users/cbun/orgs","repos_url":"https://api.github.com/users/cbun/repos","events_url":"https://api.github.com/users/cbun/events{/privacy}","received_events_url":"https://api.github.com/users/cbun/received_events","type":"User","site_admin":false},"created_at":"2015-02-23T17:33:10Z","updated_at":"2015-02-23T17:33:10Z","body":"``'OperationFailure: database error: too much data for sort() with no index.  add an index or specify a smaller limit```\r\n\r\nI think the current implementation does something dumb and queries all user jobs first and limits the records in Python space.  We need to limit the query in the mongo call instead."}}