{"action":"created","issue":{"url":"https://api.github.com/repos/boyter/searchcode/issues/27","labels_url":"https://api.github.com/repos/boyter/searchcode/issues/27/labels{/name}","comments_url":"https://api.github.com/repos/boyter/searchcode/issues/27/comments","events_url":"https://api.github.com/repos/boyter/searchcode/issues/27/events","html_url":"https://github.com/boyter/searchcode/issues/27","id":103888046,"number":27,"title":"In code_search_I API, next_page field doesn't correctly signal end-of-data","user":{"login":"njsmith","id":609896,"avatar_url":"https://avatars.githubusercontent.com/u/609896?v=3","gravatar_id":"","url":"https://api.github.com/users/njsmith","html_url":"https://github.com/njsmith","followers_url":"https://api.github.com/users/njsmith/followers","following_url":"https://api.github.com/users/njsmith/following{/other_user}","gists_url":"https://api.github.com/users/njsmith/gists{/gist_id}","starred_url":"https://api.github.com/users/njsmith/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/njsmith/subscriptions","organizations_url":"https://api.github.com/users/njsmith/orgs","repos_url":"https://api.github.com/users/njsmith/repos","events_url":"https://api.github.com/users/njsmith/events{/privacy}","received_events_url":"https://api.github.com/users/njsmith/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-08-29T23:05:03Z","updated_at":"2015-08-30T22:52:50Z","closed_at":"2015-08-30T22:36:14Z","body":"On [this page](https://searchcode.com/api/) it says:\r\n```\r\n    nextpage     Id of the offset of the next page. If no next page is available it will be set to null. This is a zero based index.\r\n```\r\n\r\nAs far as I can tell, though, `nextpage` never gets set to `null` -- it's always currentpage + 1. The only seemingly reliable way that I've found to fetch all searchcode.com results is to keep fetching the next page until you get a page with no results.\r\n\r\nFor example:\r\n```\r\nIn [39]: json.loads(requests.get(\"https://searchcode.com/api/codesearch_I/?q=PyUFuncObject&per_page=100&p=6\", verify=False).text)[\"nextpage\"]\r\nOut[39]: 7\r\n\r\nIn [40]: json.loads(requests.get(\"https://searchcode.com/api/codesearch_I/?q=PyUFuncObject&per_page=100&p=7\", verify=False).text)\r\nOut[40]: \r\n{u'language_filters': [{u'count': 643, u'id': 28, u'language': u'C'},\r\n  {u'count': 42, u'id': 16, u'language': u'C++'},\r\n  {u'count': 24, u'id': 15, u'language': u'C/C++ Hea...'},\r\n  {u'count': 4, u'id': 19, u'language': u'Python'}],\r\n u'matchterm': u'PyUFuncObject',\r\n u'nextpage': 8,\r\n u'page': 7,\r\n u'previouspage': 6,\r\n u'query': u'PyUFuncObject',\r\n u'results': [],\r\n u'searchterm': u'PyUFuncObject',\r\n u'source_filters': [{u'count': 636, u'id': 2, u'source': u'Github'},\r\n  {u'count': 61, u'id': 3, u'source': u'Bitbucket'},\r\n  {u'count': 9, u'id': 7, u'source': u'Fedora Pr...'},\r\n  {u'count': 7, u'id': 1, u'source': u'Google Code'}],\r\n u'total': 678}\r\n```"},"comment":{"url":"https://api.github.com/repos/boyter/searchcode/issues/comments/136212157","html_url":"https://github.com/boyter/searchcode/issues/27#issuecomment-136212157","issue_url":"https://api.github.com/repos/boyter/searchcode/issues/27","id":136212157,"user":{"login":"njsmith","id":609896,"avatar_url":"https://avatars.githubusercontent.com/u/609896?v=3","gravatar_id":"","url":"https://api.github.com/users/njsmith","html_url":"https://github.com/njsmith","followers_url":"https://api.github.com/users/njsmith/followers","following_url":"https://api.github.com/users/njsmith/following{/other_user}","gists_url":"https://api.github.com/users/njsmith/gists{/gist_id}","starred_url":"https://api.github.com/users/njsmith/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/njsmith/subscriptions","organizations_url":"https://api.github.com/users/njsmith/orgs","repos_url":"https://api.github.com/users/njsmith/repos","events_url":"https://api.github.com/users/njsmith/events{/privacy}","received_events_url":"https://api.github.com/users/njsmith/received_events","type":"User","site_admin":false},"created_at":"2015-08-30T22:52:50Z","updated_at":"2015-08-30T22:52:50Z","body":"It looks like you now have: *Id of the offset of the next page. Is always set to +1 the current page. To determine the end or the results query till no results are returned.*\r\n\r\nI'm afraid I still find this somewhat unclear. Partly because of the \"or\"/\"of\" typo, but...\r\n\r\nI'd write: \"Id of the offset of the next page. Is always set to the current page + 1, even if you have reached the end of the results\"\r\n\r\nAnd then in the documentation for \"results\", I'd write: \"If the 'results' list is empty, then this indicates that you have reached the end of the available results. To fetch all results for a given query, keep incrementing 'p' until you get a page with an empty 'results' list.\""}}