{"action":"created","issue":{"url":"https://api.github.com/repos/mathcamp/flywheel/issues/34","labels_url":"https://api.github.com/repos/mathcamp/flywheel/issues/34/labels{/name}","comments_url":"https://api.github.com/repos/mathcamp/flywheel/issues/34/comments","events_url":"https://api.github.com/repos/mathcamp/flywheel/issues/34/events","html_url":"https://github.com/mathcamp/flywheel/issues/34","id":121445680,"number":34,"title":"gen() iterator should support resumable paged queries/scans","user":{"login":"natecode","id":1581257,"avatar_url":"https://avatars.githubusercontent.com/u/1581257?v=3","gravatar_id":"","url":"https://api.github.com/users/natecode","html_url":"https://github.com/natecode","followers_url":"https://api.github.com/users/natecode/followers","following_url":"https://api.github.com/users/natecode/following{/other_user}","gists_url":"https://api.github.com/users/natecode/gists{/gist_id}","starred_url":"https://api.github.com/users/natecode/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/natecode/subscriptions","organizations_url":"https://api.github.com/users/natecode/orgs","repos_url":"https://api.github.com/users/natecode/repos","events_url":"https://api.github.com/users/natecode/events{/privacy}","received_events_url":"https://api.github.com/users/natecode/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-12-10T09:55:52Z","updated_at":"2015-12-11T08:11:21Z","closed_at":null,"body":"I’d like to be able to provide an API to our users that is backed by a query that returns a large number of results. We’d like to be able to retrieve and store a cursor from a result set (query or scan), then create a new result set that picks up where the last one left off, possibly minutes later. I know that dynamo3 supports this internally, but it's not exported via flywheel yet.\r\n\r\nSteve replied via email:\r\n\r\nI think you'd have to have a new terminator on the query (like gen(), all()). I think you'd need page() to fetch a single page from DynamoDB so that you have some knowledge of where the database query left off. The default gen() implementation just provides a straight-up iterator that handles it all transparently. We could make page() return an object that extends list and has a 'cursor' attribute. Then you make the query terminators also take 'cursor' as a keyword argument, pass that through to dynamo3 and it should just work. I'll have to make a couple of changes in dynamo3 to support fetching pages instead of just iterators, but that should be pretty easy."},"comment":{"url":"https://api.github.com/repos/mathcamp/flywheel/issues/comments/163871211","html_url":"https://github.com/mathcamp/flywheel/issues/34#issuecomment-163871211","issue_url":"https://api.github.com/repos/mathcamp/flywheel/issues/34","id":163871211,"user":{"login":"stevearc","id":506791,"avatar_url":"https://avatars.githubusercontent.com/u/506791?v=3","gravatar_id":"","url":"https://api.github.com/users/stevearc","html_url":"https://github.com/stevearc","followers_url":"https://api.github.com/users/stevearc/followers","following_url":"https://api.github.com/users/stevearc/following{/other_user}","gists_url":"https://api.github.com/users/stevearc/gists{/gist_id}","starred_url":"https://api.github.com/users/stevearc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stevearc/subscriptions","organizations_url":"https://api.github.com/users/stevearc/orgs","repos_url":"https://api.github.com/users/stevearc/repos","events_url":"https://api.github.com/users/stevearc/events{/privacy}","received_events_url":"https://api.github.com/users/stevearc/received_events","type":"User","site_admin":false},"created_at":"2015-12-11T08:11:21Z","updated_at":"2015-12-11T08:11:21Z","body":"I ended up going about it in a slightly different way than initially planned, but it's pretty similar.\r\n\r\nYou can see an example of what the flow would look like [in this test case](https://github.com/mathcamp/flywheel/blob/586e0f6cc15d1f4e822725bc6702f87e344779e5/tests/test_scan.py#L186)\r\n\r\nYou should be able to configure the query behavior to do what you want with the [Limit](https://github.com/stevearc/dynamo3/blob/86484d2ff669f133dab012785b50e27a5a763047/dynamo3/result.py#L440) class. It's in dynamo3, but it's also [imported in flywheel](https://github.com/mathcamp/flywheel/blob/586e0f6cc15d1f4e822725bc6702f87e344779e5/flywheel/__init__.py#L4) so you can import it directly from there.\r\n\r\nReleased flywheel 0.4.6. Let me know if this works for you!"}}