{"action":"created","issue":{"url":"https://api.github.com/repos/mozilla-services/readinglist/issues/25","labels_url":"https://api.github.com/repos/mozilla-services/readinglist/issues/25/labels{/name}","comments_url":"https://api.github.com/repos/mozilla-services/readinglist/issues/25/comments","events_url":"https://api.github.com/repos/mozilla-services/readinglist/issues/25/events","html_url":"https://github.com/mozilla-services/readinglist/issues/25","id":53053312,"number":25,"title":"Pagination","user":{"login":"rnewman","id":91722,"avatar_url":"https://avatars.githubusercontent.com/u/91722?v=3","gravatar_id":"","url":"https://api.github.com/users/rnewman","html_url":"https://github.com/rnewman","followers_url":"https://api.github.com/users/rnewman/followers","following_url":"https://api.github.com/users/rnewman/following{/other_user}","gists_url":"https://api.github.com/users/rnewman/gists{/gist_id}","starred_url":"https://api.github.com/users/rnewman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rnewman/subscriptions","organizations_url":"https://api.github.com/users/rnewman/orgs","repos_url":"https://api.github.com/users/rnewman/repos","events_url":"https://api.github.com/users/rnewman/events{/privacy}","received_events_url":"https://api.github.com/users/rnewman/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/mozilla-services/readinglist/labels/API-design","name":"API-design","color":"fbca04"},{"url":"https://api.github.com/repos/mozilla-services/readinglist/labels/question","name":"question","color":"cc317c"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":10,"created_at":"2014-12-29T16:43:55Z","updated_at":"2015-02-05T14:54:16Z","closed_at":null,"body":"#13 and #14 raise the question of how to fetch large numbers of items.\r\n\r\nWe can expect some number of users on day one to have hundreds of items. For illustration, there are a number of folks on the mobile team who have more than 1500 items in their current Pocket queue.\r\n\r\nIn ordinary operation — frequently connected — we can expect downloaded changesets to be small; a handful of new and changed items in each fetch.\r\n\r\nIn protracted disconnected operation we'll need batching for uploads, which is #2.\r\n\r\nThat leaves us worrying about first-time download on a device, and subsequent downloads after a very protracted disconnected period (or after an import).\r\n\r\nWe can approach this in one of a few ways.\r\n\r\n1. Ignore the problem. Always maintain a full list of items on the client. Fetch everything in one go. After all, the metadata for even a thousand items will probably only be a few hundred KB. This is fine for initial development, but probably not enough for a global release — even a few hundred KB is too much in surprisingly many locales.\r\n\r\n2. Ignore the problem except for downloads. Always maintain a full list of items on the client, but fetch items in batches. This is easy if the server supports sorting (already desirable: #14) and one more thing: range queries over some immutable attribute. E.g.,\r\n\r\n```\r\narticles/?sort=-stored_on&stored_on_less_than=123456\r\n```\r\n\r\nThis solves some reliability/interrupted download issues, but still triggers significant space and bandwidth costs immediately after install. (What would you think if Gmail downloaded the headers for every message to your phone after signing in?)\r\n\r\n3. Go fully server-canonical, and expect most clients to have only a partial cache of the articles that are stored on the server. For example, clients might not bother fetching metadata about archived articles until the user tries to see that list. Pocket falls into this category: it'll fetch new batches of articles as you scroll through your list.\r\n\r\nThis requires much the same support from the server, but imposes additional smarts on the client.\r\n\r\nI think we should go with (1) during development, then shift to (2) or (3) as soon as we can."},"comment":{"url":"https://api.github.com/repos/mozilla-services/readinglist/issues/comments/73058642","html_url":"https://github.com/mozilla-services/readinglist/issues/25#issuecomment-73058642","issue_url":"https://api.github.com/repos/mozilla-services/readinglist/issues/25","id":73058642,"user":{"login":"Natim","id":229453,"avatar_url":"https://avatars.githubusercontent.com/u/229453?v=3","gravatar_id":"","url":"https://api.github.com/users/Natim","html_url":"https://github.com/Natim","followers_url":"https://api.github.com/users/Natim/followers","following_url":"https://api.github.com/users/Natim/following{/other_user}","gists_url":"https://api.github.com/users/Natim/gists{/gist_id}","starred_url":"https://api.github.com/users/Natim/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Natim/subscriptions","organizations_url":"https://api.github.com/users/Natim/orgs","repos_url":"https://api.github.com/users/Natim/repos","events_url":"https://api.github.com/users/Natim/events{/privacy}","received_events_url":"https://api.github.com/users/Natim/received_events","type":"User","site_admin":false},"created_at":"2015-02-05T14:54:15Z","updated_at":"2015-02-05T14:54:15Z","body":"I might have an alternative solution. I will think furthermore about it."}}