{"action":"created","issue":{"url":"https://api.github.com/repos/octokit/octokit.net/issues/740","labels_url":"https://api.github.com/repos/octokit/octokit.net/issues/740/labels{/name}","comments_url":"https://api.github.com/repos/octokit/octokit.net/issues/740/comments","events_url":"https://api.github.com/repos/octokit/octokit.net/issues/740/events","html_url":"https://github.com/octokit/octokit.net/pull/740","id":59756918,"number":740,"title":"[WIP] implement first-class paging support","user":{"login":"shiftkey","id":359239,"avatar_url":"https://avatars.githubusercontent.com/u/359239?v=3","gravatar_id":"","url":"https://api.github.com/users/shiftkey","html_url":"https://github.com/shiftkey","followers_url":"https://api.github.com/users/shiftkey/followers","following_url":"https://api.github.com/users/shiftkey/following{/other_user}","gists_url":"https://api.github.com/users/shiftkey/gists{/gist_id}","starred_url":"https://api.github.com/users/shiftkey/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shiftkey/subscriptions","organizations_url":"https://api.github.com/users/shiftkey/orgs","repos_url":"https://api.github.com/users/shiftkey/repos","events_url":"https://api.github.com/users/shiftkey/events{/privacy}","received_events_url":"https://api.github.com/users/shiftkey/received_events","type":"User","site_admin":true},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-03-04T06:39:26Z","updated_at":"2015-03-04T09:22:36Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/octokit/octokit.net/pulls/740","html_url":"https://github.com/octokit/octokit.net/pull/740","diff_url":"https://github.com/octokit/octokit.net/pull/740.diff","patch_url":"https://github.com/octokit/octokit.net/pull/740.patch"},"body":":cactus: :cactus: :cactus: :cactus: :cactus: :cactus: :cactus: :cactus: :cactus: **WORK IN PROGRESS** :cactus: :cactus: :cactus: :cactus: :cactus: :cactus: :cactus: :cactus: :cactus: \r\n:cactus: :cactus: :cactus: :cactus: :cactus: :cactus: :cactus: :cactus: :cactus:  **DO NOT MERGE** :cactus: :cactus: :cactus::cactus: :cactus: :cactus: :cactus: :cactus: :cactus:  \r\n\r\nI'd been noodling around with how to tackle paging in Octokit, and here's my first attempt at a solution.\r\n\r\n#### The Problem\r\n\r\nBy default, if there's new data to fetch, the Octokit client will go and fetch it. Thanks to the [pagination](https://developer.github.com/v3/#pagination) APIs, this is rather easy to do. For my use case, that's been Good Enough. But I'm at the point where I'd like to do things like:\r\n\r\n - adjust the number of results per page\r\n - stop at a certain page number\r\n - skip certain pages\r\n - opt-in to preview API behaviour by overriding accepts header\r\n\r\nSome endpoints support other sorts of filtering (for example, [issues search](https://developer.github.com/v3/issues/#list-issues-for-a-repository) has a `since` parameter. I'm not worrying about those here - this approach should work for all endpoints that return arrays of results.\r\n\r\nThis isn't something I've focused on given I spent most of my days working with desktop apps, but I know other platforms would love having this level of control to conserve data.\r\n\r\nAnd we have #259 and #625 as open issues, so why not do try and resolve this?\r\n\r\n#### The Approach\r\n\r\nYou can await any type in C# with the presence of `async`/`await` - and it's pretty easy once you know how to get started. For backwards-compatibility, I didn't want to force you to update your code - if you didn't opt-in to this behaviour, you should get the existing behaviour.\r\n\r\nSo I started with this ideal API:\r\n\r\n```\r\nvar repositories = await client.Repository.GetAllForCurrent()\r\n    .WithPageCount(10)\r\n    .SelectPages(10);\r\n ```\r\n\r\nInstead of returning `Task<IReadOnlyList<T>>` here I'm returning `ILazyRequest<T>` - and it's this type that has the extra options necessary. These is how one can tweak the internal state of the request before it executes.\r\n\r\nTo get the `await` *magic* to work, you need to define a public static method in a static type that takes your type as an extension method and returns a `TaskAwaiter<T>`.\r\n\r\n```\r\npublic static class CustomAwaiter\r\n{\r\n    public static TaskAwaiter<IReadOnlyList<T>> GetAwaiter<T>(this ILazyRequest<T> request)\r\n    {\r\n        Ensure.ArgumentNotNull(request, \"request\");\r\n\r\n        return request.ToTask();\r\n    }\r\n}\r\n```\r\n\r\nThis lives under the `Octokit` namespace, so it should Just Work with existing users.\r\n\r\n`ToTask()` just defers to the existing behaviour, but this gives me the hook necessary to inject the additional parameters before executing the request.\r\n\r\nThere's also some enhancements necessary to `IApiPagination` to allow clients to opt-out to paging early. I should be able to extract this dependency from `ApiConnection` by the time this is done, so I'm fine with changing that behaviour around.\r\n\r\n#### Limitations\r\n\r\nI need to think about how to do this with the Observable-based APIs - currently we just cheat and transform the Task-based API, but because we surface an  `IObservable<T>` we'd need to do something similar here. Plus there are extension methods like `Skip` and `Take` available that we should be respecting (and not fetching all the records in the first place). Perhaps this is when the two libraries diverge, implementation-wise :cry:\r\n\r\nEDIT: we actually use a different implementation inside the `GetAll`s for the observable clients - `GetAndFlattenAllPages` - so I'm gonna leave this aside for now and tackle that another day.\r\n\r\n#### TODO\r\n\r\n - [x] unbreak all the tests\r\n - [x] resolve muted fx:cop: issues\r\n - [ ] finish `RepositoriesClient` migration\r\n - ~~[ ] ponder on how to do this with `Octokit.Reactive`~~ :boot: punt to another time"},"comment":{"url":"https://api.github.com/repos/octokit/octokit.net/issues/comments/77122273","html_url":"https://github.com/octokit/octokit.net/pull/740#issuecomment-77122273","issue_url":"https://api.github.com/repos/octokit/octokit.net/issues/740","id":77122273,"user":{"login":"shiftkey","id":359239,"avatar_url":"https://avatars.githubusercontent.com/u/359239?v=3","gravatar_id":"","url":"https://api.github.com/users/shiftkey","html_url":"https://github.com/shiftkey","followers_url":"https://api.github.com/users/shiftkey/followers","following_url":"https://api.github.com/users/shiftkey/following{/other_user}","gists_url":"https://api.github.com/users/shiftkey/gists{/gist_id}","starred_url":"https://api.github.com/users/shiftkey/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shiftkey/subscriptions","organizations_url":"https://api.github.com/users/shiftkey/orgs","repos_url":"https://api.github.com/users/shiftkey/repos","events_url":"https://api.github.com/users/shiftkey/events{/privacy}","received_events_url":"https://api.github.com/users/shiftkey/received_events","type":"User","site_admin":true},"created_at":"2015-03-04T09:22:35Z","updated_at":"2015-03-04T09:22:35Z","body":"I'll pause here before going further and see if people have feedback on this approach."}}