{"action":"created","issue":{"url":"https://api.github.com/repos/getclump/clump/issues/109","labels_url":"https://api.github.com/repos/getclump/clump/issues/109/labels{/name}","comments_url":"https://api.github.com/repos/getclump/clump/issues/109/comments","events_url":"https://api.github.com/repos/getclump/clump/issues/109/events","html_url":"https://github.com/getclump/clump/pull/109","id":111893727,"number":109,"title":"Allow sources to fail individual results in a batch fetch","user":{"login":"stevenheidel","id":155646,"avatar_url":"https://avatars.githubusercontent.com/u/155646?v=3","gravatar_id":"","url":"https://api.github.com/users/stevenheidel","html_url":"https://github.com/stevenheidel","followers_url":"https://api.github.com/users/stevenheidel/followers","following_url":"https://api.github.com/users/stevenheidel/following{/other_user}","gists_url":"https://api.github.com/users/stevenheidel/gists{/gist_id}","starred_url":"https://api.github.com/users/stevenheidel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stevenheidel/subscriptions","organizations_url":"https://api.github.com/users/stevenheidel/orgs","repos_url":"https://api.github.com/users/stevenheidel/repos","events_url":"https://api.github.com/users/stevenheidel/events{/privacy}","received_events_url":"https://api.github.com/users/stevenheidel/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-10-16T19:13:50Z","updated_at":"2015-10-16T19:16:02Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/getclump/clump/pulls/109","html_url":"https://github.com/getclump/clump/pull/109","diff_url":"https://github.com/getclump/clump/pull/109.diff","patch_url":"https://github.com/getclump/clump/pull/109.patch"},"body":"__PREVIEW ONLY__: STILL NEED TO WRITE DOCS AND TESTS\r\n\r\nSome RPC systems (like rest.li used at LinkedIn) allow a batch response to contain both successful and failed entries. For example, if I fetch the profiles for ids 1 and 2, it's possible that 1 could return a profile and 2 will return an UnauthorizedAccess. Visualized:\r\n```scala\r\nMap(\r\n  1 -> Entity(Profile),\r\n  2 -> Error(UnauthorizedAccess)\r\n)\r\n```\r\n\r\nCurrently I have only two choices to deal with this:\r\n1. Remove all error entries from the Map. But then you lose all the error information which makes it difficult to debug why things are missing.\r\n2. If I need access to the error, I can fail the entire fetch, but then you lose the profiles that were successfully returned.\r\n\r\nThis change fixes that by introducing the ability to explicitly define a Try in the resulting map. If the Try is a Failure, then that Clump will be failed. Otherwise it will behave as normal.\r\n\r\nAdded a `.sourceTry` method to create these types of sources. Everything else stays the same. Now if you use `.sourceTry` then the example above changes to the expected behaviour:\r\n`profileSource.get(1)` will return a Clump containing the Profile.\r\n`profileSource.get(2)` fails the Clump with an UnauthorizedAccess exception.\r\n\r\nA good side effect of this is that `.sourceSingle` now handles exceptions much better as well. If one call to the single source throws an exception it only fails that Clump instead of failing everything in that \"batch\" fetch. I also took some time to refactor the single source creation into a parameterize method following the pattern of the other types of sources."},"comment":{"url":"https://api.github.com/repos/getclump/clump/issues/comments/148810599","html_url":"https://github.com/getclump/clump/pull/109#issuecomment-148810599","issue_url":"https://api.github.com/repos/getclump/clump/issues/109","id":148810599,"user":{"login":"stevenheidel","id":155646,"avatar_url":"https://avatars.githubusercontent.com/u/155646?v=3","gravatar_id":"","url":"https://api.github.com/users/stevenheidel","html_url":"https://github.com/stevenheidel","followers_url":"https://api.github.com/users/stevenheidel/followers","following_url":"https://api.github.com/users/stevenheidel/following{/other_user}","gists_url":"https://api.github.com/users/stevenheidel/gists{/gist_id}","starred_url":"https://api.github.com/users/stevenheidel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stevenheidel/subscriptions","organizations_url":"https://api.github.com/users/stevenheidel/orgs","repos_url":"https://api.github.com/users/stevenheidel/repos","events_url":"https://api.github.com/users/stevenheidel/events{/privacy}","received_events_url":"https://api.github.com/users/stevenheidel/received_events","type":"User","site_admin":false},"created_at":"2015-10-16T19:16:02Z","updated_at":"2015-10-16T19:16:02Z","body":"In the future this means we can also add the ability to retry individual resources. So for example, if the batch fetch returns the following map:\r\n```\r\nMap(\r\n  1 -> Entity(Profile),\r\n  2 -> Error(DatabaseTimeout),\r\n  3 -> Error(DatabaseTimeout)\r\n)\r\n```\r\n\r\nWe could send another fetch request with just keys 2 and 3."}}