{"action":"created","issue":{"url":"https://api.github.com/repos/greghendershott/aws/issues/48","labels_url":"https://api.github.com/repos/greghendershott/aws/issues/48/labels{/name}","comments_url":"https://api.github.com/repos/greghendershott/aws/issues/48/comments","events_url":"https://api.github.com/repos/greghendershott/aws/issues/48/events","html_url":"https://github.com/greghendershott/aws/issues/48","id":104072358,"number":48,"title":"multipart-put and multipart-put/file: Better error handling","user":{"login":"greghendershott","id":265119,"avatar_url":"https://avatars.githubusercontent.com/u/265119?v=3","gravatar_id":"","url":"https://api.github.com/users/greghendershott","html_url":"https://github.com/greghendershott","followers_url":"https://api.github.com/users/greghendershott/followers","following_url":"https://api.github.com/users/greghendershott/following{/other_user}","gists_url":"https://api.github.com/users/greghendershott/gists{/gist_id}","starred_url":"https://api.github.com/users/greghendershott/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/greghendershott/subscriptions","organizations_url":"https://api.github.com/users/greghendershott/orgs","repos_url":"https://api.github.com/users/greghendershott/repos","events_url":"https://api.github.com/users/greghendershott/events{/privacy}","received_events_url":"https://api.github.com/users/greghendershott/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/greghendershott/aws/labels/enhancement","name":"enhancement","color":"bfe5bf"}],"state":"open","locked":false,"assignee":{"login":"greghendershott","id":265119,"avatar_url":"https://avatars.githubusercontent.com/u/265119?v=3","gravatar_id":"","url":"https://api.github.com/users/greghendershott","html_url":"https://github.com/greghendershott","followers_url":"https://api.github.com/users/greghendershott/followers","following_url":"https://api.github.com/users/greghendershott/following{/other_user}","gists_url":"https://api.github.com/users/greghendershott/gists{/gist_id}","starred_url":"https://api.github.com/users/greghendershott/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/greghendershott/subscriptions","organizations_url":"https://api.github.com/users/greghendershott/orgs","repos_url":"https://api.github.com/users/greghendershott/repos","events_url":"https://api.github.com/users/greghendershott/events{/privacy}","received_events_url":"https://api.github.com/users/greghendershott/received_events","type":"User","site_admin":false},"milestone":null,"comments":2,"created_at":"2015-08-31T15:08:44Z","updated_at":"2015-09-03T15:10:30Z","closed_at":null,"body":"As discussed in #46, the convenience functions `multipart-put` and `multipart-put/file` ought to handle things like `exn:fail:network?`. After all, being able to resume an interrupted upload is one of the main advantages of multipart uploads. Also, as a default in case the user doesn't want to deal with attempting to resume, the functions should automatically use `abort-multipart-upload` to clean up (so the user isn't paying for parts sitting on S3).\r\n\r\nQuick/rough brain dump:\r\n\r\n- First, just double-check that the `upload-part` function is handling things like 5xx errors with exponential retry. i.e. Better not to fail at all, if possible.\r\n- Perhaps the put functions should take a new `failure-proc` arg that defaults to `abort-multipart-upload`, but may be a user-supplied function that stores the upload-id and parts-list, and can give them to a new `resume-multipart-upload` function?\r\n- Should there be a new `suspend-multipart-upload` function to interrupt intentionally?  (That could be called from a break handler, for example?)\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/greghendershott/aws/issues/comments/137479849","html_url":"https://github.com/greghendershott/aws/issues/48#issuecomment-137479849","issue_url":"https://api.github.com/repos/greghendershott/aws/issues/48","id":137479849,"user":{"login":"greghendershott","id":265119,"avatar_url":"https://avatars.githubusercontent.com/u/265119?v=3","gravatar_id":"","url":"https://api.github.com/users/greghendershott","html_url":"https://github.com/greghendershott","followers_url":"https://api.github.com/users/greghendershott/followers","following_url":"https://api.github.com/users/greghendershott/following{/other_user}","gists_url":"https://api.github.com/users/greghendershott/gists{/gist_id}","starred_url":"https://api.github.com/users/greghendershott/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/greghendershott/subscriptions","organizations_url":"https://api.github.com/users/greghendershott/orgs","repos_url":"https://api.github.com/users/greghendershott/repos","events_url":"https://api.github.com/users/greghendershott/events{/privacy}","received_events_url":"https://api.github.com/users/greghendershott/received_events","type":"User","site_admin":false},"created_at":"2015-09-03T15:10:30Z","updated_at":"2015-09-03T15:10:30Z","body":"A `suspend` function will be needed internally, to use when handling e.g. `exn:fail` exceptions. Multipart uploads use a pool of 4 threads. If one fails unrecoverably then the whole pool needs to be stopped gracefully.\r\n\r\nAfter that's figured out, correctly, I think it would be helpful to `provide` it, too.\r\n\r\nExample: Racket presents break, kill and hangup signals as `exn:break` exceptions. If the aws client is going to be killed intentionally, it would be helpful for it to catch these using `with-handlers` or `call-with-exception-handler` and suspend the multipart upload in a way that could be resumed later. (For example in many places residential broadband is not so fast, especially at uploads. Having to start over from scratch isn't great.)\r\n\r\nUnlike `exn:fail`, I feel breaks should be left to the client of the aws library to handle, and it might want to use a `suspend`. Although I suppose I could handle breaks and re-`raise` them for the client. Just thinking out loud, here.\r\n\r\n---\r\n\r\nIn any case, it may be awhile before I can work on this item, at all...."}}