{"action":"closed","issue":{"url":"https://api.github.com/repos/kezabelle/django-jackfrost/issues/9","labels_url":"https://api.github.com/repos/kezabelle/django-jackfrost/issues/9/labels{/name}","comments_url":"https://api.github.com/repos/kezabelle/django-jackfrost/issues/9/comments","events_url":"https://api.github.com/repos/kezabelle/django-jackfrost/issues/9/events","html_url":"https://github.com/kezabelle/django-jackfrost/issues/9","id":85461470,"number":9,"title":"Split URLBuilder into URLReader and URLWriter?","user":{"login":"kezabelle","id":118377,"avatar_url":"https://avatars.githubusercontent.com/u/118377?v=3","gravatar_id":"","url":"https://api.github.com/users/kezabelle","html_url":"https://github.com/kezabelle","followers_url":"https://api.github.com/users/kezabelle/followers","following_url":"https://api.github.com/users/kezabelle/following{/other_user}","gists_url":"https://api.github.com/users/kezabelle/gists{/gist_id}","starred_url":"https://api.github.com/users/kezabelle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kezabelle/subscriptions","organizations_url":"https://api.github.com/users/kezabelle/orgs","repos_url":"https://api.github.com/users/kezabelle/repos","events_url":"https://api.github.com/users/kezabelle/events{/privacy}","received_events_url":"https://api.github.com/users/kezabelle/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-06-05T07:27:34Z","updated_at":"2015-06-09T19:17:23Z","closed_at":"2015-06-09T19:17:23Z","body":"URLReader should fetch all the responses, and shove them into an OrderedDict where the key is the target output filename and the value is the response content itself.\r\n\r\nURLWriter should take the data from URLReader and write it to the storage backend defined.\r\n\r\nThis'll end up taking up more memory, especially if files are returned (eg PDFs, streaming responses etc) but does mean it'd be easier to hash the file contents and keep a store of that, and avoid writing unchanged files to the backend at all. This would be most useful in non-local-filesystem backends, I suspect, where the latency wouldn't be worth worrying about."}}