{"action":"created","issue":{"url":"https://api.github.com/repos/relentless/Http.fs/issues/65","labels_url":"https://api.github.com/repos/relentless/Http.fs/issues/65/labels{/name}","comments_url":"https://api.github.com/repos/relentless/Http.fs/issues/65/comments","events_url":"https://api.github.com/repos/relentless/Http.fs/issues/65/events","html_url":"https://github.com/relentless/Http.fs/issues/65","id":66426810,"number":65,"title":"[Body] Support the different sorts of bodies","user":{"login":"haf","id":193115,"avatar_url":"https://avatars.githubusercontent.com/u/193115?v=3","gravatar_id":"","url":"https://api.github.com/users/haf","html_url":"https://github.com/haf","followers_url":"https://api.github.com/users/haf/followers","following_url":"https://api.github.com/users/haf/following{/other_user}","gists_url":"https://api.github.com/users/haf/gists{/gist_id}","starred_url":"https://api.github.com/users/haf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/haf/subscriptions","organizations_url":"https://api.github.com/users/haf/orgs","repos_url":"https://api.github.com/users/haf/repos","events_url":"https://api.github.com/users/haf/events{/privacy}","received_events_url":"https://api.github.com/users/haf/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-04-05T10:45:11Z","updated_at":"2015-05-12T07:23:51Z","closed_at":null,"body":"### Description\r\n\r\nAs a part of making Http.fs a nice general-purpose Http client for .Net, we should support `multipart/form-data` bodies. They are needed to avoid the exploding size of application/x-www-form-urlencoded. They are needed to name file uploads and to upload files in general. By using them, we can set a MIME type for each body entity.\r\n\r\n## Implementation\r\n\r\nSuggesting that we implement a discriminated union for each of:\r\n\r\n - multipart/form-data bodies\r\n   - multipart/form-data -- **DONE**\r\n   - multipart/mixed inside form-data -- **DONE**\r\n   - plain encoding of files -- **DONE**\r\n   - base64 encoding of files -- **DONE**\r\n   - binary encoding of files -- this could be done when we have the 'socket' type, by composing it nicely\r\n - application/x-www-form-urlencoded -- **DONE**\r\n - raw bytes\r\n   - base64 encoded bytes -- **DONE**\r\n   - binary write-straight-to-stream bytes\r\n - socket -- awaiting discussion\r\n\r\nThe raw-bytes body is well put to use for intermediate uploads, from 30MB to half a gig, depending on your machines' memory configuration.\r\n\r\nThe socket body can be used to stream data to the target web server, and if we choose to do #64, will fit well together with it. It would allow plugins like a websocket client, SSE client, webrtc client etc.\r\n\r\nThe form-data body type will contain randomized boundaries, so there should be a way to generate bodies deterministically.\r\n\r\n### Handling data structure\r\n\r\nWe have these sorts of data; binary blob, streamed (both of which are easy and low on abstraction), form-data (key-value like), file-data (key-key-mime-base64 like). Both form-data and file-data can be formatted for `multipart/form-data` and `application/x-www-form-urlencoded` so I suggest a **typed model** for key-value and key-key-mime-base64 typed values, with can then be written using **formatters**. RawBytes bodies could be a specialisation of Socket (/cc @ademar - couldn't we do this for the new 'writePreamble' in Suave, too? Composing writers, one of which would be responsible for writing the preamble, into the SocketTask?).\r\n\r\nLet's see what I can cook; right now I have a need for form-data and urlencoded, so I will focus on that."},"comment":{"url":"https://api.github.com/repos/relentless/Http.fs/issues/comments/101165209","html_url":"https://github.com/relentless/Http.fs/issues/65#issuecomment-101165209","issue_url":"https://api.github.com/repos/relentless/Http.fs/issues/65","id":101165209,"user":{"login":"haf","id":193115,"avatar_url":"https://avatars.githubusercontent.com/u/193115?v=3","gravatar_id":"","url":"https://api.github.com/users/haf","html_url":"https://github.com/haf","followers_url":"https://api.github.com/users/haf/followers","following_url":"https://api.github.com/users/haf/following{/other_user}","gists_url":"https://api.github.com/users/haf/gists{/gist_id}","starred_url":"https://api.github.com/users/haf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/haf/subscriptions","organizations_url":"https://api.github.com/users/haf/orgs","repos_url":"https://api.github.com/users/haf/repos","events_url":"https://api.github.com/users/haf/events{/privacy}","received_events_url":"https://api.github.com/users/haf/received_events","type":"User","site_admin":false},"created_at":"2015-05-12T07:23:51Z","updated_at":"2015-05-12T07:23:51Z","body":"I've had some time to test my branch with some production code and it seems there are a few issues in many web servers accepting base64-encoded data over just the inline-binary-blob that is the alternative.\r\n\r\nI've updated the TODO list with this.\r\n\r\nI would like to get your input on the getResponse changes that I've done - basically allowing you to get the raw bytes if you're not getting a string. So while the ability to get bytes is very important, the current API I rushed through to support it (getResponse(Async|Body|...)) is shit and requires you to check for ResponseString | ResponseBytes every time. So I want to get help to design that API:\r\n\r\n - We need a getResponse function that returns 'everything' in a way that doesn't pre-consume the stream - what if I'm downloading a large image for example (my use-case) and I just want to proxy/forward the stream -- (Suave.Proxy here we come)? In that case it doesn't make sense that the client does all the Content-Type matching -> returning the discriminated union corresponding to the header. Instead there should be a Response and the option to get a body, just like MSFT's HttpClient has.\r\n - I've been looking at @h2non's amazing work with https://www.npmjs.com/package/resilient -- I'd love for our language to have a similar http client. Perhaps you could browse his code and steal some of his ideas for this project? =)"}}