{"action":"created","issue":{"url":"https://api.github.com/repos/jstedfast/MimeKit/issues/113","labels_url":"https://api.github.com/repos/jstedfast/MimeKit/issues/113/labels{/name}","comments_url":"https://api.github.com/repos/jstedfast/MimeKit/issues/113/comments","events_url":"https://api.github.com/repos/jstedfast/MimeKit/issues/113/events","html_url":"https://github.com/jstedfast/MimeKit/issues/113","id":62787890,"number":113,"title":"Suggestion for Content-Length extension header, to avoid base64 encoding","user":{"login":"polterguy","id":8466933,"avatar_url":"https://avatars.githubusercontent.com/u/8466933?v=3","gravatar_id":"","url":"https://api.github.com/users/polterguy","html_url":"https://github.com/polterguy","followers_url":"https://api.github.com/users/polterguy/followers","following_url":"https://api.github.com/users/polterguy/following{/other_user}","gists_url":"https://api.github.com/users/polterguy/gists{/gist_id}","starred_url":"https://api.github.com/users/polterguy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/polterguy/subscriptions","organizations_url":"https://api.github.com/users/polterguy/orgs","repos_url":"https://api.github.com/users/polterguy/repos","events_url":"https://api.github.com/users/polterguy/events{/privacy}","received_events_url":"https://api.github.com/users/polterguy/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-03-18T20:14:28Z","updated_at":"2015-03-18T20:48:48Z","closed_at":null,"body":"Howdy, I was thinking about your answer to my previous question, where you said that text/xxx parts didn't need to be base64 encoded, due to that the boundary was \"statistically globally unique\", due to its underlying randomness, and it made me start thinking ...\r\n\r\nWould it be totally out of the question to create support for an \"extension header\" named e.g. \"Content-Length\", which would completely eliminate the needs to base64 encoding, also of binary data, I think, since then MimeKit could just read a \"fixed length\", if that header exists, instead of looking for a boundary, meaning the boundary becomes meaningless for such cases, making it possible to avoid base64 encoding of also binary types of data ...?\r\n\r\nI know this might sound like EEE, and \"destroying of standards\", but I don't really think so, since _only_ if that header exists, MimeKit would go into \"read a fixed length bytes\", instead of reading until boundary. Besides, the boundary could still be created as usual, making clients that don't support it, probably just simply ignore it, and read until boundary. While MimeKit, which supports that header, would then instead of reading until \"boundary\", would read a fixed length bytes.\r\n\r\nFirst of all, I think this might improve the load-speed of MimeParts, since reading a fixed length, obviously must be faster than to always having to check for the boundary during read operations. Beside, and more importantly, not having to base64 encode stuff, while still having a _guarantee_ of that the boundary does not exists within binary data, would save apprx. 33% of space, for every binary data ...\r\n\r\nMaybe it's a stupid suggestion, and maybe it wouldn't work for other reasons, which I cannot see, but think about it ...\r\n\r\nI realize this would be an \"edge-case\", but I think that MIME in general serves greater purposes than \"only email stuff\", which is why I am creating a WebService API on top of it, by combining REST Web Services with MIME for instance ...\r\n\r\nI know this sounds \"weird\" probably, but let's say you query a REST Web Service for \"contacts\", at e.g. someurl.com/contacts. By returning multipart messages, instead of for instance pure JSON or XML, one can easily allow returning \"associated objects\", such as \"Contact Images\" in the same request, reducing the number of HTTP requests a Web Service client needs to do to request all \"contacts\".\r\n\r\nI think the above use-case is kind of in the spirit of HTTP2.0 ...?"},"comment":{"url":"https://api.github.com/repos/jstedfast/MimeKit/issues/comments/83166058","html_url":"https://github.com/jstedfast/MimeKit/issues/113#issuecomment-83166058","issue_url":"https://api.github.com/repos/jstedfast/MimeKit/issues/113","id":83166058,"user":{"login":"jstedfast","id":338984,"avatar_url":"https://avatars.githubusercontent.com/u/338984?v=3","gravatar_id":"","url":"https://api.github.com/users/jstedfast","html_url":"https://github.com/jstedfast","followers_url":"https://api.github.com/users/jstedfast/followers","following_url":"https://api.github.com/users/jstedfast/following{/other_user}","gists_url":"https://api.github.com/users/jstedfast/gists{/gist_id}","starred_url":"https://api.github.com/users/jstedfast/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jstedfast/subscriptions","organizations_url":"https://api.github.com/users/jstedfast/orgs","repos_url":"https://api.github.com/users/jstedfast/repos","events_url":"https://api.github.com/users/jstedfast/events{/privacy}","received_events_url":"https://api.github.com/users/jstedfast/received_events","type":"User","site_admin":false},"created_at":"2015-03-18T20:48:47Z","updated_at":"2015-03-18T20:48:47Z","body":"MIME actually already supports `Content-Transfer-Encoding: binary` and MimeKit already supports it :-)\r\n\r\nA `Content-Length` header is not actually needed for that since even binary data is not likely to contain the random set of characters that make up the boundary string. There is actually already a `Content-Length` header in use where the purpose is more-or-less exactly what you have envisioned, but it was never accepted as a standard because of the need to maintain compatibility with existing software that followed specifications that did not have that requirement.\r\n\r\nFor example, say MimeKit sends a multipart containing an unencoded binary attachment that contains within it the exact boundary sequence that would signal to a parser that it has reached the end of the content. BUT! MimeKit adds a Content-Length header to specify the binary content length... well, that's all great and wonderful for receiving clients that implement support for Content-Length, but if they don't, then they'll blow up.\r\n\r\nIt also has problems when data passes through a system that uses a different end-of-line sequence, since the Content-Length header value relies on byte counts which change if the end-of-line sequence is CRLF vs LF. For the transport phase, it doesn't matter since the convention is to canonicalize to CRLF to send over the wire, but mail software tends to store the message in the system format.\r\n\r\nSince the boundary is of the form `\"\\r\\n--{boundary}\\r\\n\"` or `\"\\r\\n--{boundary}--\\r\\n\"`, it's still incredibly rare for even binary data to contain the boundary string sequence, so it's still not necessary to base64 encode or anything simply based on binary data vs text. Arguably, text is more likely to contain the sequence than binary data is."}}