{"action":"created","issue":{"url":"https://api.github.com/repos/whatwg/streams/issues/253","labels_url":"https://api.github.com/repos/whatwg/streams/issues/253/labels{/name}","comments_url":"https://api.github.com/repos/whatwg/streams/issues/253/comments","events_url":"https://api.github.com/repos/whatwg/streams/issues/253/events","html_url":"https://github.com/whatwg/streams/issues/253","id":51620411,"number":253,"title":"Support reading bytes into buffers allocated by user code on platforms where only async read is available","user":{"login":"domenic","id":617481,"avatar_url":"https://avatars.githubusercontent.com/u/617481?v=3","gravatar_id":"","url":"https://api.github.com/users/domenic","html_url":"https://github.com/domenic","followers_url":"https://api.github.com/users/domenic/followers","following_url":"https://api.github.com/users/domenic/following{/other_user}","gists_url":"https://api.github.com/users/domenic/gists{/gist_id}","starred_url":"https://api.github.com/users/domenic/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/domenic/subscriptions","organizations_url":"https://api.github.com/users/domenic/orgs","repos_url":"https://api.github.com/users/domenic/repos","events_url":"https://api.github.com/users/domenic/events{/privacy}","received_events_url":"https://api.github.com/users/domenic/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/whatwg/streams/labels/byte%20streams","name":"byte streams","color":"5319e7"}],"state":"open","locked":false,"assignee":{"login":"domenic","id":617481,"avatar_url":"https://avatars.githubusercontent.com/u/617481?v=3","gravatar_id":"","url":"https://api.github.com/users/domenic","html_url":"https://github.com/domenic","followers_url":"https://api.github.com/users/domenic/followers","following_url":"https://api.github.com/users/domenic/following{/other_user}","gists_url":"https://api.github.com/users/domenic/gists{/gist_id}","starred_url":"https://api.github.com/users/domenic/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/domenic/subscriptions","organizations_url":"https://api.github.com/users/domenic/orgs","repos_url":"https://api.github.com/users/domenic/repos","events_url":"https://api.github.com/users/domenic/events{/privacy}","received_events_url":"https://api.github.com/users/domenic/received_events","type":"User","site_admin":false},"milestone":null,"comments":88,"created_at":"2014-12-10T22:32:46Z","updated_at":"2015-03-05T21:54:49Z","closed_at":null,"body":"Had a thought-provoking conversation with @piscisaureus today. He works on libuv and was giving me their perspective on how they do I/O and how it interfaces with Node.js's streams.\r\n\r\nHe pointed out that resources like files and pipes shared with other processes (seekable resources, he called them) don't have the epoll + read interface that ReadableByteStream's current design is based on. In Node.js, they do blocking I/O in a thread pool with pre-allocated buffers.\r\n\r\nThis works for a `ready` + `read()` JS interface, since then the implementation can pre-allocate (say) a 1 MiB buffer, read into it off-thread using blocking I/O, then fulfill the `ready` promise. When JS calls `read()`, it just returns the pre-allocated buffer.\r\n\r\nIt doesn't work as well with ReadableByteStream's `ready` + `readInto(ab, offset, sizeDesired)` interface, since we don't know what size buffer to allocate until too late. If we pre-allocate something too small, `readInto` will always be returning a number below the desired `size`, which is a bit user-hostile. If we pre-allocate too large, we are at the least wasting memory, and at the worst getting ourselves into a bunch of trouble as we need to figure out how to merge the \"leftovers\" with other chunks we read in the future.\r\n\r\n(I am assuming here it is easy in V8 to map the [offset, offset + max(sizeRead, sizeDesired)] portion of `ab` onto a given C++ backing buffer.)\r\n\r\nA `readIntoAsync(ab, offset, desiredSize)` model would work a bit better, since then we'd know to pre-allocate a buffer of size `desiredSize`. But, I was curious if you guys had any other thoughts? Did you think of this issue when designing the API, @tyoshino? I imagine @willchan could be helpful here too."},"comment":{"url":"https://api.github.com/repos/whatwg/streams/issues/comments/77460396","html_url":"https://github.com/whatwg/streams/issues/253#issuecomment-77460396","issue_url":"https://api.github.com/repos/whatwg/streams/issues/253","id":77460396,"user":{"login":"tyoshino","id":4511440,"avatar_url":"https://avatars.githubusercontent.com/u/4511440?v=3","gravatar_id":"","url":"https://api.github.com/users/tyoshino","html_url":"https://github.com/tyoshino","followers_url":"https://api.github.com/users/tyoshino/followers","following_url":"https://api.github.com/users/tyoshino/following{/other_user}","gists_url":"https://api.github.com/users/tyoshino/gists{/gist_id}","starred_url":"https://api.github.com/users/tyoshino/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tyoshino/subscriptions","organizations_url":"https://api.github.com/users/tyoshino/orgs","repos_url":"https://api.github.com/users/tyoshino/repos","events_url":"https://api.github.com/users/tyoshino/events{/privacy}","received_events_url":"https://api.github.com/users/tyoshino/received_events","type":"User","site_admin":false},"created_at":"2015-03-05T21:54:49Z","updated_at":"2015-03-05T21:54:49Z","body":"We may use { EOS, true } for the distinction. Then, the `read(undefined)` user needs to check `value` but still see get `done`. Confusing."}}