{"action":"created","issue":{"url":"https://api.github.com/repos/nodejs/io.js/issues/1750","labels_url":"https://api.github.com/repos/nodejs/io.js/issues/1750/labels{/name}","comments_url":"https://api.github.com/repos/nodejs/io.js/issues/1750/comments","events_url":"https://api.github.com/repos/nodejs/io.js/issues/1750/events","html_url":"https://github.com/nodejs/io.js/pull/1750","id":78663534,"number":1750,"title":"[Discussion] FFI - Giving Buffer more low-level C functionality","user":{"login":"TooTallNate","id":71256,"avatar_url":"https://avatars.githubusercontent.com/u/71256?v=3","gravatar_id":"","url":"https://api.github.com/users/TooTallNate","html_url":"https://github.com/TooTallNate","followers_url":"https://api.github.com/users/TooTallNate/followers","following_url":"https://api.github.com/users/TooTallNate/following{/other_user}","gists_url":"https://api.github.com/users/TooTallNate/gists{/gist_id}","starred_url":"https://api.github.com/users/TooTallNate/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TooTallNate/subscriptions","organizations_url":"https://api.github.com/users/TooTallNate/orgs","repos_url":"https://api.github.com/users/TooTallNate/repos","events_url":"https://api.github.com/users/TooTallNate/events{/privacy}","received_events_url":"https://api.github.com/users/TooTallNate/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/nodejs/io.js/labels/buffer","name":"buffer","color":"f7c6c7"},{"url":"https://api.github.com/repos/nodejs/io.js/labels/c++","name":"c++","color":"e11d21"},{"url":"https://api.github.com/repos/nodejs/io.js/labels/discuss","name":"discuss","color":"02d7e1"},{"url":"https://api.github.com/repos/nodejs/io.js/labels/ffi","name":"ffi","color":"fbca04"},{"url":"https://api.github.com/repos/nodejs/io.js/labels/semver-minor","name":"semver-minor","color":"fef2c0"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":10,"created_at":"2015-05-20T19:31:13Z","updated_at":"2015-05-24T06:08:34Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/nodejs/io.js/pulls/1750","html_url":"https://github.com/nodejs/io.js/pull/1750","diff_url":"https://github.com/nodejs/io.js/pull/1750.diff","patch_url":"https://github.com/nodejs/io.js/pull/1750.patch"},"body":"This is a PR, with the intention of starting/[continuing](https://github.com/joyent/node/issues/8285) the discussion about bringing FFI into node/iojs core.\r\n\r\nOverall, the biggest benefit to integrating an FFI interface to node-core is that we can avoid C++ compiled addons, for the common case (where extreme performance requirements would be the uncommon case). No more compiling is huge for the UX, especially on Windows.\r\n\r\nAs of now, this PR adds some low-level functions to the Buffer object. I believe that these are the building blocks necessary to remove the C++ compiled portion of `ref`, and leaves room to implement other kinds of native interfaces (we could build `js-ctypes` API on top of these low-level functions as well… I'm planning on doing exactly that if this gains momentum!)\r\n\r\n# New Buffer APIs\r\n\r\n####  `readInt64BE()`, `readInt64LE()`, `readUInt64BE()`, and `readUInt64LE()`\r\n\r\nReads an (u)int64 value from the Buffer. Similar to the other read* functions, but if the int64 value falls outside the JS Number max value (±9007199254740992) then a String will be returned instead of a Number. This falls in-line with most Int64 library APIs, including the [ctypes Int64 constructor](https://developer.mozilla.org/en-US/docs/Mozilla/js-ctypes/js-ctypes_reference/Int64#Int64()), which accepts either a Number or String.\r\n\r\n---\r\n\r\n####  `writeInt64BE()`, `writeInt64LE()`, `writeUInt64BE()`, and `writeUInt64LE()`\r\n\r\nComplimentary to the read* functions, these allow int64 range integers to be written to Buffer instances. A JS Number may be passed in, and it will be written accordingly. Alternatively, a JS String may be passed in and it will be converted to an (u)int64 via `strtoll`/`strtoull` and then written to the Buffer instance (there would probably be value in accepting a Buffer instance that already has a (u)int64 written to it, but that's not currently implemented).\r\n\r\n---\r\n\r\n#### `readPointerBE()` and `readPointerLE()`\r\n\r\nThese functions create a Buffer (pointer) instance from the memory address written to the parent buffer. This is essentially the \"dereference\" operator in C. Very important for interacting with C libraries via FFI!\r\n\r\nNot implemented at the moment, but we could add an optional callback function here that would be invoked during the Buffer free callback, which the user could hook on to to free allocated memory from a C library, for example. This is easy to add if there's no objections.\r\n\r\n---\r\n\r\n#### `writePointerBE()` and `writePointerLE()`\r\n\r\nThe compliment to the readPointer* functions. Essentially the \"reference\" operator in C. Allows the user to write the memory address of one Buffer instance to an offset in another buffer.\r\n\r\n---\r\n\r\n#### `address()`\r\n\r\nReturns a String representation of the hex address of the pointer in memory.\r\n\r\n----\r\n\r\n#### `inspect()`\r\n\r\nThe REPL `inspect()` function for Buffer gets a cute little update as well thanks to the new `address()` function. We include the hex pointer address after the constructor name:\r\n\r\n```\r\n> new Buffer(8)\r\n<Buffer@0x10187aa08 00 20 00 00 00 00 00 00>\r\n```\r\n\r\n# Missing pieces\r\n\r\nSo like mentioned above, this is really just the low-level C functionality that we need added as methods to Buffer. There's still some new interfaces we'll need to expose in order to implement a fully-featured FFI interface, or implement `js-ctypes`. Perhaps we can hash out the details of these in this thread as well:\r\n\r\n * <del>`sizeof` mapping for standard types (`int`, `double`, etc.), for allocing Buffer instances of the correct size. Perhaps `require('os').sizeof`?</del> **See #1759**.\r\n * <del>-`alignof` mapping for standard types (`int`, `double`, etc.), for creating structs, etc. Perhaps `require('os').alignof`?</del> **See #1759**.\r\n * <del>dynamic library loader (`uv_dlopen()` and friends, but exposed to JS). Perhaps `require('_dl')` or something? There is the [`dlopen`](https://github.com/tootallnate/node-dlopen) package which we could get inspiration from.</del> **See #1762**. Additionally, this could allow for `process.dlopen()` (the `.node` module loader) to be re-written in 90% JavaScript, which would be kind of cool.\r\n * low-level libffi bindings (bundle the [gypified `libffi`](https://github.com/gypified/libffi) into the node binary, expose a very basic JS interface). Perhaps `require('_ffi')`? [`node-ffi`](https://github.com/node-ffi/node-ffi) is great, but it's a bit too high-level for core IMO. It aims to be a more complete FFI solution, a la ctypes, rather than simple low-level bindings in which ctypes could be implemented on top of.\r\n\r\n(not sure about the `_` prefix on ffi and dl... just wanted to be conscious about not trampling over the existing npm package names)\r\n\r\n---\r\n\r\nObviously, we'll need to compliment this with good docs and tests (some of which I could pull from `ref` most likely) before it's ready for merge. I just wanted to get this PR open first to see if there's momentum, and other comments/adjustments that will be needed to be addressed as well. Let's get the :crystal_ball: rolling!\r\n\r\nThanks for reading. Cheers! :beers:\r\n\r\n/cc @bnoordhuis @piscisaureus @rvagg @indutny @isaacs @kkoopa @trevnorris @rauchg @saghul @tracker1 @nodejs/build et al."},"comment":{"url":"https://api.github.com/repos/nodejs/io.js/issues/comments/104983359","html_url":"https://github.com/nodejs/io.js/pull/1750#issuecomment-104983359","issue_url":"https://api.github.com/repos/nodejs/io.js/issues/1750","id":104983359,"user":{"login":"trevnorris","id":289174,"avatar_url":"https://avatars.githubusercontent.com/u/289174?v=3","gravatar_id":"","url":"https://api.github.com/users/trevnorris","html_url":"https://github.com/trevnorris","followers_url":"https://api.github.com/users/trevnorris/followers","following_url":"https://api.github.com/users/trevnorris/following{/other_user}","gists_url":"https://api.github.com/users/trevnorris/gists{/gist_id}","starred_url":"https://api.github.com/users/trevnorris/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/trevnorris/subscriptions","organizations_url":"https://api.github.com/users/trevnorris/orgs","repos_url":"https://api.github.com/users/trevnorris/repos","events_url":"https://api.github.com/users/trevnorris/events{/privacy}","received_events_url":"https://api.github.com/users/trevnorris/received_events","type":"User","site_admin":false},"created_at":"2015-05-24T06:08:34Z","updated_at":"2015-05-24T06:08:34Z","body":"@TooTallNate Finished my second round of review."}}