{"action":"created","issue":{"url":"https://api.github.com/repos/MightyPirates/OpenComputers/issues/1077","labels_url":"https://api.github.com/repos/MightyPirates/OpenComputers/issues/1077/labels{/name}","comments_url":"https://api.github.com/repos/MightyPirates/OpenComputers/issues/1077/comments","events_url":"https://api.github.com/repos/MightyPirates/OpenComputers/issues/1077/events","html_url":"https://github.com/MightyPirates/OpenComputers/issues/1077","id":69149964,"number":1077,"title":"Data processor block/upgrade.","user":{"login":"Kubuxu","id":2259282,"avatar_url":"https://avatars.githubusercontent.com/u/2259282?v=3","gravatar_id":"","url":"https://api.github.com/users/Kubuxu","html_url":"https://github.com/Kubuxu","followers_url":"https://api.github.com/users/Kubuxu/followers","following_url":"https://api.github.com/users/Kubuxu/following{/other_user}","gists_url":"https://api.github.com/users/Kubuxu/gists{/gist_id}","starred_url":"https://api.github.com/users/Kubuxu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Kubuxu/subscriptions","organizations_url":"https://api.github.com/users/Kubuxu/orgs","repos_url":"https://api.github.com/users/Kubuxu/repos","events_url":"https://api.github.com/users/Kubuxu/events{/privacy}","received_events_url":"https://api.github.com/users/Kubuxu/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/MightyPirates/OpenComputers/labels/feature-suggested","name":"feature-suggested","color":"bfe5bf"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-04-17T14:29:52Z","updated_at":"2015-04-17T17:55:18Z","closed_at":null,"body":"Data processor would allow advanced data processing (pun not intended).\r\nIts main purpose would be to provide functions for bulk data storage or lower level functions on data.\r\n\r\nProposed functions:\r\n\r\n* ```tobytes(primitive: Primitive) -> bytes: ByteArray (String with non writeable characters)``` - returns byte encoded primitive.\r\n* ```frombytes(bytes: ByteArray, type: String) -> Primitive (of type 'type')``` - returns decoded primitive from bytes.\r\n* ```deflate(bytes: ByteArray) -> ByteArray``` - applies DEFLATE compression.\r\n* ```inflate(bytes: ByteArray) -> ByteArray``` - applies INFLATE decompression.\r\n* ```dataoutputstream(size: Number) -> Object``` - returns an object which exposes methods/has methods similar to [DataOutputStream](http://docs.oracle.com/javase/7/docs/api/java/io/DataOutputStream.html). Preallocates memory of size bytes and releases it after usage of ```tobytes()``` which also locks the stream up (only calls to ```tobytes()``` are valid, rest throw errors).\r\n* ```datainputstream(bytes :ByteArray) -> Object``` - returns an object which exposes methods/has methods similar to [DataInputStream](http://docs.oracle.com/javase/7/docs/api/java/io/DataInputStream.html)\r\n* ```endianness() -> String``` - returns \"big\" in case of big-endian systems and \"little\" in case of little-endian systems.\r\n* ```encode64(bytes: ByteArray) -> ByteArray``` - returns bytes encoded to base64.\r\n* ```decode64(bytes: ByteArray) -> ByteArray``` - returns bytes decoded from bese64.\r\n\r\nAlmost all of those functions can be implemented from inside of Lua *but* the implementation is never perfect and is usually very slow; [Example](https://gist.github.com/Kubuxu/e5e04c028d8aaeab4be8).\r\n\r\nEDIT:\r\nApplying @dgelessus suggestions.\r\n* ```tobytes(data1: Primitive[, data2: Primitive[, ...]]) -> ByteArray, format: String``` - throws error if There is more than one argument and it is string with ```null``` byte in it.\r\n* ```tobytesf(format: String, ...) -> ByteArray, format: String``` - Accepts format string and variable number of arguments. Returns ByteArray and the same format string for consistency with ```tobytes```.\r\n* ```frombyte```-  not changed.\r\n* ```frombytef(format: String, bytes: ByteArray) -> Primitive, ...``` - pareses the byte array according to given format string.\r\n* **Format String** uses single characets to denote Lua's basic types:\r\n    * `d`: for double\r\n    * `b`: for boolean\r\n    * `s`: for string (String can't contain `0` byte.)\r\n    * `r` (like `raw`): for byte array. `r` is required to be followed by decimal number of bytes.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/MightyPirates/OpenComputers/issues/comments/94039205","html_url":"https://github.com/MightyPirates/OpenComputers/issues/1077#issuecomment-94039205","issue_url":"https://api.github.com/repos/MightyPirates/OpenComputers/issues/1077","id":94039205,"user":{"login":"dgelessus","id":6641959,"avatar_url":"https://avatars.githubusercontent.com/u/6641959?v=3","gravatar_id":"","url":"https://api.github.com/users/dgelessus","html_url":"https://github.com/dgelessus","followers_url":"https://api.github.com/users/dgelessus/followers","following_url":"https://api.github.com/users/dgelessus/following{/other_user}","gists_url":"https://api.github.com/users/dgelessus/gists{/gist_id}","starred_url":"https://api.github.com/users/dgelessus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dgelessus/subscriptions","organizations_url":"https://api.github.com/users/dgelessus/orgs","repos_url":"https://api.github.com/users/dgelessus/repos","events_url":"https://api.github.com/users/dgelessus/events{/privacy}","received_events_url":"https://api.github.com/users/dgelessus/received_events","type":"User","site_admin":false},"created_at":"2015-04-17T17:55:18Z","updated_at":"2015-04-17T17:55:18Z","body":"Maybe `tobytes` should accept both forms. If given a single argument, that object dumped in some default form. If there are multiple arguments, the first one is a format string and all others are the values to be dumped.\r\n\r\n`frombytes` always needs an extra parameter to tell it in what format the input bytes are. This could accept both Lua type names and format strings, since `boolean`, `number`, `string` and `table` are not valid format strings. When loading more than one value, strings should *always* require an explicit length. That way single strings can always be dumped/loaded without needing a format string, and for multiple strings there's no need to rely on null bytes.\r\n\r\nRegarding endianness, there is no situation in which it would be necessary to know the real computer's endianness. None of the functions usable from Lua manipulate bytes in a way that endianness matters. The data processor component would be the only exception, and its default endianness should be consistent, no matter what the host computer uses."}}