{"action":"created","issue":{"url":"https://api.github.com/repos/atom/asar/issues/16","labels_url":"https://api.github.com/repos/atom/asar/issues/16/labels{/name}","comments_url":"https://api.github.com/repos/atom/asar/issues/16/comments","events_url":"https://api.github.com/repos/atom/asar/issues/16/events","html_url":"https://github.com/atom/asar/issues/16","id":52599393,"number":16,"title":"Changes to the asar format? Magic number, checksum, filesize","user":{"login":"bwin","id":1673361,"avatar_url":"https://avatars.githubusercontent.com/u/1673361?v=3","gravatar_id":"","url":"https://api.github.com/users/bwin","html_url":"https://github.com/bwin","followers_url":"https://api.github.com/users/bwin/followers","following_url":"https://api.github.com/users/bwin/following{/other_user}","gists_url":"https://api.github.com/users/bwin/gists{/gist_id}","starred_url":"https://api.github.com/users/bwin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bwin/subscriptions","organizations_url":"https://api.github.com/users/bwin/orgs","repos_url":"https://api.github.com/users/bwin/repos","events_url":"https://api.github.com/users/bwin/events{/privacy}","received_events_url":"https://api.github.com/users/bwin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2014-12-21T16:28:26Z","updated_at":"2015-01-26T23:22:19Z","closed_at":null,"body":"Currently an asar archive consists of these parts:\r\n```\r\nHEADERSIZE HEADER FILES\r\n```\r\n\r\nI would like to add a magic number(/string) to the beginning like 'ASAR' or something similar.\r\nThis would be backwards incompatible, but we could read it in the old format if it doesn't start with the magic string and show a deprecation notice for now.\r\nWhy? Currently we would read any file, interpret the first 8 bytes as UINT64 and try to read that much bytes and try to interpret that as json. Although this works, IMHO a magic number (or file signature) would be a nicer way.\r\n\r\nIf you're (@zcbenz) ok with the idea, how about adding a checksum too? (When you do incompatible changes do them all at once.) In normal operation the checksum can be ignored, but it could be used to check the integrity of the file when someone wants to.\r\nIt would also be possible to put the checksum into the json-header, but I don't like that.\r\n\r\nI'm also in favor of #4, putting the filesize at the end. (This one doesn't break anything.)\r\n\r\nBTW is there a reason that the header (json-filelist) gets pickled and not just written to the file? Is this faster or better in any way?\r\n\r\nI'm also aware, that these changes need to be reflected in atom-shell to have any meaning at all.\r\n\r\nIt could look like this:\r\n```\r\nMAGIC HEADERSIZE CHECKSUM HEADER FILES FILESIZE\r\n```\r\n\r\nThe position of HEADERSIZE and CHECKSUM could also be switched, I don't care."},"comment":{"url":"https://api.github.com/repos/atom/asar/issues/comments/71559956","html_url":"https://github.com/atom/asar/issues/16#issuecomment-71559956","issue_url":"https://api.github.com/repos/atom/asar/issues/16","id":71559956,"user":{"login":"bwin","id":1673361,"avatar_url":"https://avatars.githubusercontent.com/u/1673361?v=3","gravatar_id":"","url":"https://api.github.com/users/bwin","html_url":"https://github.com/bwin","followers_url":"https://api.github.com/users/bwin/followers","following_url":"https://api.github.com/users/bwin/following{/other_user}","gists_url":"https://api.github.com/users/bwin/gists{/gist_id}","starred_url":"https://api.github.com/users/bwin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bwin/subscriptions","organizations_url":"https://api.github.com/users/bwin/orgs","repos_url":"https://api.github.com/users/bwin/repos","events_url":"https://api.github.com/users/bwin/events{/privacy}","received_events_url":"https://api.github.com/users/bwin/received_events","type":"User","site_admin":false},"created_at":"2015-01-26T23:22:19Z","updated_at":"2015-01-26T23:22:19Z","body":"I changed my mind (after playing around with bwin/asar-util) about a few things.\r\n\r\n* I would put the header *behind* the payload. This makes changing an archive easier/possible. Also appending is simple (overwrite current header with new payload and new header).\r\nAlso this makes compressing files in one pass possible (because with the header in front, you need to know the compressed payload-file-size up front).\r\n* I would put a `version` field in the root of the json header. This is the file format version.\r\n* header-size should IMHO be an UINT32. I think json header > 4.2 GB is unrealistic. (otherwise UINT64)\r\n* max-file-size and max-archive-size is 1<<53 (max-header-size is 1<<32)\r\n* I'm not really sure about the checksum-part. You (Cheng) said, you want the checksum in json, but I don't like that. IMO generating this checksum is an uneccessary PITA because you have to null the checksum entry, to avoid including the checksum in generating the checksum. (I wrote that in a confusing way, admitted.)\r\nI don't really like my version either, because you're stuck with a fixed checksum size.\r\n\r\nHow to parse?\r\n- check magic\r\n- read headerSize and archiveSize at constant offset from end (depending on what happens with the checksum)\r\n- read header at constant offset from start (archiveSize - headerSize - sizeOfOtherFieldsAtEnd, avoids additional headerOffset value)\r\n- ...\r\n\r\nThis is what it would look like\r\n![asar-format](https://cloud.githubusercontent.com/assets/1673361/5910347/6e3d4060-a5b9-11e4-9c85-cad5f693c6ae.png)\r\n"}}