{"action":"created","issue":{"url":"https://api.github.com/repos/audreyr/binaryornot/issues/10","labels_url":"https://api.github.com/repos/audreyr/binaryornot/issues/10/labels{/name}","comments_url":"https://api.github.com/repos/audreyr/binaryornot/issues/10/comments","events_url":"https://api.github.com/repos/audreyr/binaryornot/issues/10/events","html_url":"https://github.com/audreyr/binaryornot/issues/10","id":100720853,"number":10,"title":"Binary pixel stream is detected as text","user":{"login":"pombredanne","id":675997,"avatar_url":"https://avatars.githubusercontent.com/u/675997?v=3","gravatar_id":"","url":"https://api.github.com/users/pombredanne","html_url":"https://github.com/pombredanne","followers_url":"https://api.github.com/users/pombredanne/followers","following_url":"https://api.github.com/users/pombredanne/following{/other_user}","gists_url":"https://api.github.com/users/pombredanne/gists{/gist_id}","starred_url":"https://api.github.com/users/pombredanne/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pombredanne/subscriptions","organizations_url":"https://api.github.com/users/pombredanne/orgs","repos_url":"https://api.github.com/users/pombredanne/repos","events_url":"https://api.github.com/users/pombredanne/events{/privacy}","received_events_url":"https://api.github.com/users/pombredanne/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":11,"created_at":"2015-08-13T09:09:49Z","updated_at":"2015-08-14T06:58:46Z","closed_at":null,"body":"This is originally from https://github.com/nexB/scancode-toolkit/issues/50\r\nIn this archive https://rubygems.org/downloads/chunky_png-1.2.8.gem the file:\r\nchunky_png-1.2.8.gem-extract/data.tar.gz-extract/spec/resources/pixelstream.rgb\r\nis detected as text even though this is clearly a binary data stream.\r\n\r\nlibmagic file detects it as octet/stream or data which is correct (for instance with this libmagic ctypes binding: https://github.com/nexB/scancode-toolkit/blob/master/src/typecode/magic2.py )\r\n\r\nThe issue comes from the fact that bytes above 127 are treated as text, and the test file first 1024 bytes are FF. The original Perl binary detection looked for bytes below 127 only and did some extra stuffs. \r\n"},"comment":{"url":"https://api.github.com/repos/audreyr/binaryornot/issues/comments/130997716","html_url":"https://github.com/audreyr/binaryornot/issues/10#issuecomment-130997716","issue_url":"https://api.github.com/repos/audreyr/binaryornot/issues/10","id":130997716,"user":{"login":"audreyr","id":74739,"avatar_url":"https://avatars.githubusercontent.com/u/74739?v=3","gravatar_id":"","url":"https://api.github.com/users/audreyr","html_url":"https://github.com/audreyr","followers_url":"https://api.github.com/users/audreyr/followers","following_url":"https://api.github.com/users/audreyr/following{/other_user}","gists_url":"https://api.github.com/users/audreyr/gists{/gist_id}","starred_url":"https://api.github.com/users/audreyr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/audreyr/subscriptions","organizations_url":"https://api.github.com/users/audreyr/orgs","repos_url":"https://api.github.com/users/audreyr/repos","events_url":"https://api.github.com/users/audreyr/events{/privacy}","received_events_url":"https://api.github.com/users/audreyr/received_events","type":"User","site_admin":false},"created_at":"2015-08-14T06:58:46Z","updated_at":"2015-08-14T06:58:46Z","body":"No issue, those are very nice tests. The more test files, the merrier. And isBinaryFile looks great, something I'll be studying for ideas :smile:"}}