{"action":"created","issue":{"url":"https://api.github.com/repos/appc/docker2aci/issues/89","labels_url":"https://api.github.com/repos/appc/docker2aci/issues/89/labels{/name}","comments_url":"https://api.github.com/repos/appc/docker2aci/issues/89/comments","events_url":"https://api.github.com/repos/appc/docker2aci/issues/89/events","html_url":"https://github.com/appc/docker2aci/issues/89","id":106995341,"number":89,"title":"Docker images with PaxHeaders generate ACI images with unequal hashes","user":{"login":"blixtra","id":115075,"avatar_url":"https://avatars.githubusercontent.com/u/115075?v=3","gravatar_id":"","url":"https://api.github.com/users/blixtra","html_url":"https://github.com/blixtra","followers_url":"https://api.github.com/users/blixtra/followers","following_url":"https://api.github.com/users/blixtra/following{/other_user}","gists_url":"https://api.github.com/users/blixtra/gists{/gist_id}","starred_url":"https://api.github.com/users/blixtra/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/blixtra/subscriptions","organizations_url":"https://api.github.com/users/blixtra/orgs","repos_url":"https://api.github.com/users/blixtra/repos","events_url":"https://api.github.com/users/blixtra/events{/privacy}","received_events_url":"https://api.github.com/users/blixtra/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-09-17T14:18:29Z","updated_at":"2015-09-17T14:44:53Z","closed_at":null,"body":"When fetching the same Docker image one should get an ACI image with identical hashes. With docker://busybox for example this is the case.\r\n\r\n```\r\n~/W/s/rkt: sudo rkt --insecure-skip-verify fetch docker://busybox\r\nDownloading cf2616975b4a: [====================================] 32 B/32 B\r\nDownloading 6ce2e90b0bc7: [====================================] 1.15 MB/1.15 MB\r\nDownloading 8c2e06607696: [====================================] 32 B/32 B\r\nsha512-9d710100ce6769569b12a39100318bfe\r\n~/W/s/rkt: sudo rkt --insecure-skip-verify fetch docker://busybox\r\nDownloading cf2616975b4a: [====================================] 32 B/32 B\r\nDownloading 6ce2e90b0bc7: [====================================] 1.15 MB/1.15 MB\r\nDownloading 8c2e06607696: [====================================] 32 B/32 B\r\nsha512-9d710100ce6769569b12a39100318bfe\r\n```\r\n\r\nHowever, with the redis image this is sadly not the case.\r\n\r\n```\r\n~/W/s/rkt: sudo rkt --insecure-skip-verify fetch docker://redis\r\nDownloading ba249489d0b6: [====================================] 37.2 MB/37.2 MB\r\nDownloading 19de96c112fc: [====================================] 32 B/32 B\r\nDownloading d990a769a35e: [====================================] 1.7 KB/1.7 KB\r\nDownloading 8656a511ce9c: [====================================] 5.94 MB/5.94 MB\r\nDownloading f7022ac152fb: [====================================] 93.6 KB/93.6 KB\r\nDownloading 8e84d9ce7554: [====================================] 611 KB/611 KB \r\nDownloading c9e5dd2a9302: [====================================] 32 B/32 B\r\nDownloading 27b967cdd519: [====================================] 32 B/32 B\r\nDownloading 3024bf5093a1: [====================================] 32 B/32 B\r\nDownloading e6a9eb403efb: [====================================] 3.04 MB/3.04 MB\r\nDownloading c3532a4c89bc: [====================================] 98 B/98 B\r\nDownloading 35fc08946add: [====================================] 32 B/32 B\r\nDownloading d586de7d17cd: [====================================] 32 B/32 B\r\nDownloading 1f677d77a8fa: [====================================] 196 B/196 B\r\nDownloading ed09b32b8ab1: [====================================] 32 B/32 B\r\nDownloading 54647d88bc19: [====================================] 32 B/32 B\r\nDownloading 2f2578ff984f: [====================================] 32 B/32 B\r\nsha512-039afdb1311dc35a802a48352b6b29f2\r\n~/W/s/rkt: sudo rkt --insecure-skip-verify fetch docker://redis\r\nDownloading ba249489d0b6: [====================================] 37.2 MB/37.2 MB\r\nDownloading 19de96c112fc: [====================================] 32 B/32 B\r\nDownloading d990a769a35e: [====================================] 1.7 KB/1.7 KB\r\nDownloading 8656a511ce9c: [====================================] 5.94 MB/5.94 MB\r\nDownloading f7022ac152fb: [====================================] 93.6 KB/93.6 KB\r\nDownloading 8e84d9ce7554: [====================================] 611 KB/611 KB\r\nDownloading c9e5dd2a9302: [====================================] 32 B/32 B\r\nDownloading 27b967cdd519: [====================================] 32 B/32 B\r\nDownloading 3024bf5093a1: [====================================] 32 B/32 B\r\nDownloading e6a9eb403efb: [====================================] 3.04 MB/3.04 MB\r\nDownloading c3532a4c89bc: [====================================] 98 B/98 B\r\nDownloading 35fc08946add: [====================================] 32 B/32 B\r\nDownloading d586de7d17cd: [====================================] 32 B/32 B\r\nDownloading 1f677d77a8fa: [====================================] 196 B/196 B\r\nDownloading ed09b32b8ab1: [====================================] 32 B/32 B\r\nDownloading 54647d88bc19: [====================================] 32 B/32 B\r\nDownloading 2f2578ff984f: [====================================] 32 B/32 B\r\nsha512-4a60165ef194356a7a3070f5af0f1601\r\n```\r\n\r\nInspecting the images with `xxd` we can see that the difference is the PaxHeader.\r\n\r\n```\r\ndiff  -u <(xxd redis-latest.aci) <(xxd redis-latest1.aci)\r\n...\r\n@@ -1271809,13 +1271809,13 @@\r\n 01368000: 726f 6f74 6673 2f75 7372 2f73 6861 7265  rootfs/usr/share\r\n 01368010: 2f63 612d 6365 7274 6966 6963 6174 6573  /ca-certificates\r\n 01368020: 2f6d 6f7a 696c 6c61 2f50 6178 4865 6164  /mozilla/PaxHead\r\n-01368030: 6572 732e 3233 3837 302f 5442 5441 4b5f  ers.23870/TBTAK_\r\n+01368030: 6572 732e 3233 3831 302f 5442 5441 4b5f  ers.23810/TBTAK_\r\n 01368040: 5545 4b41 455f 4b6b 5f53 6572 7469 6669  UEKAE_Kk_Sertifi\r\n 01368050: 6b61 5f48 697a 6d65 745f 5361 6c61 7963  ka_Hizmet_Salayc\r\n 01368060: 735f 2d5f 3030 3030 3030 3000 3030 3030  s_-_0000000.0000\r\n 01368070: 3030 3000 3030 3030 3030 3000 3030 3030  000.0000000.0000\r\n 01368080: 3030 3030 3137 3000 3132 3431 3036 3034  0000170.12410604\r\n-01368090: 3530 3000 3033 3134 3630 0020 7800 0000  500.031460. x...\r\n+01368090: 3530 3000 3033 3134 3532 0020 7800 0000  500.031452. x...\r\n 013680a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................\r\n 013680b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................\r\n 013680c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................\r\n```\r\n\r\nThe result for the `rkt` user is that she sees lots of redis images listed when running `rkt image list`\r\n\r\n```\r\nKEY\t\t\t\t\t\t\t\t\tNAME\t\t\t\t\t\tIMPORTTIME\t\t\t\tLATEST\r\nsha512-b6fd68c419b8b42aed109ca0e18d31b3010d6daa784b71f94dc660e00d0496a5\tcoreos.com/rkt/stage1:0.8.0+gitb18cf28\t\t2015-09-10 16:03:33.805 +0200 CEST\tfalse\t\r\nsha512-b843248e28fa9132e23e1e763142049d17a61bab7873dff1e1ff105f9ddb2708\tredis:latest\t\t\t\t\t2015-09-17 13:27:04.24 +0200 CEST\ttrue\t\r\nsha512-44b881600b101eb45c7ea626a2bef9cac97e8ae2c194258fd06f5dd97bcf463c\tredis:latest\t\t\t\t\t2015-09-17 14:55:31.19 +0200 CEST\ttrue\t\r\nsha512-fb6b47cc9e7ee29f67422c6585c8f517c16e0e0ee9f4cf8b8cafd8e1c1d29233\tredis:latest\t\t\t\t\t2015-09-17 14:57:36.779 +0200 CEST\ttrue\t\r\nsha512-cbe0ddf1053e9c0141744b3929bbdde2938976c3dfee9a8976b05cf3cb22c365\tcoreos.com/rkt/stage1:0.8.0+git68b56a6-dirty\t2015-09-17 14:58:07.649 +0200 CEST\tfalse\t\r\nsha512-1d318506c1f328e2d58f29db710889001a631063d148973f86a784d316d3e91b\tredis:latest\t\t\t\t\t2015-09-17 14:59:34.478 +0200 CEST\ttrue\t\r\nsha512-91e98d7f1679a097c878203c9659f2a26ae394656b3147963324c61fa3832f15\tcoreos.com/etcd:v2.0.9\t\t\t\t2015-09-17 14:59:41.335 +0200 CEST\tfalse\t\r\nsha512-039afdb1311dc35a802a48352b6b29f2e120b804fd0362fd2866ba1cb6ab301e\tredis:latest\t\t\t\t\t2015-09-17 16:03:51.421 +0200 CEST\ttrue\t\r\nsha512-4a60165ef194356a7a3070f5af0f1601427e05c8fc65c5c4eec3b34cbd9ec662\tredis:latest\t\t\t\t\t2015-09-17 16:05:33.074 +0200 CEST\ttrue\t\r\nsha512-9d710100ce6769569b12a39100318bfed5b6b98115ee6315b724c11658db3751\tbusybox:latest\t\t\t\t\t2015-09-17 16:07:30.582 +0200 CEST\ttrue\r\n```"},"comment":{"url":"https://api.github.com/repos/appc/docker2aci/issues/comments/141109417","html_url":"https://github.com/appc/docker2aci/issues/89#issuecomment-141109417","issue_url":"https://api.github.com/repos/appc/docker2aci/issues/89","id":141109417,"user":{"login":"iaguis","id":507354,"avatar_url":"https://avatars.githubusercontent.com/u/507354?v=3","gravatar_id":"","url":"https://api.github.com/users/iaguis","html_url":"https://github.com/iaguis","followers_url":"https://api.github.com/users/iaguis/followers","following_url":"https://api.github.com/users/iaguis/following{/other_user}","gists_url":"https://api.github.com/users/iaguis/gists{/gist_id}","starred_url":"https://api.github.com/users/iaguis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/iaguis/subscriptions","organizations_url":"https://api.github.com/users/iaguis/orgs","repos_url":"https://api.github.com/users/iaguis/repos","events_url":"https://api.github.com/users/iaguis/events{/privacy}","received_events_url":"https://api.github.com/users/iaguis/received_events","type":"User","site_admin":false},"created_at":"2015-09-17T14:44:53Z","updated_at":"2015-09-17T14:44:53Z","body":"These are some files that change between different runs of docker2aci:\r\n\r\n```\r\nCertinomis_-_Autorité_Racine.pem\r\nEBG_Elektronik_Sertifika_Hizmet_Sağlayıcısı.pem\r\nAC_Raíz_Certicámara_S.A..pem\r\nrootfs/etc/ssl/certs/China_Internet_Network_Information_Center_EV_Certificates_Root.pem -> /usr/share/ca-certificates/mozilla/China_Internet_Network_Information_Center_EV_Certificates_Root.crt\r\n```\r\n\r\nIt seems too long link targets and non-ASCII characters make go's implementation of tar write Pax Headers:\r\n\r\nhttps://github.com/golang/go/blob/f271f928d9f1beb892dc576d397f37a3fddd6c0b/src/archive/tar/writer.go#L162"}}