{"action":"created","issue":{"url":"https://api.github.com/repos/docker/docker/issues/3672","labels_url":"https://api.github.com/repos/docker/docker/issues/3672/labels{/name}","comments_url":"https://api.github.com/repos/docker/docker/issues/3672/comments","events_url":"https://api.github.com/repos/docker/docker/issues/3672/events","html_url":"https://github.com/docker/docker/issues/3672","id":25909617,"number":3672,"title":"Remote ADD should use HTTP cache headers","user":{"login":"amuino","id":47512,"avatar_url":"https://avatars.githubusercontent.com/u/47512?v=3","gravatar_id":"","url":"https://api.github.com/users/amuino","html_url":"https://github.com/amuino","followers_url":"https://api.github.com/users/amuino/followers","following_url":"https://api.github.com/users/amuino/following{/other_user}","gists_url":"https://api.github.com/users/amuino/gists{/gist_id}","starred_url":"https://api.github.com/users/amuino/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amuino/subscriptions","organizations_url":"https://api.github.com/users/amuino/orgs","repos_url":"https://api.github.com/users/amuino/repos","events_url":"https://api.github.com/users/amuino/events{/privacy}","received_events_url":"https://api.github.com/users/amuino/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":11,"created_at":"2014-01-20T10:21:09Z","updated_at":"2015-01-21T20:08:07Z","closed_at":"2014-11-17T16:55:16Z","body":"The current ADD implementation for remote files seems (see comment https://github.com/dotcloud/docker/blob/4f53722dee6c8ea18a9c30bde2cc6fee24b03d32/buildfile.go#L376 ) to download the actual file and then calculate its checksum to determine if it is a cache hit or miss.\r\n\r\nIt should trust the http headers instead to build the checksum to save time and bandwidth (if the server responds that the file has not changed, there is no need to download it again).\r\n\r\nCurrently (client version 0.7.6), using a command to download and process the file is a workaround:\r\n\r\n    RUN apt-get install wget\r\n    RUN wget -O - http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.0.tar.gz | tar xz\r\n"},"comment":{"url":"https://api.github.com/repos/docker/docker/issues/comments/70911716","html_url":"https://github.com/docker/docker/issues/3672#issuecomment-70911716","issue_url":"https://api.github.com/repos/docker/docker/issues/3672","id":70911716,"user":{"login":"oncletom","id":138627,"avatar_url":"https://avatars.githubusercontent.com/u/138627?v=3","gravatar_id":"","url":"https://api.github.com/users/oncletom","html_url":"https://github.com/oncletom","followers_url":"https://api.github.com/users/oncletom/followers","following_url":"https://api.github.com/users/oncletom/following{/other_user}","gists_url":"https://api.github.com/users/oncletom/gists{/gist_id}","starred_url":"https://api.github.com/users/oncletom/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/oncletom/subscriptions","organizations_url":"https://api.github.com/users/oncletom/orgs","repos_url":"https://api.github.com/users/oncletom/repos","events_url":"https://api.github.com/users/oncletom/events{/privacy}","received_events_url":"https://api.github.com/users/oncletom/received_events","type":"User","site_admin":false},"created_at":"2015-01-21T20:08:07Z","updated_at":"2015-01-21T20:08:07Z","body":"> @duglin no. I'd rather have a slow builder than a broken one.\r\n\r\nWell, a use case we have at the BBC is to download pre-computed data models to build data-only volumes. To be honest, when you get 16GB from Amazon S3, you feel a massive pain, especially because of a typo in the last filename.\r\n\r\nAmazon S3 provides proper headers:\r\n\r\n```bash\r\ncurl --head https://s3-eu-west-1.amazonaws.com/.../dbpedia_index.db\r\nHTTP/1.1 200 OK\r\nx-amz-id-2: FfrZ6IOlKapHyfs2hcx9IAnMhBAwXlQxpc77fQ5f4cZ0xt06NpIARMANO/EU6zakJ/g0sRnDxvo=\r\nx-amz-request-id: 50132DAA1E90E2A7\r\nDate: Wed, 21 Jan 2015 20:06:23 GMT\r\nLast-Modified: Thu, 26 Jun 2014 09:17:04 GMT\r\nETag: \"20fe762a4bf2340164a65f73acb517c1-91\"\r\nAccept-Ranges: bytes\r\nContent-Type: application/octet-stream\r\nContent-Length: 6060023808\r\nServer: AmazonS3\r\n```\r\n\r\nIs there a way to take in account the headers when they exist, and to drop any kind of cache (actual behaviour) when you judge it is unsafe to rely on them?"}}