{"action":"closed","issue":{"url":"https://api.github.com/repos/laurilehmijoki/s3_website/issues/156","labels_url":"https://api.github.com/repos/laurilehmijoki/s3_website/issues/156/labels{/name}","comments_url":"https://api.github.com/repos/laurilehmijoki/s3_website/issues/156/comments","events_url":"https://api.github.com/repos/laurilehmijoki/s3_website/issues/156/events","html_url":"https://github.com/laurilehmijoki/s3_website/issues/156","id":67416161,"number":156,"title":"Fails to upload changed file when multiple S3 keys have the same md5sum","user":{"login":"mgreensmith","id":1601951,"avatar_url":"https://avatars.githubusercontent.com/u/1601951?v=3","gravatar_id":"","url":"https://api.github.com/users/mgreensmith","html_url":"https://github.com/mgreensmith","followers_url":"https://api.github.com/users/mgreensmith/followers","following_url":"https://api.github.com/users/mgreensmith/following{/other_user}","gists_url":"https://api.github.com/users/mgreensmith/gists{/gist_id}","starred_url":"https://api.github.com/users/mgreensmith/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mgreensmith/subscriptions","organizations_url":"https://api.github.com/users/mgreensmith/orgs","repos_url":"https://api.github.com/users/mgreensmith/repos","events_url":"https://api.github.com/users/mgreensmith/events{/privacy}","received_events_url":"https://api.github.com/users/mgreensmith/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-04-09T17:52:21Z","updated_at":"2015-04-10T19:06:08Z","closed_at":"2015-04-10T19:06:08Z","body":"If a file is changed, but a different file key already exists on S3 with the same md5sum, the file is not recognized as changed, and is not uploaded.\r\n\r\nIt appears that the UploadHelper just takes the set of S3 md5sums and assumes that if the md5sum of the current file is found at all, then the file is up to date. However, that md5sum could belong to a different file with the same content.(https://github.com/laurilehmijoki/s3_website/blob/master/src/main/scala/s3/website/UploadHelper.scala#L30)\r\n\r\nExample:\r\n\r\n```\r\n~$ find ./dupetest -type f -print0 | xargs -0 md5\r\nMD5 (./dupetest/1.txt) = c157a79031e1c40f85931829bc5fc552   # file contents: 'bar'\r\nMD5 (./dupetest/2.txt) = d3b07384d113edec49eaa6238ad5ff00  # file contents: 'foo'\r\nMD5 (./dupetest/3.txt) = c157a79031e1c40f85931829bc5fc552   # file contents: 'bar'\r\n\r\n~$ bundle exec s3_website push --site dupetest --verbose\r\n[debg] Using /usr/local/var/rbenv/versions/2.0.0-p451/gemsets/marketing-jekyll/gems/s3_website-2.8.4/s3_website-2.8.4.jar\r\n[info] Deploying dupetest/* to mg-testbucket\r\n[debg] Querying S3 files\r\n[succ] Created 3.txt (text/plain; charset=utf-8 | 4 B | 0 B/s)\r\n[succ] Created 1.txt (text/plain; charset=utf-8 | 4 B | 0 B/s)\r\n[succ] Created 2.txt (text/plain; charset=utf-8 | 4 B | 0 B/s)\r\n[info] Summary: Created 3 files. Transferred 12 B, 0 B/s.\r\n[info] Successfully pushed the website to http://mg-testbucket.s3-website-us-east-1.amazonaws.com\r\n\r\n~$ echo \"foo\" > dupetest/3.txt\r\n\r\n~$ find ./dupetest -type f -print0 | xargs -0 md5\r\nMD5 (./dupetest/1.txt) = c157a79031e1c40f85931829bc5fc552   # file contents: 'bar'\r\nMD5 (./dupetest/2.txt) = d3b07384d113edec49eaa6238ad5ff00  # file contents: 'foo'\r\nMD5 (./dupetest/3.txt) = d3b07384d113edec49eaa6238ad5ff00  # file contents: 'foo'\r\n\r\n~$ bundle exec s3_website push --site dupetest --verbose\r\n[debg] Using /usr/local/var/rbenv/versions/2.0.0-p451/gemsets/marketing-jekyll/gems/s3_website-2.8.4/s3_website-2.8.4.jar\r\n[info] Deploying dupetest/* to mg-testbucket\r\n[debg] Querying S3 files\r\n[info] Summary: There was nothing to push\r\n```\r\n\r\nExpected: `3.txt` to be uploaded.\r\nActual: `3.txt` was not uploaded, because md5sum of the file was already present via `2.txt`\r\n\r\nPossible solution: the `UploadHelper` could map the existing S3 file key names to their md5sums and compare md5sums for the file in question only, rather than searching all md5sums.\r\n"}}