{"action":"created","issue":{"url":"https://api.github.com/repos/choonkeat/attache/issues/3","labels_url":"https://api.github.com/repos/choonkeat/attache/issues/3/labels{/name}","comments_url":"https://api.github.com/repos/choonkeat/attache/issues/3/comments","events_url":"https://api.github.com/repos/choonkeat/attache/issues/3/events","html_url":"https://github.com/choonkeat/attache/issues/3","id":108277739,"number":3,"title":"When images did not get backup to config.storage","user":{"login":"choonkeat","id":473,"avatar_url":"https://avatars.githubusercontent.com/u/473?v=3","gravatar_id":"","url":"https://api.github.com/users/choonkeat","html_url":"https://github.com/choonkeat","followers_url":"https://api.github.com/users/choonkeat/followers","following_url":"https://api.github.com/users/choonkeat/following{/other_user}","gists_url":"https://api.github.com/users/choonkeat/gists{/gist_id}","starred_url":"https://api.github.com/users/choonkeat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/choonkeat/subscriptions","organizations_url":"https://api.github.com/users/choonkeat/orgs","repos_url":"https://api.github.com/users/choonkeat/repos","events_url":"https://api.github.com/users/choonkeat/events{/privacy}","received_events_url":"https://api.github.com/users/choonkeat/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-09-25T07:35:54Z","updated_at":"2015-09-25T11:10:35Z","closed_at":null,"body":"Upload was deem slow when uploads were backed up to cloud (e.g. s3) synchronously: file transfer from browser to server then server to s3. This would result in a timeout error for uploads, esp when hosted on heroku.\r\n\r\nThus the logic was https://github.com/choonkeat/attache/blob/master/lib/attache/upload.rb#L23-L28 \r\n- first, the file is written to local cache (and would fail hard if that didn't happen)\r\n- then the browser would get a thumbsup and everything proceed\r\n- background job will upload that file to cloud\r\n\r\nSo now, if the background job fail for some reason, after retries, what is the recourse?"},"comment":{"url":"https://api.github.com/repos/choonkeat/attache/issues/comments/143188845","html_url":"https://github.com/choonkeat/attache/issues/3#issuecomment-143188845","issue_url":"https://api.github.com/repos/choonkeat/attache/issues/3","id":143188845,"user":{"login":"choonkeat","id":473,"avatar_url":"https://avatars.githubusercontent.com/u/473?v=3","gravatar_id":"","url":"https://api.github.com/users/choonkeat","html_url":"https://github.com/choonkeat","followers_url":"https://api.github.com/users/choonkeat/followers","following_url":"https://api.github.com/users/choonkeat/following{/other_user}","gists_url":"https://api.github.com/users/choonkeat/gists{/gist_id}","starred_url":"https://api.github.com/users/choonkeat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/choonkeat/subscriptions","organizations_url":"https://api.github.com/users/choonkeat/orgs","repos_url":"https://api.github.com/users/choonkeat/repos","events_url":"https://api.github.com/users/choonkeat/events{/privacy}","received_events_url":"https://api.github.com/users/choonkeat/received_events","type":"User","site_admin":false},"created_at":"2015-09-25T11:10:35Z","updated_at":"2015-09-25T11:10:35Z","body":"> Also.. then does https://github.com/choonkeat/attache/blob/master/lib/attache/job.rb#L7-L9 overlap or override the default retry mechanism in sidekiq?\r\n\r\nYes it does, and that's what I'm trying to say: we're not leveraging the framework retry mechanism. This is partially because we fallback to using `sucker_punch` which does not have built-in retry mechanism. But mostly it is because we want to always requeue a failed job.\r\n\r\n> Do we have a web interface that we can verify if that job is in the \"Dead Job Queue\"?\r\n\r\nNo.\r\n\r\nI'm thinking since the supposedly robust always-retry-on-error didn't pan out, I'm going to\r\n- treat all uploads as error by default: dump the key value somewhere on disk permanent (no LRU purge) before uploading to cloud\r\n- clear the error when cloud upload is successful\r\n"}}