{"action":"created","issue":{"url":"https://api.github.com/repos/deis/deis/issues/4778","labels_url":"https://api.github.com/repos/deis/deis/issues/4778/labels{/name}","comments_url":"https://api.github.com/repos/deis/deis/issues/4778/comments","events_url":"https://api.github.com/repos/deis/deis/issues/4778/events","html_url":"https://github.com/deis/deis/issues/4778","id":120452036,"number":4778,"title":"Intermittent router 404's","user":{"login":"sstarcher","id":3778974,"avatar_url":"https://avatars.githubusercontent.com/u/3778974?v=3","gravatar_id":"","url":"https://api.github.com/users/sstarcher","html_url":"https://github.com/sstarcher","followers_url":"https://api.github.com/users/sstarcher/followers","following_url":"https://api.github.com/users/sstarcher/following{/other_user}","gists_url":"https://api.github.com/users/sstarcher/gists{/gist_id}","starred_url":"https://api.github.com/users/sstarcher/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sstarcher/subscriptions","organizations_url":"https://api.github.com/users/sstarcher/orgs","repos_url":"https://api.github.com/users/sstarcher/repos","events_url":"https://api.github.com/users/sstarcher/events{/privacy}","received_events_url":"https://api.github.com/users/sstarcher/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-12-04T17:49:35Z","updated_at":"2015-12-04T18:09:25Z","closed_at":null,"body":"Setup: AWS, Stateless, Deis 1.12.1, ELB SSL Termination, EnforceHTTPS, EnforceWhitelist\r\n\r\nWe have health checks on all of our Deis apps that hit the ELB->Router->App for health checking.\r\nWe have noticed for a while intermittent 404's coming back through these health checks.  For a while I assumed the apps themselves were randomly having issues.  The last day we have started seeing these failures more often.  I set a health check to directly test against the container running the app and the last few occurences of the router health check have intermittently failed, but health check going directly against the container and by passing the router is working find.\r\n\r\nI just separately tested each router and 1 of the 3 routers is intermittently failing with 404s.\r\nI take that back a second router started getting 404's 50% of the time.\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/deis/deis/issues/comments/162037894","html_url":"https://github.com/deis/deis/issues/4778#issuecomment-162037894","issue_url":"https://api.github.com/repos/deis/deis/issues/4778","id":162037894,"user":{"login":"sstarcher","id":3778974,"avatar_url":"https://avatars.githubusercontent.com/u/3778974?v=3","gravatar_id":"","url":"https://api.github.com/users/sstarcher","html_url":"https://github.com/sstarcher","followers_url":"https://api.github.com/users/sstarcher/followers","following_url":"https://api.github.com/users/sstarcher/following{/other_user}","gists_url":"https://api.github.com/users/sstarcher/gists{/gist_id}","starred_url":"https://api.github.com/users/sstarcher/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sstarcher/subscriptions","organizations_url":"https://api.github.com/users/sstarcher/orgs","repos_url":"https://api.github.com/users/sstarcher/repos","events_url":"https://api.github.com/users/sstarcher/events{/privacy}","received_events_url":"https://api.github.com/users/sstarcher/received_events","type":"User","site_admin":false},"created_at":"2015-12-04T18:09:25Z","updated_at":"2015-12-04T18:09:25Z","body":"I previously had 3 routers running.  I restarted each router individually that did not seem to fix the problem.  I scaled the routers up to 5.  The below screen shot is a simple curl hitting each router directly.\r\n\r\n$1 = ip of the router\r\n```curl -s -I -H 'Host: hound.cluster.private' -H 'X-Forwarded-Proto: https' $1 | head -n 1```\r\n\r\n<img width=\"2310\" alt=\"screen shot 2015-12-04 at 1 07 02 pm\" src=\"https://cloud.githubusercontent.com/assets/3778974/11597184/0b748c44-9a88-11e5-8a13-078b8523a371.png\">"}}