{"action":"created","issue":{"url":"https://api.github.com/repos/twitter/twemproxy/issues/382","labels_url":"https://api.github.com/repos/twitter/twemproxy/issues/382/labels{/name}","comments_url":"https://api.github.com/repos/twitter/twemproxy/issues/382/comments","events_url":"https://api.github.com/repos/twitter/twemproxy/issues/382/events","html_url":"https://github.com/twitter/twemproxy/issues/382","id":91902277,"number":382,"title":"Issue with socket permissions","user":{"login":"dbers","id":4643805,"avatar_url":"https://avatars.githubusercontent.com/u/4643805?v=3","gravatar_id":"","url":"https://api.github.com/users/dbers","html_url":"https://github.com/dbers","followers_url":"https://api.github.com/users/dbers/followers","following_url":"https://api.github.com/users/dbers/following{/other_user}","gists_url":"https://api.github.com/users/dbers/gists{/gist_id}","starred_url":"https://api.github.com/users/dbers/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dbers/subscriptions","organizations_url":"https://api.github.com/users/dbers/orgs","repos_url":"https://api.github.com/users/dbers/repos","events_url":"https://api.github.com/users/dbers/events{/privacy}","received_events_url":"https://api.github.com/users/dbers/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":11,"created_at":"2015-06-29T20:30:39Z","updated_at":"2015-07-01T04:51:50Z","closed_at":"2015-07-01T04:51:50Z","body":"We had a working twemproxy with about 9 containers configured.  I added a group of read/write clusters and a few of the permissions on the new and existing sockets got messed up.\r\n\r\nAfter going through all the logs and error reporting and trying to figure it out i just re-arranged the entries and everything started to work.  But i can not figure out why this would of made a difference.  Additional there are no errors on why the socket permissions are wrong.  We also checked character encoding, hidden characters, ext.\r\n\r\nWe are using version 0.4.0\r\nBelow are  the sockets (good and bad) and the configs (good and bad).  Any insight into why this is happening would be useful.  I have another system that is also having these issues but they have over 31 containers defined and simply re-arranging them isn't working.\r\n\r\nBad sockets:\r\n============\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:31 9500.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:31 9501.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:31 9502.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:31 9503.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:31 9504.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:31 9505.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:31 9506.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:31 9507.sock\r\nsr-S--Sr-T 1 apache apache 0 Jun 29 15:31 9508.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:31 9510.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:31 9511.sock\r\nsr-S--Sr-T 1 apache apache 0 Jun 29 15:31 9512.sock\r\nsr-x--x-wx 1 apache apache 0 Jun 29 15:31 9513.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:31 9514.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:31 9515.sock\r\n\r\nGood sockets:\r\n=============\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:38 9500.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:38 9501.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:38 9502.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:38 9503.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:38 9504.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:38 9505.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:38 9506.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:38 9507.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:38 9508.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:38 9510.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:38 9511.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:38 9512.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:38 9513.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:38 9514.sock\r\nsrwxrwxrwx 1 apache apache 0 Jun 29 15:38 9515.sock\r\n\r\n\r\nBad config\r\n==========\r\n\r\n\r\ngroup1:\r\n  listen:  /mnt/9500.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:16855:1 REDIS_HOST_USERS\r\n\r\ngroup2:\r\n  listen: /mnt/9501.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:12786:1 REDIS_HOST_BROADCASTS\r\n\r\ngroup3:\r\n  listen: /mnt/9502.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:19473:1 REDIS_HOST_ONLINE_USERS\r\n\r\ngroup4:\r\n  listen: /mnt/9503.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:17674:1 REDIS_HOST_SESSIONS\r\n\r\ngroup5:\r\n  listen: /mnt/9504.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:19861:1 REDIS_HOST_TAGS\r\n\r\ngroup6:\r\n  listen: /mnt/9505.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:12639:1 REDIS_HOST_CHAT\r\n\r\ngroup7:\r\n  listen: /mnt/9506.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:12639:1 REDIS_HOST_TRENDING\r\n\r\ngroup8:\r\n  listen: /mnt/9507.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:11749:1 REDIS_NOTIFICATIONS\r\n\r\ngroup9:\r\n  listen: /mnt/9508.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.cache.amazonaws.com:6379:1 REDIS_HOST_CHAT_LOGS\r\n\r\ngroup10:\r\n  listen: /mnt/9510.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 1\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:18754:1 REDIS_HOST_ONLINE_USERS2\r\n\r\ngroup11:\r\n  listen: /mnt/9511.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  servers:\r\n   - ************************************.garantiadata.com:19473:1 REDIS_HOST_ONLINE_USERS\r\n\r\ngroup12:\r\n  listen: /mnt/9512.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  server_retry_timeout: 5000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.cache.amazonaws.com:6379:1 REDIS_WRITE_CLUSTER_1\r\n\r\ngroup13:\r\n  listen: /mnt/9513.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  server_retry_timeout: 5000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.cache.amazonaws.com:6379:1 REDIS_WRITE_CLUSTER_2\r\n\r\ngroup14:\r\n  listen: /mnt/9514.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: true\r\n  server_failure_limit: 2\r\n  server_connections: 40\r\n  redis: true\r\n  server_retry_timeout: 5000\r\n  timeout: 1500\r\n  servers:\r\n   - ************************************.cache.amazonaws.com:6379:1 REDIS_READ_CLUSTER_1_N1\r\n   - ************************************.cache.amazonaws.com:6379:1 REDIS_READ_CLUSTER_1_N2\r\n\r\ngroup15:\r\n  listen: /mnt/9515.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: true\r\n  server_failure_limit: 2\r\n  server_connections: 40\r\n  redis: true\r\n  server_retry_timeout: 5000\r\n  timeout: 1500\r\n  servers:\r\n   - ************************************.cache.amazonaws.com:6379:1 REDIS_READ_CLUSTER_2_N1\r\n   - ************************************.cache.amazonaws.com:6379:1 REDIS_READ_CLUSTER_2_N2\r\n\r\n\r\n\r\n\r\nGood config\r\n============\r\n\r\ngroup1:\r\n  listen:  /mnt/9500.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:16855:1 REDIS_HOST_USERS\r\n\r\ngroup2:\r\n  listen: /mnt/9501.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:12786:1 REDIS_HOST_BROADCASTS\r\n\r\ngroup3:\r\n  listen: /mnt/9502.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:19473:1 REDIS_HOST_ONLINE_USERS\r\n\r\ngroup4:\r\n  listen: /mnt/9503.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:17674:1 REDIS_HOST_SESSIONS\r\n\r\ngroup5:\r\n  listen: /mnt/9504.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:19861:1 REDIS_HOST_TAGS\r\n\r\ngroup6:\r\n  listen: /mnt/9505.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:12639:1 REDIS_HOST_CHAT\r\n\r\ngroup7:\r\n  listen: /mnt/9506.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:12639:1 REDIS_HOST_TRENDING\r\n\r\ngroup9:\r\n  listen: /mnt/9508.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.cache.amazonaws.com:6379:1 REDIS_HOST_CHAT_LOGS\r\n\r\ngroup8:\r\n  listen: /mnt/9507.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:11749:1 REDIS_NOTIFICATIONS\r\n\r\ngroup10:\r\n  listen: /mnt/9510.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 1\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.garantiadata.com:18754:1 REDIS_HOST_ONLINE_USERS2\r\n\r\ngroup11:\r\n  listen: /mnt/9511.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  redis_auth: *******\r\n  server_retry_timeout: 2000\r\n  servers:\r\n   - ************************************.garantiadata.com:19473:1 REDIS_HOST_ONLINE_USERS\r\n\r\ngroup13:\r\n  listen: /mnt/9513.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  server_retry_timeout: 5000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.cache.amazonaws.com:6379:1 REDIS_WRITE_CLUSTER_2\r\n\r\ngroup14:\r\n  listen: /mnt/9514.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: true\r\n  server_failure_limit: 2\r\n  server_connections: 40\r\n  redis: true\r\n  server_retry_timeout: 5000\r\n  timeout: 1500\r\n  servers:\r\n   - ************************************.cache.amazonaws.com:6379:1 REDIS_READ_CLUSTER_1_N2\r\n\r\ngroup15:\r\n  listen: /mnt/9515.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: true\r\n  server_failure_limit: 2\r\n  server_connections: 40\r\n  redis: true\r\n  server_retry_timeout: 5000\r\n  timeout: 1500\r\n  servers:\r\n   - ************************************.cache.amazonaws.com:6379:1 REDIS_READ_CLUSTER_2_N2\r\n\r\ngroup12:\r\n  listen: /mnt/9512.sock\r\n  hash: fnv1a_64\r\n  distribution: ketama\r\n  auto_eject_hosts: false\r\n  server_connections: 10\r\n  redis: true\r\n  server_retry_timeout: 5000\r\n  timeout: 500\r\n  servers:\r\n   - ************************************.cache.amazonaws.com:6379:1 REDIS_WRITE_CLUSTER_1\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/twitter/twemproxy/issues/comments/117433412","html_url":"https://github.com/twitter/twemproxy/issues/382#issuecomment-117433412","issue_url":"https://api.github.com/repos/twitter/twemproxy/issues/382","id":117433412,"user":{"login":"dbers","id":4643805,"avatar_url":"https://avatars.githubusercontent.com/u/4643805?v=3","gravatar_id":"","url":"https://api.github.com/users/dbers","html_url":"https://github.com/dbers","followers_url":"https://api.github.com/users/dbers/followers","following_url":"https://api.github.com/users/dbers/following{/other_user}","gists_url":"https://api.github.com/users/dbers/gists{/gist_id}","starred_url":"https://api.github.com/users/dbers/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dbers/subscriptions","organizations_url":"https://api.github.com/users/dbers/orgs","repos_url":"https://api.github.com/users/dbers/repos","events_url":"https://api.github.com/users/dbers/events{/privacy}","received_events_url":"https://api.github.com/users/dbers/received_events","type":"User","site_admin":false},"created_at":"2015-07-01T04:51:50Z","updated_at":"2015-07-01T04:51:50Z","body":"Yes, So far using the permissions directive is working correctly.  \r\n\r\nThanks\r\nDan"}}