{"action":"created","issue":{"url":"https://api.github.com/repos/rcbops/rpc-openstack/issues/609","labels_url":"https://api.github.com/repos/rcbops/rpc-openstack/issues/609/labels{/name}","comments_url":"https://api.github.com/repos/rcbops/rpc-openstack/issues/609/comments","events_url":"https://api.github.com/repos/rcbops/rpc-openstack/issues/609/events","html_url":"https://github.com/rcbops/rpc-openstack/issues/609","id":117819737,"number":609,"title":"rabbitmq permissions issue present in r11.1.0rc4","user":{"login":"mancdaz","id":562262,"avatar_url":"https://avatars.githubusercontent.com/u/562262?v=3","gravatar_id":"","url":"https://api.github.com/users/mancdaz","html_url":"https://github.com/mancdaz","followers_url":"https://api.github.com/users/mancdaz/followers","following_url":"https://api.github.com/users/mancdaz/following{/other_user}","gists_url":"https://api.github.com/users/mancdaz/gists{/gist_id}","starred_url":"https://api.github.com/users/mancdaz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mancdaz/subscriptions","organizations_url":"https://api.github.com/users/mancdaz/orgs","repos_url":"https://api.github.com/users/mancdaz/repos","events_url":"https://api.github.com/users/mancdaz/events{/privacy}","received_events_url":"https://api.github.com/users/mancdaz/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-11-19T14:04:03Z","updated_at":"2015-11-19T15:43:43Z","closed_at":null,"body":"```\r\nTASK: [rabbitmq_server | Enable queue mirroring] ******************************\r\nfailed: [rc4-node3_rabbit_mq_container-5ad8cc6a] => {\"cmd\": \"/usr/sbin/rabbitmqctl -q -n rabbit list_policies -p /\", \"failed\": true, \"rc\": 2}\r\nstderr: Error: unable to connect to node 'rabbit@rc4-node3_rabbit_mq_container-5ad8cc6a': nodedown\r\n\r\nDIAGNOSTICS\r\n===========\r\n\r\nattempted to contact: ['rabbit@rc4-node3_rabbit_mq_container-5ad8cc6a']\r\n\r\nrabbit@rc4-node3_rabbit_mq_container-5ad8cc6a:\r\n  * connected to epmd (port 4369) on rc4-node3_rabbit_mq_container-5ad8cc6a\r\n  * epmd reports: node 'rabbit' not running at all\r\n                  no other nodes on rc4-node3_rabbit_mq_container-5ad8cc6a\r\n  * suggestion: start the node\r\n\r\ncurrent node details:\r\n- node name: 'rabbitmq-cli-3157@rc4-node3_rabbit_mq_container-5ad8cc6a'\r\n- home dir: /var/lib/rabbitmq\r\n- cookie hash: ZXlCDDnOfAskKwiRflzNkw==\r\n\r\n\r\nmsg: Error:********@rc4-node3_rabbit_mq_container-5ad8cc6a'\r\n- home dir: /var/lib/rabbitmq\r\n- cookie hash: ZXlCDDnOfAskKwiRflzNkw==\r\nfailed: [rc4-node2_rabbit_mq_container-2701c6f1] => {\"cmd\": \"/usr/sbin/rabbitmqctl -q -n rabbit list_policies -p /\", \"failed\": true, \"rc\": 2}\r\nstderr: Error: unable to connect to node 'rabbit@rc4-node2_rabbit_mq_container-2701c6f1': nodedown\r\n\r\nDIAGNOSTICS\r\n===========\r\n\r\nattempted to contact: ['rabbit@rc4-node2_rabbit_mq_container-2701c6f1']\r\n\r\nrabbit@rc4-node2_rabbit_mq_container-2701c6f1:\r\n  * connected to epmd (port 4369) on rc4-node2_rabbit_mq_container-2701c6f1\r\n  * epmd reports: node 'rabbit' not running at all\r\n                  no other nodes on rc4-node2_rabbit_mq_container-2701c6f1\r\n  * suggestion: start the node\r\n\r\ncurrent node details:\r\n- node name: 'rabbitmq-cli-3157@rc4-node2_rabbit_mq_container-2701c6f1'\r\n- home dir: /var/lib/rabbitmq\r\n- cookie hash: ZXlCDDnOfAskKwiRflzNkw==\r\n\r\n\r\nmsg: Error:********@rc4-node2_rabbit_mq_container-2701c6f1'\r\n- home dir: /var/lib/rabbitmq\r\n- cookie hash: ZXlCDDnOfAskKwiRflzNkw==\r\nchanged: [rc4-node1_rabbit_mq_container-5f865e07]\r\n\r\nFATAL: all hosts have already failed -- aborting\r\n\r\nPLAY RECAP ********************************************************************\r\n           to retry, use: --limit @/root/setup-infrastructure.retry\r\n```\r\n\r\nThis review in openstack-ansible fixed the problem:\r\n\r\nhttps://review.openstack.org/#/c/243227/\r\n\r\nHowever, rc4 does not include this since the last osa commit is from 20th October.  I understand that we need to not move the goalposts while this is being tested, but currently openstack-ansible is undeployable.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/rcbops/rpc-openstack/issues/comments/158094932","html_url":"https://github.com/rcbops/rpc-openstack/issues/609#issuecomment-158094932","issue_url":"https://api.github.com/repos/rcbops/rpc-openstack/issues/609","id":158094932,"user":{"login":"jacobwagner","id":2229998,"avatar_url":"https://avatars.githubusercontent.com/u/2229998?v=3","gravatar_id":"","url":"https://api.github.com/users/jacobwagner","html_url":"https://github.com/jacobwagner","followers_url":"https://api.github.com/users/jacobwagner/followers","following_url":"https://api.github.com/users/jacobwagner/following{/other_user}","gists_url":"https://api.github.com/users/jacobwagner/gists{/gist_id}","starred_url":"https://api.github.com/users/jacobwagner/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jacobwagner/subscriptions","organizations_url":"https://api.github.com/users/jacobwagner/orgs","repos_url":"https://api.github.com/users/jacobwagner/repos","events_url":"https://api.github.com/users/jacobwagner/events{/privacy}","received_events_url":"https://api.github.com/users/jacobwagner/received_events","type":"User","site_admin":false},"created_at":"2015-11-19T15:43:43Z","updated_at":"2015-11-19T15:43:43Z","body":"I was seeing this in rc3, but a rerun of the rabbitmq plays fixed it so i didnt investigate further, i thought it was a non issue but we will look at it again. If it is a known issue with a fix i would suggest pulling it into a rc5."}}