{"action":"created","issue":{"url":"https://api.github.com/repos/bloomberg/chef-bach/issues/345","labels_url":"https://api.github.com/repos/bloomberg/chef-bach/issues/345/labels{/name}","comments_url":"https://api.github.com/repos/bloomberg/chef-bach/issues/345/comments","events_url":"https://api.github.com/repos/bloomberg/chef-bach/issues/345/events","html_url":"https://github.com/bloomberg/chef-bach/issues/345","id":120515015,"number":345,"title":"cluster-assign-roles.sh fails to Chef Vault os/ssh","user":{"login":"cbaenziger","id":4192761,"avatar_url":"https://avatars.githubusercontent.com/u/4192761?v=3","gravatar_id":"","url":"https://api.github.com/users/cbaenziger","html_url":"https://github.com/cbaenziger","followers_url":"https://api.github.com/users/cbaenziger/followers","following_url":"https://api.github.com/users/cbaenziger/following{/other_user}","gists_url":"https://api.github.com/users/cbaenziger/gists{/gist_id}","starred_url":"https://api.github.com/users/cbaenziger/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cbaenziger/subscriptions","organizations_url":"https://api.github.com/users/cbaenziger/orgs","repos_url":"https://api.github.com/users/cbaenziger/repos","events_url":"https://api.github.com/users/cbaenziger/events{/privacy}","received_events_url":"https://api.github.com/users/cbaenziger/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/bloomberg/chef-bach/labels/Chef","name":"Chef","color":"fbca04"}],"state":"open","locked":false,"assignee":{"login":"http-418","id":1435487,"avatar_url":"https://avatars.githubusercontent.com/u/1435487?v=3","gravatar_id":"","url":"https://api.github.com/users/http-418","html_url":"https://github.com/http-418","followers_url":"https://api.github.com/users/http-418/followers","following_url":"https://api.github.com/users/http-418/following{/other_user}","gists_url":"https://api.github.com/users/http-418/gists{/gist_id}","starred_url":"https://api.github.com/users/http-418/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/http-418/subscriptions","organizations_url":"https://api.github.com/users/http-418/orgs","repos_url":"https://api.github.com/users/http-418/repos","events_url":"https://api.github.com/users/http-418/events{/privacy}","received_events_url":"https://api.github.com/users/http-418/received_events","type":"User","site_admin":false},"milestone":null,"comments":6,"created_at":"2015-12-05T00:41:29Z","updated_at":"2015-12-10T16:56:43Z","closed_at":null,"body":"On an established cluster, for some reason I am hitting the following. This is when running `./cluster-assign-roles.sh CLUSTER hadoop`. Per the `cluster-assign-roles.sh` changes in Chef-BACH #212 and Chef-BACH #245 it was believed that nodes would have keys updated for their visibility as needed; it seems that is not happening?\r\n\r\nFor reference, the bootstrap node can see the vault item fine:\r\n````\r\nbootstrap:~/chef-bcpc$ sudo knife vault show os ssh -m client\r\nid:          ssh\r\nprivate-key: <key here>\r\n````\r\n\r\nWhile the node being Chef'd can not:\r\n````\r\n  * ruby_block[chef_vault_secret] action create\r\n    - execute the ruby block chef_vault_secret\r\n\r\n  * chef_vault_secret[ssh] action create_if_missing  \r\n================================================================================\r\nError executing action `create_if_missing` on resource 'chef_vault_secret[ssh]'\r\n================================================================================\r\n\r\n\r\nChefVault::Exceptions::SecretDecryption\r\n---------------------------------------\r\nos/ssh is not encrypted with your public key.  Contact an administrator of the vault item to encrypt for you!\r\n\r\n\r\nCookbook Trace:\r\n---------------\r\n/var/chef/cache/cookbooks/chef-vault/libraries/chef_vault_secret_provider.rb:67:in `load_current_resource'\r\n\r\n\r\nResource Declaration:\r\n---------------------\r\n# In /var/chef/cache/cookbooks/bcpc/recipes/certs.rb\r\n\r\n 54: chef_vault_secret \"ssh\" do\r\n 55:   data_bag 'os'\r\n 56:   raw_data({ \"private-key\" => key.to_pem })\r\n 57:   admins \"#{ nodes },#{ bootstrap }\"\r\n 58:   search '*:*'\r\n 59:   action :create_if_missing\r\n 60: end\r\n 61:\r\n\r\n\r\n\r\nCompiled Resource:\r\n------------------\r\n# Declared in /var/chef/cache/cookbooks/bcpc/recipes/certs.rb:54:in `from_file'\r\n\r\nchef_vault_secret(\"ssh\") do\r\n  action [:create_if_missing]\r\n  retries 0\r\n  retry_delay 2\r\n  guard_interpreter :default\r\n  cookbook_name \"bcpc\"\r\n  recipe_name \"certs\"\r\n  data_bag \"os\"\r\n  raw_data {\"private-key\"=>\"<new key here since node can not read old key>\"}\r\n  admins \"cluster-r1n02.example.com,cluster-r1n03.example.com,cluster-r1n04.example.com,cluster-r1n05.example.com,cluster-r1n06.example.com,cluster-r1n07.example.com,cluster-r1n08.example.com,cluster-r1n10.example.com,cluster-r2n03.example.com,cluster-r2n04.example.com,cluster-r2n05.example.com,cluster-r2n06.example.com,cluster-r2n07.example.com,cluster-r2n08.example.com,cluster-r2n11.example.com,bcpc-cluster-bootstrap\"\r\n  search \"*:*\"\r\n  id \"ssh\"\r\nend\r\n\r\n\r\n````"},"comment":{"url":"https://api.github.com/repos/bloomberg/chef-bach/issues/comments/163686520","html_url":"https://github.com/bloomberg/chef-bach/issues/345#issuecomment-163686520","issue_url":"https://api.github.com/repos/bloomberg/chef-bach/issues/345","id":163686520,"user":{"login":"http-418","id":1435487,"avatar_url":"https://avatars.githubusercontent.com/u/1435487?v=3","gravatar_id":"","url":"https://api.github.com/users/http-418","html_url":"https://github.com/http-418","followers_url":"https://api.github.com/users/http-418/followers","following_url":"https://api.github.com/users/http-418/following{/other_user}","gists_url":"https://api.github.com/users/http-418/gists{/gist_id}","starred_url":"https://api.github.com/users/http-418/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/http-418/subscriptions","organizations_url":"https://api.github.com/users/http-418/orgs","repos_url":"https://api.github.com/users/http-418/repos","events_url":"https://api.github.com/users/http-418/events{/privacy}","received_events_url":"https://api.github.com/users/http-418/received_events","type":"User","site_admin":false},"created_at":"2015-12-10T16:56:43Z","updated_at":"2015-12-10T16:56:43Z","body":"After manually decrypting the shared secret with a non-working client's key, I am successfully able to retrieve the encrypted data bag.   We do not have a data problem.  Something isn't working right in the chef client/server.\r\n\r\nNext steps: debugger."}}