{"action":"created","issue":{"url":"https://api.github.com/repos/chef/chef/issues/2786","labels_url":"https://api.github.com/repos/chef/chef/issues/2786/labels{/name}","comments_url":"https://api.github.com/repos/chef/chef/issues/2786/comments","events_url":"https://api.github.com/repos/chef/chef/issues/2786/events","html_url":"https://github.com/chef/chef/issues/2786","id":55184262,"number":2786,"title":"chef-client 11.18.0 can't verify s3 certs","user":{"login":"glasschef","id":6391892,"avatar_url":"https://avatars.githubusercontent.com/u/6391892?v=3","gravatar_id":"","url":"https://api.github.com/users/glasschef","html_url":"https://github.com/glasschef","followers_url":"https://api.github.com/users/glasschef/followers","following_url":"https://api.github.com/users/glasschef/following{/other_user}","gists_url":"https://api.github.com/users/glasschef/gists{/gist_id}","starred_url":"https://api.github.com/users/glasschef/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/glasschef/subscriptions","organizations_url":"https://api.github.com/users/glasschef/orgs","repos_url":"https://api.github.com/users/glasschef/repos","events_url":"https://api.github.com/users/glasschef/events{/privacy}","received_events_url":"https://api.github.com/users/glasschef/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-01-22T17:38:02Z","updated_at":"2015-01-29T21:45:46Z","closed_at":"2015-01-29T21:45:46Z","body":"When using Hosted Chef and specifying `verify_api_cert true` and/or `ssl_verify_mode :verify_peer` in client.rb, chef-client 11.18.0 will error out when trying to verify the S3 SSL certs when downloading cookbooks initially or modified cookbook data (templates, etc).  Not even adding the s3 certs to the `trusted_certs` directory will help.\r\n\r\n\r\n## Initial cookbook download failure:\r\n```\r\nroot@machine:/etc/chef# chef-client\r\nStarting Chef Client, version 11.18.0\r\nresolving cookbooks for run list: [\"ntp\"]\r\nSynchronizing Cookbooks:\r\n[2015-01-22T12:08:35-05:00] ERROR: SSL Validation failure connecting to host: s3-external-1.amazonaws.com - SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed\r\n[2015-01-22T12:08:35-05:00] ERROR: SSL Validation failure connecting to host: s3-external-1.amazonaws.com - SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed\r\n[2015-01-22T12:08:35-05:00] ERROR: SSL Validation failure connecting to host: s3-external-1.amazonaws.com - SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed\r\n[2015-01-22T12:08:35-05:00] ERROR: SSL Validation failure connecting to host: s3-external-1.amazonaws.com - SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed\r\n[2015-01-22T12:08:35-05:00] ERROR: SSL Validation failure connecting to host: s3-external-1.amazonaws.com - SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed\r\n\r\n  ================================================================================\r\n  Error Syncing Cookbooks:\r\n  ================================================================================\r\n  [2015-01-22T12:08:35-05:00] ERROR: SSL Validation failure connecting to host: s3-external-1.amazonaws.com - SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed\r\n\r\n  Unexpected Error:\r\n  -----------------\r\n  OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed\r\n\r\n\r\n  Running handlers:\r\n[2015-01-22T12:08:35-05:00] ERROR: Running exception handlers\r\n  Running handlers complete\r\n[2015-01-22T12:08:35-05:00] ERROR: Exception handlers complete\r\n[2015-01-22T12:08:35-05:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out\r\n  Chef Client failed. 0 resources updated in 3.455441684 seconds\r\n[2015-01-22T12:08:35-05:00] ERROR: SSL Validation failure connecting to host: s3-external-1.amazonaws.com - SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed\r\n[2015-01-22T12:08:35-05:00] ERROR: SSL Validation failure connecting to host: s3-external-1.amazonaws.com - SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed\r\n[2015-01-22T12:08:35-05:00] ERROR: SSL Validation failure connecting to host: s3-external-1.amazonaws.com - SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed\r\n[2015-01-22T12:08:35-05:00] ERROR: SSL Validation failure connecting to host: s3-external-1.amazonaws.com - SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed\r\n[2015-01-22T12:08:36-05:00] ERROR: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed\r\n[2015-01-22T12:08:36-05:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)\r\n```\r\n\r\n## Updating existing cookbooks:\r\n```\r\nroot@machine:~# chef-client\r\nStarting Chef Client, version 11.18.0\r\nresolving cookbooks for run list: [\"ntp\"]\r\nSynchronizing Cookbooks:\r\n  - ntp\r\nCompiling Cookbooks...\r\nConverging 9 resources\r\nRecipe: ntp::default\r\n  * package[ntp] action install (up to date)\r\n  * package[ntpdate] action install (up to date)\r\n  * directory[/var/lib/ntp] action create (up to date)\r\n  * directory[/var/log/ntpstats/] action create (up to date)\r\n  * cookbook_file[/etc/ntp.leapseconds] action create (up to date)\r\nRecipe: ntp::apparmor\r\n  * service[apparmor] action nothing (skipped due to action :nothing)\r\n  * cookbook_file[/etc/apparmor.d/usr.sbin.ntpd] action create (up to date)\r\nRecipe: ntp::default\r\n  * template[/etc/ntp.conf] action create[2015-01-22T12:31:36-05:00] ERROR: SSL Validation failure connecting to host: s3-external-1.amazonaws.com - SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed\r\n\r\n\r\n    ================================================================================\r\n    Error executing action `create` on resource 'template[/etc/ntp.conf]'\r\n    ================================================================================\r\n\r\n    OpenSSL::SSL::SSLError\r\n    ----------------------\r\n    SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed\r\n\r\n    Resource Declaration:\r\n    ---------------------\r\n    # In /var/chef/cache/cookbooks/ntp/recipes/default.rb\r\n\r\n     77: template node['ntp']['conffile'] do\r\n     78:   source   'ntp.conf.erb'\r\n     79:   owner    node['ntp']['conf_owner']\r\n     80:   group    node['ntp']['conf_group']\r\n     81:   mode     '0644'\r\n     82:   notifies :restart, \"service[#{node['ntp']['service']}]\"\r\n     83:   variables(\r\n     84:       :ntpd_supports_native_leapfiles => leapfile_enabled\r\n     85:   )\r\n     86: end\r\n     87:\r\n\r\n    Compiled Resource:\r\n    ------------------\r\n    # Declared in /var/chef/cache/cookbooks/ntp/recipes/default.rb:77:in `from_file'\r\n\r\n    template(\"/etc/ntp.conf\") do\r\n      provider Chef::Provider::Template\r\n      action \"create\"\r\n      retries 0\r\n      retry_delay 2\r\n      guard_interpreter :default\r\n      path \"/etc/ntp.conf\"\r\n      backup 5\r\n      atomic_update true\r\n      source \"ntp.conf.erb\"\r\n      variables {:ntpd_supports_native_leapfiles=>true}\r\n      cookbook_name \"ntp\"\r\n      recipe_name \"default\"\r\n      owner \"root\"\r\n      group \"root\"\r\n      mode \"0644\"\r\n    end\r\n\r\n\r\nRunning handlers:\r\n[2015-01-22T12:31:36-05:00] ERROR: Running exception handlers\r\nRunning handlers complete\r\n[2015-01-22T12:31:36-05:00] ERROR: Exception handlers complete\r\n[2015-01-22T12:31:36-05:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out\r\nChef Client failed. 0 resources updated in 5.549682464 seconds\r\n[2015-01-22T12:31:37-05:00] ERROR: template[/etc/ntp.conf] (ntp::default line 77) had an error: OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed\r\n[2015-01-22T12:31:37-05:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)\r\n```\r\n\r\n## Skipping SSL verification works fine:\r\n\r\n\r\n```\r\nroot@machine:/etc/chef# chef-client\r\n[2015-01-22T12:11:51-05:00] WARN:\r\n* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\r\nSSL validation of HTTPS requests is disabled. HTTPS connections are still\r\nencrypted, but chef is not able to detect forged replies or man in the middle\r\nattacks.\r\n\r\nTo fix this issue add an entry like this to your configuration file:\r\n\r\n...\r\n  # Verify all HTTPS connections (recommended)\r\n  ssl_verify_mode :verify_peer\r\n\r\n  # OR, Verify only connections to chef-server\r\n  verify_api_cert true\r\n...\r\n\r\nTo check your SSL configuration, or troubleshoot errors, you can use the\r\n`knife ssl check` command like so:\r\n\r\n...\r\n  knife ssl check -c /etc/chef/client.rb\r\n...\r\n\r\n* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\r\n\r\nStarting Chef Client, version 11.18.0\r\nresolving cookbooks for run list: [\"ntp\"]\r\nSynchronizing Cookbooks:\r\n  - ntp\r\nCompiling Cookbooks...\r\nConverging 9 resources\r\nRecipe: ntp::default\r\n  * package[ntp] action install (up to date)\r\n  * package[ntpdate] action install (up to date)\r\n  * directory[/var/lib/ntp] action create (up to date)\r\n  * directory[/var/log/ntpstats/] action create (up to date)\r\n  * cookbook_file[/etc/ntp.leapseconds] action create (up to date)\r\nRecipe: ntp::apparmor\r\n  * service[apparmor] action nothing (skipped due to action :nothing)\r\n  * cookbook_file[/etc/apparmor.d/usr.sbin.ntpd] action create (up to date)\r\nRecipe: ntp::default\r\n  * template[/etc/ntp.conf] action create (up to date)\r\n  * service[ntp] action enable (up to date)\r\n  * service[ntp] action start (up to date)\r\n\r\nRunning handlers:\r\nRunning handlers complete\r\nChef Client finished, 0/9 resources updated in 5.937778964 seconds\r\n```\r\n\r\n## chef-client 12 works fine with ssl verification:\r\n\r\n```\r\nroot@machine:~# chef-client\r\nStarting Chef Client, version 12.0.3\r\nresolving cookbooks for run list: []\r\nSynchronizing Cookbooks:\r\nCompiling Cookbooks...\r\n[2015-01-22T12:16:10-05:00] WARN: Node bizarre has an empty run list.\r\nConverging 0 resources\r\n\r\nRunning handlers:\r\nRunning handlers complete\r\nChef Client finished, 0/0 resources updated in 4.710564134 seconds\r\nroot@bizarre:~# chef-client\r\nStarting Chef Client, version 12.0.3\r\nresolving cookbooks for run list: [\"ntp\"]\r\nSynchronizing Cookbooks:\r\n  - ntp\r\nCompiling Cookbooks...\r\nConverging 9 resources\r\nRecipe: ntp::default\r\n  * apt_package[ntp] action install (up to date)\r\n  * apt_package[ntpdate] action install (up to date)\r\n  * directory[/var/lib/ntp] action create (up to date)\r\n  * directory[/var/log/ntpstats/] action create (up to date)\r\n  * cookbook_file[/etc/ntp.leapseconds] action create (up to date)\r\nRecipe: ntp::apparmor\r\n  * service[apparmor] action nothing (skipped due to action :nothing)\r\n  * cookbook_file[/etc/apparmor.d/usr.sbin.ntpd] action create (up to date)\r\nRecipe: ntp::default\r\n  * template[/etc/ntp.conf] action create (up to date)\r\n  * service[ntp] action enable (up to date)\r\n  * service[ntp] action start (up to date)\r\n\r\nRunning handlers:\r\nRunning handlers complete\r\nChef Client finished, 0/9 resources updated in 9.751752921 seconds\r\n```"},"comment":{"url":"https://api.github.com/repos/chef/chef/issues/comments/72110629","html_url":"https://github.com/chef/chef/issues/2786#issuecomment-72110629","issue_url":"https://api.github.com/repos/chef/chef/issues/2786","id":72110629,"user":{"login":"jdmundrawala","id":27443,"avatar_url":"https://avatars.githubusercontent.com/u/27443?v=3","gravatar_id":"","url":"https://api.github.com/users/jdmundrawala","html_url":"https://github.com/jdmundrawala","followers_url":"https://api.github.com/users/jdmundrawala/followers","following_url":"https://api.github.com/users/jdmundrawala/following{/other_user}","gists_url":"https://api.github.com/users/jdmundrawala/gists{/gist_id}","starred_url":"https://api.github.com/users/jdmundrawala/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jdmundrawala/subscriptions","organizations_url":"https://api.github.com/users/jdmundrawala/orgs","repos_url":"https://api.github.com/users/jdmundrawala/repos","events_url":"https://api.github.com/users/jdmundrawala/events{/privacy}","received_events_url":"https://api.github.com/users/jdmundrawala/received_events","type":"User","site_admin":false},"created_at":"2015-01-29T21:45:46Z","updated_at":"2015-01-29T21:45:46Z","body":"Should be fixed, please open another issue if there are still problems"}}