{"action":"created","issue":{"url":"https://api.github.com/repos/letsencrypt/boulder/issues/1003","labels_url":"https://api.github.com/repos/letsencrypt/boulder/issues/1003/labels{/name}","comments_url":"https://api.github.com/repos/letsencrypt/boulder/issues/1003/comments","events_url":"https://api.github.com/repos/letsencrypt/boulder/issues/1003/events","html_url":"https://github.com/letsencrypt/boulder/issues/1003","id":112222495,"number":1003,"title":"boulder-ca should handle certain PKCS11 errors specially","user":{"login":"jsha","id":220205,"avatar_url":"https://avatars.githubusercontent.com/u/220205?v=3","gravatar_id":"","url":"https://api.github.com/users/jsha","html_url":"https://github.com/jsha","followers_url":"https://api.github.com/users/jsha/followers","following_url":"https://api.github.com/users/jsha/following{/other_user}","gists_url":"https://api.github.com/users/jsha/gists{/gist_id}","starred_url":"https://api.github.com/users/jsha/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jsha/subscriptions","organizations_url":"https://api.github.com/users/jsha/orgs","repos_url":"https://api.github.com/users/jsha/repos","events_url":"https://api.github.com/users/jsha/events{/privacy}","received_events_url":"https://api.github.com/users/jsha/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/letsencrypt/boulder/milestones/6","html_url":"https://github.com/letsencrypt/boulder/milestones/General%20Availability","labels_url":"https://api.github.com/repos/letsencrypt/boulder/milestones/6/labels","id":1121569,"number":6,"title":"General Availability","description":"","creator":{"login":"bdaehlie","id":84762,"avatar_url":"https://avatars.githubusercontent.com/u/84762?v=3","gravatar_id":"","url":"https://api.github.com/users/bdaehlie","html_url":"https://github.com/bdaehlie","followers_url":"https://api.github.com/users/bdaehlie/followers","following_url":"https://api.github.com/users/bdaehlie/following{/other_user}","gists_url":"https://api.github.com/users/bdaehlie/gists{/gist_id}","starred_url":"https://api.github.com/users/bdaehlie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bdaehlie/subscriptions","organizations_url":"https://api.github.com/users/bdaehlie/orgs","repos_url":"https://api.github.com/users/bdaehlie/repos","events_url":"https://api.github.com/users/bdaehlie/events{/privacy}","received_events_url":"https://api.github.com/users/bdaehlie/received_events","type":"User","site_admin":false},"open_issues":17,"closed_issues":114,"state":"open","created_at":"2015-05-20T16:53:19Z","updated_at":"2015-10-19T23:45:01Z","due_on":"2015-11-20T08:00:00Z","closed_at":null},"comments":1,"created_at":"2015-10-19T20:07:53Z","updated_at":"2015-10-19T23:45:24Z","closed_at":null,"body":"If boulder-ca gets a CKR_DEVICE_ERROR, it's likely to be unrecoverable without manual intervention. Rather than trying repeatedly and logging and AUDIT error each time, boulder-ca should have a global \"HSM down\" flag, protected by a lock, which it sets if it ever gets a CKR_DEVICE_ERROR. When this happens, the CA starts responding to all RPC requests with a \"service unavailable\" error. It also sets a stat to be collected and alerted on.\r\n\r\nSince there may be multiple inflight requests at the time of the error, the CA may still send some number of requests to the HSM after the first error. This is fine, and the only way to prevent it would be to hold the lock on the \"HSM down\" flag through the whole signing operation, which would kill performance.\r\n\r\nOpen question: I'm assuming the CA should only log as AUDIT the first errors it gets, not each subsequent \"service unavailable\" error?"},"comment":{"url":"https://api.github.com/repos/letsencrypt/boulder/issues/comments/149378364","html_url":"https://github.com/letsencrypt/boulder/issues/1003#issuecomment-149378364","issue_url":"https://api.github.com/repos/letsencrypt/boulder/issues/1003","id":149378364,"user":{"login":"jsha","id":220205,"avatar_url":"https://avatars.githubusercontent.com/u/220205?v=3","gravatar_id":"","url":"https://api.github.com/users/jsha","html_url":"https://github.com/jsha","followers_url":"https://api.github.com/users/jsha/followers","following_url":"https://api.github.com/users/jsha/following{/other_user}","gists_url":"https://api.github.com/users/jsha/gists{/gist_id}","starred_url":"https://api.github.com/users/jsha/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jsha/subscriptions","organizations_url":"https://api.github.com/users/jsha/orgs","repos_url":"https://api.github.com/users/jsha/repos","events_url":"https://api.github.com/users/jsha/events{/privacy}","received_events_url":"https://api.github.com/users/jsha/received_events","type":"User","site_admin":false},"created_at":"2015-10-19T23:45:24Z","updated_at":"2015-10-19T23:45:24Z","body":"Assigning to GA because it caused issues already."}}