{"action":"created","issue":{"url":"https://api.github.com/repos/saltstack/salt/issues/29546","labels_url":"https://api.github.com/repos/saltstack/salt/issues/29546/labels{/name}","comments_url":"https://api.github.com/repos/saltstack/salt/issues/29546/comments","events_url":"https://api.github.com/repos/saltstack/salt/issues/29546/events","html_url":"https://github.com/saltstack/salt/issues/29546","id":121130232,"number":29546,"title":"Can't seem to get orchestrate to recognize a failed state","user":{"login":"jefferyharrell","id":8659114,"avatar_url":"https://avatars.githubusercontent.com/u/8659114?v=3","gravatar_id":"","url":"https://api.github.com/users/jefferyharrell","html_url":"https://github.com/jefferyharrell","followers_url":"https://api.github.com/users/jefferyharrell/followers","following_url":"https://api.github.com/users/jefferyharrell/following{/other_user}","gists_url":"https://api.github.com/users/jefferyharrell/gists{/gist_id}","starred_url":"https://api.github.com/users/jefferyharrell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jefferyharrell/subscriptions","organizations_url":"https://api.github.com/users/jefferyharrell/orgs","repos_url":"https://api.github.com/users/jefferyharrell/repos","events_url":"https://api.github.com/users/jefferyharrell/events{/privacy}","received_events_url":"https://api.github.com/users/jefferyharrell/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/saltstack/salt/labels/Bug","name":"Bug","color":"e11d21"},{"url":"https://api.github.com/repos/saltstack/salt/labels/Confirmed","name":"Confirmed","color":"c7def8"},{"url":"https://api.github.com/repos/saltstack/salt/labels/Core","name":"Core","color":"fef2c0"},{"url":"https://api.github.com/repos/saltstack/salt/labels/High%20Severity","name":"High Severity","color":"ff9143"},{"url":"https://api.github.com/repos/saltstack/salt/labels/P1","name":"P1","color":"2181ee"},{"url":"https://api.github.com/repos/saltstack/salt/labels/State%20Module","name":"State Module","color":"006b75"},{"url":"https://api.github.com/repos/saltstack/salt/labels/TEAM%20Core","name":"TEAM Core","color":"A9A9A9"}],"state":"open","locked":false,"assignee":{"login":"basepi","id":702318,"avatar_url":"https://avatars.githubusercontent.com/u/702318?v=3","gravatar_id":"","url":"https://api.github.com/users/basepi","html_url":"https://github.com/basepi","followers_url":"https://api.github.com/users/basepi/followers","following_url":"https://api.github.com/users/basepi/following{/other_user}","gists_url":"https://api.github.com/users/basepi/gists{/gist_id}","starred_url":"https://api.github.com/users/basepi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/basepi/subscriptions","organizations_url":"https://api.github.com/users/basepi/orgs","repos_url":"https://api.github.com/users/basepi/repos","events_url":"https://api.github.com/users/basepi/events{/privacy}","received_events_url":"https://api.github.com/users/basepi/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/saltstack/salt/milestones/70","html_url":"https://github.com/saltstack/salt/milestones/B%207","labels_url":"https://api.github.com/repos/saltstack/salt/milestones/70/labels","id":1425604,"number":70,"title":"B 7","description":"","creator":{"login":"meggiebot","id":12242451,"avatar_url":"https://avatars.githubusercontent.com/u/12242451?v=3","gravatar_id":"","url":"https://api.github.com/users/meggiebot","html_url":"https://github.com/meggiebot","followers_url":"https://api.github.com/users/meggiebot/followers","following_url":"https://api.github.com/users/meggiebot/following{/other_user}","gists_url":"https://api.github.com/users/meggiebot/gists{/gist_id}","starred_url":"https://api.github.com/users/meggiebot/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/meggiebot/subscriptions","organizations_url":"https://api.github.com/users/meggiebot/orgs","repos_url":"https://api.github.com/users/meggiebot/repos","events_url":"https://api.github.com/users/meggiebot/events{/privacy}","received_events_url":"https://api.github.com/users/meggiebot/received_events","type":"User","site_admin":false},"open_issues":35,"closed_issues":18,"state":"open","created_at":"2015-11-23T17:33:02Z","updated_at":"2015-12-10T17:58:05Z","due_on":"2015-12-14T07:00:00Z","closed_at":null},"comments":4,"created_at":"2015-12-09T00:07:22Z","updated_at":"2015-12-10T18:57:23Z","closed_at":null,"body":"(This is reminiscent of, and may be somehow related to, #29465.)\r\n\r\nI can't seem to get `state.orchestrate` to recognize when a state has failed. My minimal test case consists of two state files and an orchestration.\r\n\r\nstate/whatever/test1.sls:\r\n```salt\r\nteststate1:\r\n  test.fail_without_changes: []\r\n```\r\n\r\nstate/whatever/test2.sls:\r\n```salt\r\nteststate2:\r\n  test.succeed_without_changes: []\r\n```\r\n\r\norchestrate/testorchestration.sls:\r\n```salt\r\ntest1:\r\n  salt.state:\r\n    - tgt: bonnie.dallas.charlietango.com\r\n    - sls:\r\n      - state.whatever.test1\r\n\r\ntest2:\r\n  salt.state:\r\n    - tgt: bonnie.dallas.charlietango.com\r\n    - sls:\r\n      - state.whatever.test2\r\n    - require:\r\n      - salt: test1\r\n```\r\n\r\nWhat *should* happen, assuming I've done this right, is that `test1` should fail and `test2` should never be executed. But instead I get this:\r\n\r\n```\r\n[admin@bonnie ~]$ sudo salt-run state.orchestrate orchestrate.testorchestration\r\nbonnie.dallas.charlietango.com_master:\r\n----------\r\n          ID: test1\r\n    Function: salt.state\r\n      Result: True\r\n     Comment: States ran successfully. No changes made to bonnie.dallas.charlietango.com.\r\n     Started: 17:56:11.371163\r\n    Duration: 424.23 ms\r\n     Changes:   \r\n----------\r\n          ID: test2\r\n    Function: salt.state\r\n      Result: True\r\n     Comment: States ran successfully. No changes made to bonnie.dallas.charlietango.com.\r\n     Started: 17:56:11.795647\r\n    Duration: 427.469 ms\r\n     Changes:   \r\n\r\nSummary for bonnie.dallas.charlietango.com_master\r\n------------\r\nSucceeded: 2\r\nFailed:    0\r\n------------\r\nTotal states run:     2\r\nTotal run time: 851.699 ms\r\n```\r\n\r\nHere's the relevant excerpt from the event bus:\r\n\r\n```\r\nsalt/run/20151208175609296042/new   {\r\n    \"_stamp\": \"2015-12-08T23:56:09.514278\", \r\n    \"fun\": \"runner.state.orchestrate\", \r\n    \"jid\": \"20151208175609296042\", \r\n    \"user\": \"sudo_admin\"\r\n}\r\nsalt/job/20151208175611414293/new   {\r\n    \"_stamp\": \"2015-12-08T23:56:11.415033\", \r\n    \"arg\": [\r\n        \"state.whatever.test1\", \r\n        {\r\n            \"__kwarg__\": true, \r\n            \"concurrent\": false, \r\n            \"saltenv\": \"base\"\r\n        }\r\n    ], \r\n    \"fun\": \"state.sls\", \r\n    \"jid\": \"20151208175611414293\", \r\n    \"minions\": [\r\n        \"bonnie.dallas.charlietango.com\"\r\n    ], \r\n    \"tgt\": \"bonnie.dallas.charlietango.com\", \r\n    \"tgt_type\": \"glob\", \r\n    \"user\": \"sudo_admin\"\r\n}\r\nsalt/job/20151208175611414293/ret/bonnie.dallas.charlietango.com    {\r\n    \"_stamp\": \"2015-12-08T23:56:11.759752\", \r\n    \"cmd\": \"_return\", \r\n    \"fun\": \"state.sls\", \r\n    \"fun_args\": [\r\n        \"state.whatever.test1\", \r\n        {\r\n            \"concurrent\": false, \r\n            \"saltenv\": \"base\"\r\n        }\r\n    ], \r\n    \"id\": \"bonnie.dallas.charlietango.com\", \r\n    \"jid\": \"20151208175611414293\", \r\n    \"out\": \"highstate\", \r\n    \"retcode\": 2, \r\n    \"return\": {\r\n        \"test_|-teststate1_|-teststate1_|-fail_without_changes\": {\r\n            \"__run_num__\": 0, \r\n            \"changes\": {}, \r\n            \"comment\": \"Failure!\", \r\n            \"duration\": 1.059, \r\n            \"name\": \"teststate1\", \r\n            \"result\": false, \r\n            \"start_time\": \"17:56:11.650190\"\r\n        }\r\n    }, \r\n    \"success\": true\r\n}\r\nsalt/job/20151208175611832721/new   {\r\n    \"_stamp\": \"2015-12-08T23:56:11.833573\", \r\n    \"arg\": [\r\n        \"state.whatever.test2\", \r\n        {\r\n            \"__kwarg__\": true, \r\n            \"concurrent\": false, \r\n            \"saltenv\": \"base\"\r\n        }\r\n    ], \r\n    \"fun\": \"state.sls\", \r\n    \"jid\": \"20151208175611832721\", \r\n    \"minions\": [\r\n        \"bonnie.dallas.charlietango.com\"\r\n    ], \r\n    \"tgt\": \"bonnie.dallas.charlietango.com\", \r\n    \"tgt_type\": \"glob\", \r\n    \"user\": \"sudo_admin\"\r\n}\r\nsalt/job/20151208175611832721/ret/bonnie.dallas.charlietango.com    {\r\n    \"_stamp\": \"2015-12-08T23:56:12.186007\", \r\n    \"cmd\": \"_return\", \r\n    \"fun\": \"state.sls\", \r\n    \"fun_args\": [\r\n        \"state.whatever.test2\", \r\n        {\r\n            \"concurrent\": false, \r\n            \"saltenv\": \"base\"\r\n        }\r\n    ], \r\n    \"id\": \"bonnie.dallas.charlietango.com\", \r\n    \"jid\": \"20151208175611832721\", \r\n    \"out\": \"highstate\", \r\n    \"retcode\": 0, \r\n    \"return\": {\r\n        \"test_|-teststate2_|-teststate2_|-succeed_without_changes\": {\r\n            \"__run_num__\": 0, \r\n            \"changes\": {}, \r\n            \"comment\": \"Success!\", \r\n            \"duration\": 0.939, \r\n            \"name\": \"teststate2\", \r\n            \"result\": true, \r\n            \"start_time\": \"17:56:12.077965\"\r\n        }\r\n    }, \r\n    \"success\": true\r\n}\r\nsalt/run/20151208175609296042/ret   {\r\n    \"_stamp\": \"2015-12-08T23:56:12.227510\", \r\n    \"fun\": \"runner.state.orchestrate\", \r\n    \"jid\": \"20151208175609296042\", \r\n    \"return\": {\r\n        \"data\": {\r\n            \"bonnie.dallas.charlietango.com_master\": {\r\n                \"salt_|-test1_|-test1_|-state\": {\r\n                    \"__run_num__\": 0, \r\n                    \"changes\": {}, \r\n                    \"comment\": \"States ran successfully. No changes made to bonnie.dallas.charlietango.com.\", \r\n                    \"duration\": \"424.23 ms\", \r\n                    \"name\": \"test1\", \r\n                    \"result\": true, \r\n                    \"start_time\": \"17:56:11.371163\"\r\n                }, \r\n                \"salt_|-test2_|-test2_|-state\": {\r\n                    \"__run_num__\": 1, \r\n                    \"changes\": {}, \r\n                    \"comment\": \"States ran successfully. No changes made to bonnie.dallas.charlietango.com.\", \r\n                    \"duration\": \"427.469 ms\", \r\n                    \"name\": \"test2\", \r\n                    \"result\": true, \r\n                    \"start_time\": \"17:56:11.795647\"\r\n                }\r\n            }\r\n        }, \r\n        \"outputter\": \"highstate\"\r\n    }, \r\n    \"success\": true, \r\n    \"user\": \"sudo_admin\"\r\n}\r\n```\r\n\r\nIt looks *to me* — and please recall that I'm a rank amateur at Salt — like the test1 state is pretty clearly returning word of its failure, but that the orchestrate runner is failing to correctly interpret that ret. But again, I'm still so new at this that you can practically assume I'm making all this up as I go.\r\n\r\nIn any case, right now this is preventing me from deploying Salt in production. I really want to be able to trap failed states and interrupt orchestrations that have gone sideways before they wreak havoc. Any suggestions for such a workaround would be *incredibly* welcome.\r\n\r\nThanks!"},"comment":{"url":"https://api.github.com/repos/saltstack/salt/issues/comments/163717192","html_url":"https://github.com/saltstack/salt/issues/29546#issuecomment-163717192","issue_url":"https://api.github.com/repos/saltstack/salt/issues/29546","id":163717192,"user":{"login":"basepi","id":702318,"avatar_url":"https://avatars.githubusercontent.com/u/702318?v=3","gravatar_id":"","url":"https://api.github.com/users/basepi","html_url":"https://github.com/basepi","followers_url":"https://api.github.com/users/basepi/followers","following_url":"https://api.github.com/users/basepi/following{/other_user}","gists_url":"https://api.github.com/users/basepi/gists{/gist_id}","starred_url":"https://api.github.com/users/basepi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/basepi/subscriptions","organizations_url":"https://api.github.com/users/basepi/orgs","repos_url":"https://api.github.com/users/basepi/repos","events_url":"https://api.github.com/users/basepi/events{/privacy}","received_events_url":"https://api.github.com/users/basepi/received_events","type":"User","site_admin":false},"created_at":"2015-12-10T18:57:23Z","updated_at":"2015-12-10T18:57:23Z","body":"I think I've tracked down the bug. I'm just trying to track down where it broke so I can make sure I'm not breaking something else by fixing it."}}