{"action":"created","issue":{"url":"https://api.github.com/repos/saltstack/salt/issues/19864","labels_url":"https://api.github.com/repos/saltstack/salt/issues/19864/labels{/name}","comments_url":"https://api.github.com/repos/saltstack/salt/issues/19864/comments","events_url":"https://api.github.com/repos/saltstack/salt/issues/19864/events","html_url":"https://github.com/saltstack/salt/issues/19864","id":54864011,"number":19864,"title":"Salt syndic architecture is slow","user":{"login":"claudiupopescu","id":10415709,"avatar_url":"https://avatars.githubusercontent.com/u/10415709?v=3","gravatar_id":"","url":"https://api.github.com/users/claudiupopescu","html_url":"https://github.com/claudiupopescu","followers_url":"https://api.github.com/users/claudiupopescu/followers","following_url":"https://api.github.com/users/claudiupopescu/following{/other_user}","gists_url":"https://api.github.com/users/claudiupopescu/gists{/gist_id}","starred_url":"https://api.github.com/users/claudiupopescu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/claudiupopescu/subscriptions","organizations_url":"https://api.github.com/users/claudiupopescu/orgs","repos_url":"https://api.github.com/users/claudiupopescu/repos","events_url":"https://api.github.com/users/claudiupopescu/events{/privacy}","received_events_url":"https://api.github.com/users/claudiupopescu/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/saltstack/salt/labels/Blocker","name":"Blocker","color":"e11d21"},{"url":"https://api.github.com/repos/saltstack/salt/labels/Bug","name":"Bug","color":"e11d21"},{"url":"https://api.github.com/repos/saltstack/salt/labels/Fixed%20Pending%20Verification","name":"Fixed Pending Verification","color":"d4c5f9"}],"state":"open","locked":false,"assignee":{"login":"cachedout","id":111616,"avatar_url":"https://avatars.githubusercontent.com/u/111616?v=3","gravatar_id":"","url":"https://api.github.com/users/cachedout","html_url":"https://github.com/cachedout","followers_url":"https://api.github.com/users/cachedout/followers","following_url":"https://api.github.com/users/cachedout/following{/other_user}","gists_url":"https://api.github.com/users/cachedout/gists{/gist_id}","starred_url":"https://api.github.com/users/cachedout/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cachedout/subscriptions","organizations_url":"https://api.github.com/users/cachedout/orgs","repos_url":"https://api.github.com/users/cachedout/repos","events_url":"https://api.github.com/users/cachedout/events{/privacy}","received_events_url":"https://api.github.com/users/cachedout/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/saltstack/salt/milestones/53","html_url":"https://github.com/saltstack/salt/milestones/2014.7.3","labels_url":"https://api.github.com/repos/saltstack/salt/milestones/53/labels","id":1015263,"number":53,"title":"2014.7.3","description":"","creator":{"login":"jfindlay","id":702905,"avatar_url":"https://avatars.githubusercontent.com/u/702905?v=3","gravatar_id":"","url":"https://api.github.com/users/jfindlay","html_url":"https://github.com/jfindlay","followers_url":"https://api.github.com/users/jfindlay/followers","following_url":"https://api.github.com/users/jfindlay/following{/other_user}","gists_url":"https://api.github.com/users/jfindlay/gists{/gist_id}","starred_url":"https://api.github.com/users/jfindlay/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jfindlay/subscriptions","organizations_url":"https://api.github.com/users/jfindlay/orgs","repos_url":"https://api.github.com/users/jfindlay/repos","events_url":"https://api.github.com/users/jfindlay/events{/privacy}","received_events_url":"https://api.github.com/users/jfindlay/received_events","type":"User","site_admin":false},"open_issues":4,"closed_issues":2,"state":"open","created_at":"2015-03-10T18:48:27Z","updated_at":"2015-03-17T16:34:44Z","due_on":null,"closed_at":null},"comments":15,"created_at":"2015-01-20T10:04:04Z","updated_at":"2015-03-18T21:48:44Z","closed_at":null,"body":"Hi,\r\n\r\nI found something extremely close to what I have right now and the same problem: http://stackoverflow.com/questions/27925274/3-level-topology-with-saltstack-salt-syndic-performs-very-slowly\r\n\r\n1. Salt master\r\n2. Salt master + syndic + minion\r\n3. Sal minion\r\n\r\nWhen I run test.ping from 1 (which is master of master) it returns true for all hosts but before exiting it waits around 10 more seconds.\r\n\r\nIs there any one else with this problem? If so is some bug or something not configured correctly?\r\nFor me this started happening since I enabled: order_masters on the master of master without having the lower level master connected.\r\nI thought it will go away after I connect a lower level master but it did not."},"comment":{"url":"https://api.github.com/repos/saltstack/salt/issues/comments/83197007","html_url":"https://github.com/saltstack/salt/issues/19864#issuecomment-83197007","issue_url":"https://api.github.com/repos/saltstack/salt/issues/19864","id":83197007,"user":{"login":"rallytime","id":6547906,"avatar_url":"https://avatars.githubusercontent.com/u/6547906?v=3","gravatar_id":"","url":"https://api.github.com/users/rallytime","html_url":"https://github.com/rallytime","followers_url":"https://api.github.com/users/rallytime/followers","following_url":"https://api.github.com/users/rallytime/following{/other_user}","gists_url":"https://api.github.com/users/rallytime/gists{/gist_id}","starred_url":"https://api.github.com/users/rallytime/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rallytime/subscriptions","organizations_url":"https://api.github.com/users/rallytime/orgs","repos_url":"https://api.github.com/users/rallytime/repos","events_url":"https://api.github.com/users/rallytime/events{/privacy}","received_events_url":"https://api.github.com/users/rallytime/received_events","type":"User","site_admin":false},"created_at":"2015-03-18T21:48:44Z","updated_at":"2015-03-18T21:48:44Z","body":"After testing the fixes above from @cachedout, we have determined that the syndic architecture is now much more stable. However, the \"slow architecture\" remains an issue. \r\n\r\nBasically what is happening is when the syndic does not have a minion installed on it (as the current docs say is possible, and indeed it is) the syndic (without the minion) acts as a minion to the master. The syndic ends up in the master's expected minions list, even though it won't return since it's not actually a minion, which causes the timeouts to extend until the limit is reached. According to @thatch45, this has been an issue with the syndic even since syndic was created.\r\n\r\nThe workaround for this is simply to install a minion along with the syndic. This will reduce the amount of CLI wait time back to the expected behavior of ~5 seconds as a default value. This can be further tuned, as stated by @cachedout above, in the `syndic_wait` value."}}