{"action":"created","issue":{"url":"https://api.github.com/repos/saltstack/salt/issues/24010","labels_url":"https://api.github.com/repos/saltstack/salt/issues/24010/labels{/name}","comments_url":"https://api.github.com/repos/saltstack/salt/issues/24010/comments","events_url":"https://api.github.com/repos/saltstack/salt/issues/24010/events","html_url":"https://github.com/saltstack/salt/issues/24010","id":78867612,"number":24010,"title":"salt-master gitfs find_file is unefficient ","user":{"login":"hvnsweeting","id":1000665,"avatar_url":"https://avatars.githubusercontent.com/u/1000665?v=3","gravatar_id":"","url":"https://api.github.com/users/hvnsweeting","html_url":"https://github.com/hvnsweeting","followers_url":"https://api.github.com/users/hvnsweeting/followers","following_url":"https://api.github.com/users/hvnsweeting/following{/other_user}","gists_url":"https://api.github.com/users/hvnsweeting/gists{/gist_id}","starred_url":"https://api.github.com/users/hvnsweeting/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hvnsweeting/subscriptions","organizations_url":"https://api.github.com/users/hvnsweeting/orgs","repos_url":"https://api.github.com/users/hvnsweeting/repos","events_url":"https://api.github.com/users/hvnsweeting/events{/privacy}","received_events_url":"https://api.github.com/users/hvnsweeting/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/saltstack/salt/labels/Bug","name":"Bug","color":"e11d21"},{"url":"https://api.github.com/repos/saltstack/salt/labels/File%20Servers","name":"File Servers","color":"006b75"},{"url":"https://api.github.com/repos/saltstack/salt/labels/High%20Severity","name":"High Severity","color":"ff9143"},{"url":"https://api.github.com/repos/saltstack/salt/labels/P3","name":"P3","color":"0a3d77"},{"url":"https://api.github.com/repos/saltstack/salt/labels/Platform%20Mgmt","name":"Platform Mgmt","color":"006b75"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/saltstack/salt/milestones/13","html_url":"https://github.com/saltstack/salt/milestones/Approved","labels_url":"https://api.github.com/repos/saltstack/salt/milestones/13/labels","id":9265,"number":13,"title":"Approved","description":"All issues that are ready to be worked on, both bugs and features.","creator":{"login":"thatch45","id":507599,"avatar_url":"https://avatars.githubusercontent.com/u/507599?v=3","gravatar_id":"","url":"https://api.github.com/users/thatch45","html_url":"https://github.com/thatch45","followers_url":"https://api.github.com/users/thatch45/followers","following_url":"https://api.github.com/users/thatch45/following{/other_user}","gists_url":"https://api.github.com/users/thatch45/gists{/gist_id}","starred_url":"https://api.github.com/users/thatch45/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/thatch45/subscriptions","organizations_url":"https://api.github.com/users/thatch45/orgs","repos_url":"https://api.github.com/users/thatch45/repos","events_url":"https://api.github.com/users/thatch45/events{/privacy}","received_events_url":"https://api.github.com/users/thatch45/received_events","type":"User","site_admin":false},"open_issues":2307,"closed_issues":2047,"state":"open","created_at":"2011-05-14T04:00:56Z","updated_at":"2015-05-28T01:02:56Z","due_on":null,"closed_at":null},"comments":2,"created_at":"2015-05-21T07:15:12Z","updated_at":"2015-05-28T04:24:13Z","closed_at":null,"body":"When I troubleshooted this problem: \r\n\r\nhttps://github.com/saltstack/salt/pull/23965\r\n\r\nthe reason that BadName exception was raised because gitfs code check branch X of repo B against repo A. As branch X is not in A, it ran off all the if condition and reach https://github.com/saltstack/salt/blob/25763016313ef4dab1276b942ca82939b2a89c73/salt/fileserver/gitfs.py#L393 then raise exception.\r\n\r\nThe check in https://github.com/saltstack/salt/blob/25763016313ef4dab1276b942ca82939b2a89c73/salt/fileserver/gitfs.py#L380 is not very useful, as envs() returns ALL envs by ALL repo, it will check many branch of repo B against repo A in this line https://github.com/saltstack/salt/blob/25763016313ef4dab1276b942ca82939b2a89c73/salt/fileserver/gitfs.py#L381\r\n\r\nwhen the number of repos increased, and the number of branches also increased, this will wasted many useless loop.\r\n\r\n````python\r\n# code copy here for easier reference\r\ndef _get_tree_gitpython(repo, tgt_env):\r\n    '''\r\n    Return a git.Tree object if the branch/tag/SHA is found, otherwise None\r\n    '''\r\n    if tgt_env == 'base':\r\n        tgt_env = repo['base']\r\n    if tgt_env == repo['base'] or tgt_env in envs():\r\n        for ref in repo['repo'].refs:\r\n            if isinstance(ref, (git.RemoteReference, git.TagReference)):\r\n                parted = ref.name.partition('/')\r\n                rspec = parted[2] if parted[2] else parted[0]\r\n                if rspec == tgt_env:\r\n                    return ref.commit.tree\r\n\r\n    # Branch or tag not matched, check if 'tgt_env' is a commit\r\n    if not _env_is_exposed(tgt_env):\r\n        return None\r\n\r\n    try:\r\n        commit = repo['repo'].rev_parse(tgt_env)\r\n        return commit.tree\r\n    except gitdb.exc.ODBError:\r\n        return None\r\n````"},"comment":{"url":"https://api.github.com/repos/saltstack/salt/issues/comments/106166303","html_url":"https://github.com/saltstack/salt/issues/24010#issuecomment-106166303","issue_url":"https://api.github.com/repos/saltstack/salt/issues/24010","id":106166303,"user":{"login":"hvnsweeting","id":1000665,"avatar_url":"https://avatars.githubusercontent.com/u/1000665?v=3","gravatar_id":"","url":"https://api.github.com/users/hvnsweeting","html_url":"https://github.com/hvnsweeting","followers_url":"https://api.github.com/users/hvnsweeting/followers","following_url":"https://api.github.com/users/hvnsweeting/following{/other_user}","gists_url":"https://api.github.com/users/hvnsweeting/gists{/gist_id}","starred_url":"https://api.github.com/users/hvnsweeting/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hvnsweeting/subscriptions","organizations_url":"https://api.github.com/users/hvnsweeting/orgs","repos_url":"https://api.github.com/users/hvnsweeting/repos","events_url":"https://api.github.com/users/hvnsweeting/events{/privacy}","received_events_url":"https://api.github.com/users/hvnsweeting/received_events","type":"User","site_admin":false},"created_at":"2015-05-28T04:24:13Z","updated_at":"2015-05-28T04:24:13Z","body":"@jfindlay yes I will and I'm working on it to stabilize salt-master from daily OOM"}}