{"action":"created","issue":{"url":"https://api.github.com/repos/magit/magit/issues/2400","labels_url":"https://api.github.com/repos/magit/magit/issues/2400/labels{/name}","comments_url":"https://api.github.com/repos/magit/magit/issues/2400/comments","events_url":"https://api.github.com/repos/magit/magit/issues/2400/events","html_url":"https://github.com/magit/magit/pull/2400","id":117649727,"number":2400,"title":"Optimize refresh for large repos","user":{"login":"ewstearns","id":752459,"avatar_url":"https://avatars.githubusercontent.com/u/752459?v=3","gravatar_id":"","url":"https://api.github.com/users/ewstearns","html_url":"https://github.com/ewstearns","followers_url":"https://api.github.com/users/ewstearns/followers","following_url":"https://api.github.com/users/ewstearns/following{/other_user}","gists_url":"https://api.github.com/users/ewstearns/gists{/gist_id}","starred_url":"https://api.github.com/users/ewstearns/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ewstearns/subscriptions","organizations_url":"https://api.github.com/users/ewstearns/orgs","repos_url":"https://api.github.com/users/ewstearns/repos","events_url":"https://api.github.com/users/ewstearns/events{/privacy}","received_events_url":"https://api.github.com/users/ewstearns/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2015-11-18T18:28:09Z","updated_at":"2015-11-23T21:00:08Z","closed_at":"2015-11-21T09:38:23Z","pull_request":{"url":"https://api.github.com/repos/magit/magit/pulls/2400","html_url":"https://github.com/magit/magit/pull/2400","diff_url":"https://github.com/magit/magit/pull/2400.diff","patch_url":"https://github.com/magit/magit/pull/2400.patch"},"body":"Making a list of tracked files for a refresh operation can take a long\r\ntime on large repos on remote paths.  Check file under top directory\r\nindivitually."},"comment":{"url":"https://api.github.com/repos/magit/magit/issues/comments/159062605","html_url":"https://github.com/magit/magit/pull/2400#issuecomment-159062605","issue_url":"https://api.github.com/repos/magit/magit/issues/2400","id":159062605,"user":{"login":"ewstearns","id":752459,"avatar_url":"https://avatars.githubusercontent.com/u/752459?v=3","gravatar_id":"","url":"https://api.github.com/users/ewstearns","html_url":"https://github.com/ewstearns","followers_url":"https://api.github.com/users/ewstearns/followers","following_url":"https://api.github.com/users/ewstearns/following{/other_user}","gists_url":"https://api.github.com/users/ewstearns/gists{/gist_id}","starred_url":"https://api.github.com/users/ewstearns/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ewstearns/subscriptions","organizations_url":"https://api.github.com/users/ewstearns/orgs","repos_url":"https://api.github.com/users/ewstearns/repos","events_url":"https://api.github.com/users/ewstearns/events{/privacy}","received_events_url":"https://api.github.com/users/ewstearns/received_events","type":"User","site_admin":false},"created_at":"2015-11-23T21:00:08Z","updated_at":"2015-11-23T21:00:08Z","body":"As an experiment, I tried to call `git ls-tree -r -z --name-only HEAD` without `magit` but using TRAMP. My methodology is `M-! (shell-command)` from a `dired` buffer open to the top directory of my repo (`/scp:[user]@[machine]:/home/[user]/[repo path]`. This took even longer than `magit-refresh`, closer to 60 seconds rather than 15 seconds. This points further to TRAMP as the cause of my slow down.\r\n\r\nI also tried calling `M-x shell` while in the TRAMP `dired` buffer and tried the same command. This took significantly longer than the `shell-command` method and eventually there was a warning about exceeding the undo limit for the `*shell*` buffer.\r\n\r\nI tried calling `time git ls-tree -r -z --name-only HEAD` in a ssh connection on a regular termal and it took 0.039 seconds, so git itself is very fast.\r\n\r\nI cloned my repo on to my local machine and did the same tests. `magit-refresh` took about a third of the time, still a significant wait but more usable.\r\n\r\nThere are >18000 files in my repo.\r\n\r\nMy emacs version is `GNU Emacs 25.0.50.1 (x86_64-apple-darwin15.0.0, NS appkit-1404.13 Version 10.11.1 (Build 15B42)) of 2015-11-11`, a couple weeks off the head of the development repo installed using `brew`.\r\n\r\nA possible compromise is to run the command `git ls-tree -r  --name-only HEAD | wc -l` to get a count of the files in the repo and if it is greater than 5000 (or a configurable value) use my method of checking each file individually instead of getting the whole list."}}