{"action":"created","issue":{"url":"https://api.github.com/repos/golang/go/issues/12967","labels_url":"https://api.github.com/repos/golang/go/issues/12967/labels{/name}","comments_url":"https://api.github.com/repos/golang/go/issues/12967/comments","events_url":"https://api.github.com/repos/golang/go/issues/12967/events","html_url":"https://github.com/golang/go/issues/12967","id":111938102,"number":12967,"title":"runtime: shrinkstack during mark termination significantly increases GC STW time","user":{"login":"rhysh","id":230685,"avatar_url":"https://avatars.githubusercontent.com/u/230685?v=3","gravatar_id":"","url":"https://api.github.com/users/rhysh","html_url":"https://github.com/rhysh","followers_url":"https://api.github.com/users/rhysh/followers","following_url":"https://api.github.com/users/rhysh/following{/other_user}","gists_url":"https://api.github.com/users/rhysh/gists{/gist_id}","starred_url":"https://api.github.com/users/rhysh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rhysh/subscriptions","organizations_url":"https://api.github.com/users/rhysh/orgs","repos_url":"https://api.github.com/users/rhysh/repos","events_url":"https://api.github.com/users/rhysh/events{/privacy}","received_events_url":"https://api.github.com/users/rhysh/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/golang/go/milestones/6","html_url":"https://github.com/golang/go/milestones/Unplanned","labels_url":"https://api.github.com/repos/golang/go/milestones/6/labels","id":1055141,"number":6,"title":"Unplanned","description":"No plan to fix in any specific release.\r\n","creator":{"login":"rsc","id":104030,"avatar_url":"https://avatars.githubusercontent.com/u/104030?v=3","gravatar_id":"","url":"https://api.github.com/users/rsc","html_url":"https://github.com/rsc","followers_url":"https://api.github.com/users/rsc/followers","following_url":"https://api.github.com/users/rsc/following{/other_user}","gists_url":"https://api.github.com/users/rsc/gists{/gist_id}","starred_url":"https://api.github.com/users/rsc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rsc/subscriptions","organizations_url":"https://api.github.com/users/rsc/orgs","repos_url":"https://api.github.com/users/rsc/repos","events_url":"https://api.github.com/users/rsc/events{/privacy}","received_events_url":"https://api.github.com/users/rsc/received_events","type":"User","site_admin":false},"open_issues":1125,"closed_issues":154,"state":"open","created_at":"2015-04-06T20:47:19Z","updated_at":"2015-11-08T23:30:26Z","due_on":"2099-12-31T05:00:00Z","closed_at":null},"comments":8,"created_at":"2015-10-17T02:02:53Z","updated_at":"2015-11-08T23:41:39Z","closed_at":null,"body":"I have a process that experiences significant STW times during the mark termination phase of garbage collection. When it has around 100,000 goroutines and a live heap of around 500MB, its STW pauses take tens of milliseconds. The process runs on linux/amd64 with GOMAXPROCS=18, with a recent version of Go tip (30b9663).\r\n\r\nI profiled its CPU usage with perf_events, looking specifically at samples with runtime.gcMark on the stack. 7.5% of samples include runtime.gcMark -> runtime.freeStackSpans (non-parallelized work, discussed in #11465). 90% of samples include runtime.gcMark -> runtime.parfordo -> runtime.markroot, and 75% of samples include runtime.gcMark -> runtime.parfordo -> runtime.markroot -> runtime.shrinkstack.\r\n\r\nThe work for stack shrinking is split 18 ways in my case, but at 75% of CPU cycles it's still a huge contributor to mark termination pauses, and the pauses my program sees are well beyond the 10ms goal with only a modest heap size.\r\n\r\nDoes stack shrinking need to take place while the world is stopped?\r\n\r\n```\r\n$ go version\r\ngo version devel +30b9663 Tue Oct 13 21:06:58 2015 +0000 linux/amd64\r\n```\r\n\r\n@aclements "},"comment":{"url":"https://api.github.com/repos/golang/go/issues/comments/154887059","html_url":"https://github.com/golang/go/issues/12967#issuecomment-154887059","issue_url":"https://api.github.com/repos/golang/go/issues/12967","id":154887059,"user":{"login":"randall77","id":6889504,"avatar_url":"https://avatars.githubusercontent.com/u/6889504?v=3","gravatar_id":"","url":"https://api.github.com/users/randall77","html_url":"https://github.com/randall77","followers_url":"https://api.github.com/users/randall77/followers","following_url":"https://api.github.com/users/randall77/following{/other_user}","gists_url":"https://api.github.com/users/randall77/gists{/gist_id}","starred_url":"https://api.github.com/users/randall77/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/randall77/subscriptions","organizations_url":"https://api.github.com/users/randall77/orgs","repos_url":"https://api.github.com/users/randall77/repos","events_url":"https://api.github.com/users/randall77/events{/privacy}","received_events_url":"https://api.github.com/users/randall77/received_events","type":"User","site_admin":false},"created_at":"2015-11-08T23:41:39Z","updated_at":"2015-11-08T23:41:39Z","body":"So as I see it there are a couple of options to enable shrinking outside a STW phase.\r\n\r\n1) Have receivers allocate space for the received data on the heap.\r\n2) Have unbuffered channels allocate one slot which is used to pass the sent value.\r\n3) Shrinker must grab the channel lock.\r\n4) Sender & shrinker must grab a lock on the stack (or G, as Austin proposes)\r\n5) Shrink stacks in place.\r\n\r\n1 means allocation per channel receive.\r\n2 means some extra memory + some extra synchronization.\r\n3 might be difficult for select statements, we'd need to grab all the chan locks.\r\n4 means a second lock per send.\r\n5 means possibly greater fragmentation.\r\n\r\nAs others have also pointed out, I don't think preventing shrinking while in receive is a viable option.\r\nAny others?"}}