{"action":"opened","issue":{"url":"https://api.github.com/repos/golang/go/issues/12979","labels_url":"https://api.github.com/repos/golang/go/issues/12979/labels{/name}","comments_url":"https://api.github.com/repos/golang/go/issues/12979/comments","events_url":"https://api.github.com/repos/golang/go/issues/12979/events","html_url":"https://github.com/golang/go/issues/12979","id":112033406,"number":12979,"title":"x/build: darwin builders occasionally failing","user":{"login":"bradfitz","id":2621,"avatar_url":"https://avatars.githubusercontent.com/u/2621?v=3","gravatar_id":"","url":"https://api.github.com/users/bradfitz","html_url":"https://github.com/bradfitz","followers_url":"https://api.github.com/users/bradfitz/followers","following_url":"https://api.github.com/users/bradfitz/following{/other_user}","gists_url":"https://api.github.com/users/bradfitz/gists{/gist_id}","starred_url":"https://api.github.com/users/bradfitz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bradfitz/subscriptions","organizations_url":"https://api.github.com/users/bradfitz/orgs","repos_url":"https://api.github.com/users/bradfitz/repos","events_url":"https://api.github.com/users/bradfitz/events{/privacy}","received_events_url":"https://api.github.com/users/bradfitz/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/golang/go/labels/Builders","name":"Builders","color":"ededed"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-10-18T18:20:12Z","updated_at":"2015-10-18T18:20:12Z","closed_at":null,"body":"The mac builders are occasionally failing with:\r\n\r\n```\r\n--- FAIL: TestCgoCrashHandler (0.03s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestCgoSignalDeadlock (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestCgoTraceback (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestCgoCallbackGC (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestCgoExternalThreadPanic (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestCgoExternalThreadSIGPROF (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestCgoExternalThreadSignal (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestCrashHandler (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestSimpleDeadlock (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestInitDeadlock (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestLockedDeadlock (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestLockedDeadlock2 (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestGoexitDeadlock (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestStackOverflow (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestThreadExhaustion (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestRecursivePanic (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestGoexitCrash (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestGoNil (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestMainGoroutineId (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestNoHelperGoroutines (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestBreakpoint (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestGoexitInPanic (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestPanicAfterGoexit (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestRecoveredPanicAfterGoexit (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestNetpollDeadlock (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestCrashDumpsAllThreads (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestGcSys (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestGCFairness (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\n--- FAIL: TestLargeStringConcat (0.00s)\r\n\tcrash_test.go:110: Stale runtime.a. Run 'go install runtime'.\r\nFAIL\r\n```\r\n\r\nThis seems to be an issue related to the system time between the 4+ builders, and/or the time granularity in our .tar.gz snapshots when we shard tests between machines. (We build on one, snapshot it to a tar.gz, and then untar on N other machines....)\r\n\r\nIt appears that on a machine later running the tests from the snapshot, it thinks the runtime is stale when it actually isn't.\r\n\r\nI haven't debugged yet.\r\n\r\n/cc @adg @crawshaw "}}