{"action":"created","issue":{"url":"https://api.github.com/repos/grobian/carbonwriter/issues/1","labels_url":"https://api.github.com/repos/grobian/carbonwriter/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/grobian/carbonwriter/issues/1/comments","events_url":"https://api.github.com/repos/grobian/carbonwriter/issues/1/events","html_url":"https://github.com/grobian/carbonwriter/issues/1","id":77947162,"number":1,"title":"Under heavy load crashes","user":{"login":"JoeOrtiz","id":7970108,"avatar_url":"https://avatars.githubusercontent.com/u/7970108?v=3","gravatar_id":"","url":"https://api.github.com/users/JoeOrtiz","html_url":"https://github.com/JoeOrtiz","followers_url":"https://api.github.com/users/JoeOrtiz/followers","following_url":"https://api.github.com/users/JoeOrtiz/following{/other_user}","gists_url":"https://api.github.com/users/JoeOrtiz/gists{/gist_id}","starred_url":"https://api.github.com/users/JoeOrtiz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JoeOrtiz/subscriptions","organizations_url":"https://api.github.com/users/JoeOrtiz/orgs","repos_url":"https://api.github.com/users/JoeOrtiz/repos","events_url":"https://api.github.com/users/JoeOrtiz/events{/privacy}","received_events_url":"https://api.github.com/users/JoeOrtiz/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2015-05-19T05:54:44Z","updated_at":"2015-05-24T07:26:23Z","closed_at":null,"body":"I've been running carbonwriter pretty good for a few months, but it appears after heavy load, it crashes.\r\n\r\nCouldn't get much but crashes immediately:\r\n\r\n```\r\nMay 19 05:50:52 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: 2015/05/19 05:50:52 set GOMAXPROCS=16\r\nMay 19 05:50:52 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: 2015/05/19 05:50:52 listening on :2003, statistics via :8080\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: 2015/05/19 05:50:53 creating docker.Repository.i-40300189.registrynginx.memory.active: /storage/docker/Repository/i-40300189/registrynginx/memory/active.wsp, retention: 5s:14d,1m:21d,15m:1y (section\r\n docker), aggregationMethod: average, xFilesFactor: 0.300000 (section default_average)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: 2015/05/19 05:50:53 creating docker.DPL-Prod.t4.i-49fdca80.carbonrelay.memory.usage: /storage/docker/DPL-Prod/t4/i-49fdca80/carbonrelay/memory/usage.wsp, retention: 5s:14d,1m:21d,15m:1y (section doc\r\nker), aggregationMethod: average, xFilesFactor: 0.300000 (section default_average)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: panic: Failed to read baseInterval\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: goroutine 56 [running]:\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: runtime.panic(0x69a160, 0xc2081c6100)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/runtime/panic.c:279 +0xf5\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: github.com/grobian/go-whisper.(*Whisper).getBaseInterval(0xc2081da540, 0x7ff9a0332bf0, 0xc)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /gopath/src/github.com/grobian/go-whisper/whisper.go:506 +0x9c\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: github.com/grobian/go-whisper.(*Whisper).getPointOffset(0xc2081da540, 0x555acf1c, 0x7ff9a0332bf0, 0xc)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /gopath/src/github.com/grobian/go-whisper/whisper.go:496 +0x31\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: github.com/grobian/go-whisper.(*Whisper).Update(0xc2081da540, 0x402e000000000000, 0x555acf24, 0x0, 0x0)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /gopath/src/github.com/grobian/go-whisper/whisper.go:360 +0x219\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: main.handleConnection(0x7ff9a072cde0, 0xc2080ce060, 0xc208044680, 0xa, 0x10, 0xc20803f700, 0x4, 0x4)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /carbonwriter/main.go:166 +0x1483\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: created by main.listenAndServe\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /carbonwriter/main.go:184 +0x387\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: goroutine 16 [IO wait]:\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: net.runtime_pollWait(0x7ff9a072ccf8, 0x72, 0x0)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/runtime/netpoll.goc:146 +0x66\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: net.(*pollDesc).Wait(0xc208029e20, 0x72, 0x0, 0x0)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/net/fd_poll_runtime.go:84 +0x46\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: net.(*pollDesc).WaitRead(0xc208029e20, 0x0, 0x0)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/net/fd_poll_runtime.go:89 +0x42\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: net.(*netFD).accept(0xc208029dc0, 0x8441e8, 0x0, 0x7ff9a07232b8, 0xb)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/net/fd_unix.go:419 +0x343\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: net.(*TCPListener).AcceptTCP(0xc2080381c0, 0x7ff9a0584a98, 0x0, 0x0)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/net/tcpsock_posix.go:234 +0x5d\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: net/http.tcpKeepAliveListener.Accept(0xc2080381c0, 0x0, 0x0, 0x0, 0x0)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/net/http/server.go:1947 +0x4b\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: net/http.(*Server).Serve(0xc208004ae0, 0x7ff9a072cda8, 0xc2080381c0, 0x0, 0x0)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/net/http/server.go:1698 +0x91\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: net/http.(*Server).ListenAndServe(0xc208004ae0, 0x0, 0x0)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/net/http/server.go:1688 +0x14d\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: net/http.ListenAndServe(0xc2080b0505, 0x5, 0x0, 0x0, 0x0, 0x0)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/net/http/server.go:1778 +0x79\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: main.main()\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /carbonwriter/main.go:401 +0x17b8\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: goroutine 19 [finalizer wait]:\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: runtime.park(0x419380, 0x987cc8, 0x985729)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/runtime/proc.c:1369 +0x89\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: runtime.parkunlock(0x987cc8, 0x985729)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/runtime/proc.c:1385 +0x3b\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: runfinq()\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/runtime/mgc0.c:2644 +0xcf\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: runtime.goexit()\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/runtime/proc.c:1445\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: goroutine 21 [IO wait]:\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: net.runtime_pollWait(0x7ff9a072cc48, 0x72, 0x0)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/runtime/netpoll.goc:146 +0x66\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: net.(*pollDesc).Wait(0xc2080c8060, 0x72, 0x0, 0x0)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/net/fd_poll_runtime.go:84 +0x46\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: net.(*pollDesc).WaitRead(0xc2080c8060, 0x0, 0x0)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/net/fd_poll_runtime.go:89 +0x42\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: net.(*netFD).accept(0xc2080c8000, 0x8441e8, 0x0, 0x7ff9a07232b8, 0xb)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/net/fd_unix.go:419 +0x343\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: net.(*TCPListener).AcceptTCP(0xc2080ce000, 0x40, 0x0, 0x0)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/net/tcpsock_posix.go:234 +0x5d\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: net.(*TCPListener).Accept(0xc2080ce000, 0x0, 0x0, 0x0, 0x0)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /usr/lib/go/src/pkg/net/tcpsock_posix.go:244 +0x4b\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: main.listenAndServe(0xc2080b0500, 0x5, 0xc208044680, 0xa, 0x10, 0xc20803f700, 0x4, 0x4)\r\nMay 19 05:50:53 ip-10-170-2-90.us-west-2.compute.internal bash[6414]: /carbonwriter/main.go:179 +0x209\r\n```"},"comment":{"url":"https://api.github.com/repos/grobian/carbonwriter/issues/comments/104988055","html_url":"https://github.com/grobian/carbonwriter/issues/1#issuecomment-104988055","issue_url":"https://api.github.com/repos/grobian/carbonwriter/issues/1","id":104988055,"user":{"login":"grobian","id":667105,"avatar_url":"https://avatars.githubusercontent.com/u/667105?v=3","gravatar_id":"","url":"https://api.github.com/users/grobian","html_url":"https://github.com/grobian","followers_url":"https://api.github.com/users/grobian/followers","following_url":"https://api.github.com/users/grobian/following{/other_user}","gists_url":"https://api.github.com/users/grobian/gists{/gist_id}","starred_url":"https://api.github.com/users/grobian/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/grobian/subscriptions","organizations_url":"https://api.github.com/users/grobian/orgs","repos_url":"https://api.github.com/users/grobian/repos","events_url":"https://api.github.com/users/grobian/events{/privacy}","received_events_url":"https://api.github.com/users/grobian/received_events","type":"User","site_admin":false},"created_at":"2015-05-24T07:26:23Z","updated_at":"2015-05-24T07:26:23Z","body":"That's interesting, thanks for the analysis!  I think we should just make sure we don't panic, and either refuse to update, or discard the whole (corrupt) file."}}