{"action":"created","issue":{"url":"https://api.github.com/repos/gdamore/mangos/issues/77","labels_url":"https://api.github.com/repos/gdamore/mangos/issues/77/labels{/name}","comments_url":"https://api.github.com/repos/gdamore/mangos/issues/77/comments","events_url":"https://api.github.com/repos/gdamore/mangos/issues/77/events","html_url":"https://github.com/gdamore/mangos/issues/77","id":56915406,"number":77,"title":"shutdown test (in goq test suite) fails under latest mangos","user":{"login":"glycerine","id":445247,"avatar_url":"https://avatars.githubusercontent.com/u/445247?v=3","gravatar_id":"","url":"https://api.github.com/users/glycerine","html_url":"https://github.com/glycerine","followers_url":"https://api.github.com/users/glycerine/followers","following_url":"https://api.github.com/users/glycerine/following{/other_user}","gists_url":"https://api.github.com/users/glycerine/gists{/gist_id}","starred_url":"https://api.github.com/users/glycerine/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/glycerine/subscriptions","organizations_url":"https://api.github.com/users/glycerine/orgs","repos_url":"https://api.github.com/users/glycerine/repos","events_url":"https://api.github.com/users/glycerine/events{/privacy}","received_events_url":"https://api.github.com/users/glycerine/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-02-07T15:51:58Z","updated_at":"2015-02-08T05:34:54Z","closed_at":null,"body":"Garrett, Thanks for the recent work on Send and Connect (#68), but I think it broke some other things. Running the Goq test suite I'm now getting this failure, before I even comment back in the tests that mangos was failing before. \r\n\r\n\r\n~~~\r\ngo/src/github.com/glycerine/goq$ go test -v\r\n...\r\n... (passed tests elided) ...\r\n...\r\n=== RUN TestSubmitShutdownToRemoteJobServ\r\n\r\n  remotely, over nanomsg, goq should be able to submit a shutdown job to the server \r\n    and then server process should shut itself down cleanly \r\n took 0 10msec sleeps for address 'tcp://10.0.0.6:2776' to become available.\r\n\r\n[pid 32006] spawned a new external JobServ with pid 32038\r\n\r\njobserv with expected pid 32038 was *found* in /proc after 0 waits of 50msec\r\n2015-02-07 07:40:56.125 -0800 PST **** [jobserver pid 32038] listening for jobs on 'tcp://10.0.0.6:2776', output to 'o'. GOQ_HOME is '/home/jaten/dev/cnet/go/src/github.com/glycerine/goq/tempgoqtestdir195059649'.\r\n\r\nsent shutdown request\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\njobserv at pid 32038 is still alive...\r\nfailed to exit: dumping the goroutines on the server to see where we are stuck.\r\nSIGQUIT: quit\r\nPC=0x478433\r\n\r\ngoroutine 5 [syscall]:\r\nruntime.notetsleepg(0xd00698, 0x3b9a8a01, 0x1)\r\n\t/home/jaten/pkg/go1.4/go/src/runtime/lock_futex.go:201 +0x52 fp=0xc20801a768 sp=0xc20801a740\r\nruntime.timerproc()\r\n\t/home/jaten/pkg/go1.4/go/src/runtime/time.go:207 +0xfa fp=0xc20801a7e0 sp=0xc20801a768\r\nruntime.goexit()\r\n\t/home/jaten/pkg/go1.4/go/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc20801a7e8 sp=0xc20801a7e0\r\ncreated by runtime.addtimerLocked\r\n\t/home/jaten/pkg/go1.4/go/src/runtime/time.go:113 +0x1ba\r\n\r\ngoroutine 1 [chan receive]:\r\nmain.main()\r\n\t/home/jaten/dev/cnet/go/src/github.com/glycerine/goq/main.go:145 +0x1333\r\n\r\ngoroutine 8 [IO wait]:\r\nnet.(*pollDesc).Wait(0xc2080c6760, 0x72, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:84 +0x47\r\nnet.(*pollDesc).WaitRead(0xc2080c6760, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:89 +0x43\r\nnet.(*netFD).accept(0xc2080c6700, 0x0, 0x7f8f823ded20, 0xc2080f60d8)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_unix.go:419 +0x40b\r\nnet.(*TCPListener).AcceptTCP(0xc2080320a8, 0xc208034eb0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/tcpsock_posix.go:234 +0x4e\r\ngithub.com/gdamore/mangos/transport/tcp.(*listener).Accept(0xc2080d9590, 0x0, 0x0, 0x0, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/transport/tcp/tcp.go:116 +0xa2\r\ngithub.com/gdamore/mangos.(*listener).serve(0xc2080d9530)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:573 +0x7e\r\ncreated by github.com/gdamore/mangos.(*listener).Listen\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:595 +0x225\r\n\r\ngoroutine 9 [IO wait]:\r\nnet.(*pollDesc).Wait(0xc2080c68b0, 0x72, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:84 +0x47\r\nnet.(*pollDesc).WaitRead(0xc2080c68b0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:89 +0x43\r\nnet.(*netFD).accept(0xc2080c6850, 0x0, 0x7f8f823ded20, 0xc20802da00)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_unix.go:419 +0x40b\r\nnet.(*TCPListener).AcceptTCP(0xc2080320d8, 0xc2080e7d70, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/tcpsock_posix.go:234 +0x4e\r\nnet.(*TCPListener).Accept(0xc2080320d8, 0x0, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/tcpsock_posix.go:244 +0x4c\r\nnet/http.(*Server).Serve(0xc20805e160, 0x7f8f823dfb30, 0xc2080320d8, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/http/server.go:1728 +0x92\r\ngithub.com/glycerine/go-tigertonic.(*Server).Serve(0xc20805e160, 0x7f8f823dfb30, 0xc2080320d8, 0x0, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/glycerine/go-tigertonic/server.go:160 +0x1ae\r\ngithub.com/glycerine/go-tigertonic.(*Server).ListenAndServe(0xc20805e160, 0x0, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/glycerine/go-tigertonic/server.go:144 +0x209\r\nmain.func·015()\r\n\t/home/jaten/dev/cnet/go/src/github.com/glycerine/goq/web.go:50 +0x33\r\ncreated by main.(*WebServer).Start\r\n\t/home/jaten/dev/cnet/go/src/github.com/glycerine/goq/web.go:55 +0xab\r\n\r\ngoroutine 17 [syscall, locked to thread]:\r\nruntime.goexit()\r\n\t/home/jaten/pkg/go1.4/go/src/runtime/asm_amd64.s:2232 +0x1\r\n\r\ngoroutine 13 [IO wait]:\r\nnet.(*pollDesc).Wait(0xc2080c6a00, 0x72, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:84 +0x47\r\nnet.(*pollDesc).WaitRead(0xc2080c6a00, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:89 +0x43\r\nnet.(*netFD).Read(0xc2080c69a0, 0xc20802dc58, 0x8, 0x8, 0x0, 0x7f8f823ded20, 0xc20802dc60)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_unix.go:242 +0x40f\r\nnet.(*conn).Read(0xc208032150, 0xc20802dc58, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/net.go:121 +0xdc\r\nio.ReadAtLeast(0x7f8f823e0d58, 0xc208032150, 0xc20802dc58, 0x8, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/io/io.go:298 +0xf1\r\nio.ReadFull(0x7f8f823e0d58, 0xc208032150, 0xc20802dc58, 0x8, 0x8, 0x7f8f823d10f0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/io/io.go:316 +0x6d\r\nencoding/binary.Read(0x7f8f823e0d58, 0xc208032150, 0x7f8f823df6d8, 0xd093c8, 0x737000, 0xc20802dc50, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/encoding/binary/binary.go:148 +0x12d\r\ngithub.com/gdamore/mangos.(*conn).Recv(0xc20803dec0, 0x0, 0x0, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/conn.go:55 +0x1a7\r\ngithub.com/gdamore/mangos.(*pipe).RecvMsg(0xc208034cd0, 0x9)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/pipe.go:111 +0x40\r\ngithub.com/gdamore/mangos/protocol/pull.(*pull).receiver(0xc20800b660, 0x7f8f823e0f50, 0xc208034cd0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/pull/pull.go:35 +0x3c\r\ncreated by github.com/gdamore/mangos/protocol/pull.(*pull).AddEndpoint\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/pull/pull.go:65 +0x46\r\n\r\ngoroutine 11 [select]:\r\nmain.func·003()\r\n\t/home/jaten/dev/cnet/go/src/github.com/glycerine/goq/goq.go:805 +0x503a\r\ncreated by main.(*JobServ).Start\r\n\t/home/jaten/dev/cnet/go/src/github.com/glycerine/goq/goq.go:1056 +0x8c\r\n\r\ngoroutine 12 [select]:\r\ngithub.com/gdamore/mangos.(*socket).RecvMsg(0xc208021380, 0x8c6810, 0x0, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:238 +0x29c\r\ngithub.com/gdamore/mangos/compat.(*Socket).Recv(0xc2080d9410, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/compat/compat.go:223 +0x1b2\r\nmain.recvZjob(0xc2080d9410, 0xc2080505a0, 0x0, 0x0, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/glycerine/goq/goq.go:1690 +0x100\r\nmain.func·008()\r\n\t/home/jaten/dev/cnet/go/src/github.com/glycerine/goq/goq.go:1508 +0x2c7\r\ncreated by main.(*JobServ).ListenForJobs\r\n\t/home/jaten/dev/cnet/go/src/github.com/glycerine/goq/goq.go:1646 +0xd2\r\n\r\ngoroutine 14 [IO wait]:\r\nnet.(*pollDesc).Wait(0xc2080c6ae0, 0x72, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:84 +0x47\r\nnet.(*pollDesc).WaitRead(0xc2080c6ae0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:89 +0x43\r\nnet.(*netFD).Read(0xc2080c6a80, 0xc20802dfa8, 0x8, 0x8, 0x0, 0x7f8f823ded20, 0xc20802dfb0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_unix.go:242 +0x40f\r\nnet.(*conn).Read(0xc208032160, 0xc20802dfa8, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/net.go:121 +0xdc\r\nio.ReadAtLeast(0x7f8f823e0d58, 0xc208032160, 0xc20802dfa8, 0x8, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/io/io.go:298 +0xf1\r\nio.ReadFull(0x7f8f823e0d58, 0xc208032160, 0xc20802dfa8, 0x8, 0x8, 0x7f8f823d10f0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/io/io.go:316 +0x6d\r\nencoding/binary.Read(0x7f8f823e0d58, 0xc208032160, 0x7f8f823df6d8, 0xd093c8, 0x737000, 0xc20802dfa0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/encoding/binary/binary.go:148 +0x12d\r\ngithub.com/gdamore/mangos.(*conn).Recv(0xc20803df00, 0x0, 0x0, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/conn.go:55 +0x1a7\r\ngithub.com/gdamore/mangos.(*pipe).RecvMsg(0xc208034d20, 0x9)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/pipe.go:111 +0x40\r\ngithub.com/gdamore/mangos/protocol/pull.(*pull).receiver(0xc20800b660, 0x7f8f823e0f50, 0xc208034d20)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/pull/pull.go:35 +0x3c\r\ncreated by github.com/gdamore/mangos/protocol/pull.(*pull).AddEndpoint\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/pull/pull.go:65 +0x46\r\n\r\ngoroutine 15 [IO wait]:\r\nnet.(*pollDesc).Wait(0xc2080c6b50, 0x72, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:84 +0x47\r\nnet.(*pollDesc).WaitRead(0xc2080c6b50, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:89 +0x43\r\nnet.(*netFD).Read(0xc2080c6af0, 0xc20802dfc8, 0x8, 0x8, 0x0, 0x7f8f823ded20, 0xc20802dfd0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_unix.go:242 +0x40f\r\nnet.(*conn).Read(0xc208032168, 0xc20802dfc8, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/net.go:121 +0xdc\r\nio.ReadAtLeast(0x7f8f823e0d58, 0xc208032168, 0xc20802dfc8, 0x8, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/io/io.go:298 +0xf1\r\nio.ReadFull(0x7f8f823e0d58, 0xc208032168, 0xc20802dfc8, 0x8, 0x8, 0xc20800b660, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/io/io.go:316 +0x6d\r\nencoding/binary.Read(0x7f8f823e0d58, 0xc208032168, 0x7f8f823df6d8, 0xd093c8, 0x737000, 0xc20802dfc0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/encoding/binary/binary.go:148 +0x12d\r\ngithub.com/gdamore/mangos.(*conn).Recv(0xc20803df40, 0x0, 0x0, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/conn.go:55 +0x1a7\r\ngithub.com/gdamore/mangos.(*pipe).RecvMsg(0xc208034d70, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/pipe.go:111 +0x40\r\ngithub.com/gdamore/mangos/protocol/pull.(*pull).receiver(0xc20800b660, 0x7f8f823e0f50, 0xc208034d70)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/pull/pull.go:35 +0x3c\r\ncreated by github.com/gdamore/mangos/protocol/pull.(*pull).AddEndpoint\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/pull/pull.go:65 +0x46\r\n\r\ngoroutine 16 [IO wait]:\r\nnet.(*pollDesc).Wait(0xc2080c6bc0, 0x72, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:84 +0x47\r\nnet.(*pollDesc).WaitRead(0xc2080c6bc0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:89 +0x43\r\nnet.(*netFD).Read(0xc2080c6b60, 0xc20802dfe8, 0x8, 0x8, 0x0, 0x7f8f823ded20, 0xc20802dff0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_unix.go:242 +0x40f\r\nnet.(*conn).Read(0xc208032170, 0xc20802dfe8, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/net.go:121 +0xdc\r\nio.ReadAtLeast(0x7f8f823e0d58, 0xc208032170, 0xc20802dfe8, 0x8, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/io/io.go:298 +0xf1\r\nio.ReadFull(0x7f8f823e0d58, 0xc208032170, 0xc20802dfe8, 0x8, 0x8, 0xd00800, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/io/io.go:316 +0x6d\r\nencoding/binary.Read(0x7f8f823e0d58, 0xc208032170, 0x7f8f823df6d8, 0xd093c8, 0x737000, 0xc20802dfe0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/encoding/binary/binary.go:148 +0x12d\r\ngithub.com/gdamore/mangos.(*conn).Recv(0xc20803df80, 0x0, 0x0, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/conn.go:55 +0x1a7\r\ngithub.com/gdamore/mangos.(*pipe).RecvMsg(0xc208034dc0, 0x8b8ea0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/pipe.go:111 +0x40\r\ngithub.com/gdamore/mangos/protocol/pull.(*pull).receiver(0xc20800b660, 0x7f8f823e0f50, 0xc208034dc0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/pull/pull.go:35 +0x3c\r\ncreated by github.com/gdamore/mangos/protocol/pull.(*pull).AddEndpoint\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/pull/pull.go:65 +0x46\r\n\r\ngoroutine 18 [IO wait]:\r\nnet.(*pollDesc).Wait(0xc2080c6c30, 0x72, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:84 +0x47\r\nnet.(*pollDesc).WaitRead(0xc2080c6c30, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:89 +0x43\r\nnet.(*netFD).Read(0xc2080c6bd0, 0xc2080f6008, 0x8, 0x8, 0x0, 0x7f8f823ded20, 0xc2080f6010)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_unix.go:242 +0x40f\r\nnet.(*conn).Read(0xc208032178, 0xc2080f6008, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/net.go:121 +0xdc\r\nio.ReadAtLeast(0x7f8f823e0d58, 0xc208032178, 0xc2080f6008, 0x8, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/io/io.go:298 +0xf1\r\nio.ReadFull(0x7f8f823e0d58, 0xc208032178, 0xc2080f6008, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/io/io.go:316 +0x6d\r\nencoding/binary.Read(0x7f8f823e0d58, 0xc208032178, 0x7f8f823df6d8, 0xd093c8, 0x737000, 0xc2080f6000, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/encoding/binary/binary.go:148 +0x12d\r\ngithub.com/gdamore/mangos.(*conn).Recv(0xc20803dfc0, 0x0, 0x0, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/conn.go:55 +0x1a7\r\ngithub.com/gdamore/mangos.(*pipe).RecvMsg(0xc208034e10, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/pipe.go:111 +0x40\r\ngithub.com/gdamore/mangos/protocol/pull.(*pull).receiver(0xc20800b660, 0x7f8f823e0f50, 0xc208034e10)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/pull/pull.go:35 +0x3c\r\ncreated by github.com/gdamore/mangos/protocol/pull.(*pull).AddEndpoint\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/pull/pull.go:65 +0x46\r\n\r\ngoroutine 19 [IO wait]:\r\nnet.(*pollDesc).Wait(0xc2080c6ca0, 0x72, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:84 +0x47\r\nnet.(*pollDesc).WaitRead(0xc2080c6ca0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:89 +0x43\r\nnet.(*netFD).Read(0xc2080c6c40, 0xc2080f6028, 0x8, 0x8, 0x0, 0x7f8f823ded20, 0xc2080f6030)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_unix.go:242 +0x40f\r\nnet.(*conn).Read(0xc208032180, 0xc2080f6028, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/net.go:121 +0xdc\r\nio.ReadAtLeast(0x7f8f823e0d58, 0xc208032180, 0xc2080f6028, 0x8, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/io/io.go:298 +0xf1\r\nio.ReadFull(0x7f8f823e0d58, 0xc208032180, 0xc2080f6028, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/io/io.go:316 +0x6d\r\nencoding/binary.Read(0x7f8f823e0d58, 0xc208032180, 0x7f8f823df6d8, 0xd093c8, 0x737000, 0xc2080f6020, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/encoding/binary/binary.go:148 +0x12d\r\ngithub.com/gdamore/mangos.(*conn).Recv(0xc20807a040, 0x0, 0x0, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/conn.go:55 +0x1a7\r\ngithub.com/gdamore/mangos.(*pipe).RecvMsg(0xc208034e60, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/pipe.go:111 +0x40\r\ngithub.com/gdamore/mangos/protocol/pull.(*pull).receiver(0xc20800b660, 0x7f8f823e0f50, 0xc208034e60)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/pull/pull.go:35 +0x3c\r\ncreated by github.com/gdamore/mangos/protocol/pull.(*pull).AddEndpoint\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/pull/pull.go:65 +0x46\r\n\r\ngoroutine 20 [IO wait]:\r\nnet.(*pollDesc).Wait(0xc2080c6d10, 0x72, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:84 +0x47\r\nnet.(*pollDesc).WaitRead(0xc2080c6d10, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:89 +0x43\r\nnet.(*netFD).Read(0xc2080c6cb0, 0xc2080f6110, 0x8, 0x8, 0x0, 0x7f8f823ded20, 0xc2080f6118)\r\n\t/home/jaten/pkg/go1.4/go/src/net/fd_unix.go:242 +0x40f\r\nnet.(*conn).Read(0xc208032188, 0xc2080f6110, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/net/net.go:121 +0xdc\r\nio.ReadAtLeast(0x7f8f823e0d58, 0xc208032188, 0xc2080f6110, 0x8, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/io/io.go:298 +0xf1\r\nio.ReadFull(0x7f8f823e0d58, 0xc208032188, 0xc2080f6110, 0x8, 0x8, 0x0, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/io/io.go:316 +0x6d\r\nencoding/binary.Read(0x7f8f823e0d58, 0xc208032188, 0x7f8f823df6d8, 0xd093c8, 0x737000, 0xc2080f6108, 0x0, 0x0)\r\n\t/home/jaten/pkg/go1.4/go/src/encoding/binary/binary.go:148 +0x12d\r\ngithub.com/gdamore/mangos.(*conn).Recv(0xc20807a2c0, 0x0, 0x0, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/conn.go:55 +0x1a7\r\ngithub.com/gdamore/mangos.(*pipe).RecvMsg(0xc208034eb0, 0x0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/pipe.go:111 +0x40\r\ngithub.com/gdamore/mangos/protocol/pull.(*pull).receiver(0xc20800b660, 0x7f8f823e0f50, 0xc208034eb0)\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/pull/pull.go:35 +0x3c\r\ncreated by github.com/gdamore/mangos/protocol/pull.(*pull).AddEndpoint\r\n\t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/pull/pull.go:65 +0x46\r\n\r\nrax     0xfffffffffffffffc\r\nrbx     0x3b9a8a01\r\nrcx     0xffffffffffffffff\r\nrdx     0x0\r\nrdi     0xd00698\r\nrsi     0x0\r\nrbp     0xc20801a700\r\nrsp     0xc20801a6c0\r\nr8      0x0\r\nr9      0x0\r\nr10     0xc20801a6f8\r\nr11     0x216\r\nr12     0xffffffffffffffff\r\nr13     0xc208001b10\r\nr14     0xe225\r\nr15     0x10\r\nrip     0x478433\r\nrflags  0x216\r\ncs      0x33\r\nfs      0x0\r\ngs      0x0\r\n🔥\r\n\r\nErrors:\r\n\r\n  * /home/jaten/dev/cnet/go/src/github.com/glycerine/goq/fetch_test.go \r\n  Line 179: - jobserv with expected pid 32038 did not disappear from /proc after 3 timeouts \r\n  goroutine 96 [running]:\r\n  github.com/glycerine/goq.WaitForShutdownWithTimeout(0x7d26, 0xc208054000)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/glycerine/goq/fetch_test.go:179 +0x37c\r\n  github.com/glycerine/goq.func·033()\r\n  \t/home/jaten/dev/cnet/go/src/github.com/glycerine/goq/fetch_test.go:117 +0x44d\r\n  github.com/glycerine/goq.TestSubmitShutdownToRemoteJobServ(0xc208072480)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/glycerine/goq/fetch_test.go:126 +0x157\r\n  testing.tRunner(0xc208072480, 0xd7b1f8)\r\n  \t/home/jaten/pkg/go1.4/go/src/testing/testing.go:447 +0xbf\r\n  created by testing.RunTests\r\n  \t/home/jaten/pkg/go1.4/go/src/testing/testing.go:555 +0xa8b\r\n  \r\n  goroutine 1 [chan receive]:\r\n  testing.RunTests(0xa07940, 0xd7b120, 0x21, 0x21, 0xc2080de101)\r\n  \t/home/jaten/pkg/go1.4/go/src/testing/testing.go:556 +0xad6\r\n  testing.(*M).Run(0xc20802ecd0, 0xd9efa0)\r\n  \t/home/jaten/pkg/go1.4/go/src/testing/testing.go:485 +0x6c\r\n  main.main()\r\n  \tgithub.com/glycerine/goq/_test/_testmain.go:116 +0x1d5\r\n  \r\n  goroutine 81 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc20800b920, 0x7f4b2ce4b8d0, 0xc20802f270)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 76 [IO wait]:\r\n  net.(*pollDesc).Wait(0xc2080c6920, 0x72, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:84 +0x47\r\n  net.(*pollDesc).WaitRead(0xc2080c6920, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:89 +0x43\r\n  net.(*netFD).accept(0xc2080c68c0, 0x0, 0x7f4b2ce48248, 0xc20802b348)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_unix.go:419 +0x40b\r\n  net.(*TCPListener).AcceptTCP(0xc20802c1c8, 0xc20802f360, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/tcpsock_posix.go:234 +0x4e\r\n  github.com/gdamore/mangos/transport/tcp.(*listener).Accept(0xc20803d830, 0x0, 0x0, 0x0, 0x0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/transport/tcp/tcp.go:116 +0xa2\r\n  github.com/gdamore/mangos.(*listener).serve(0xc20803d770)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:573 +0x7e\r\n  created by github.com/gdamore/mangos.(*listener).Listen\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:595 +0x225\r\n  \r\n  goroutine 17 [syscall, locked to thread]:\r\n  runtime.goexit()\r\n  \t/home/jaten/pkg/go1.4/go/src/runtime/asm_amd64.s:2232 +0x1\r\n  \r\n  goroutine 73 [select]:\r\n  github.com/gdamore/mangos.(*dialer).dialer(0xc2080ddb00)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:537 +0x2dd\r\n  created by github.com/gdamore/mangos.(*dialer).Dial\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:488 +0x12c\r\n  \r\n  goroutine 99 [IO wait]:\r\n  net.(*pollDesc).Wait(0xc2080c6370, 0x72, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:84 +0x47\r\n  net.(*pollDesc).WaitRead(0xc2080c6370, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:89 +0x43\r\n  net.(*netFD).accept(0xc2080c6310, 0x0, 0x7f4b2ce48248, 0xc20816a640)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_unix.go:419 +0x40b\r\n  net.(*TCPListener).AcceptTCP(0xc20802c0f0, 0x672f676b702f6e65, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/tcpsock_posix.go:234 +0x4e\r\n  github.com/gdamore/mangos/transport/tcp.(*listener).Accept(0xc2080f87b0, 0x0, 0x0, 0x0, 0x0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/transport/tcp/tcp.go:116 +0xa2\r\n  github.com/gdamore/mangos.(*listener).serve(0xc2080f8750)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:573 +0x7e\r\n  created by github.com/gdamore/mangos.(*listener).Listen\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:595 +0x225\r\n  \r\n  goroutine 13 [IO wait]:\r\n  net.(*pollDesc).Wait(0xc208010ca0, 0x72, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:84 +0x47\r\n  net.(*pollDesc).WaitRead(0xc208010ca0, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:89 +0x43\r\n  net.(*netFD).accept(0xc208010c40, 0x0, 0x7f4b2ce48248, 0xc2080de890)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_unix.go:419 +0x40b\r\n  net.(*TCPListener).AcceptTCP(0xc20802c218, 0xc20802e5a0, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/tcpsock_posix.go:234 +0x4e\r\n  github.com/gdamore/mangos/transport/tcp.(*listener).Accept(0xc208100360, 0x0, 0x0, 0x0, 0x0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/transport/tcp/tcp.go:116 +0xa2\r\n  github.com/gdamore/mangos.(*listener).serve(0xc208100300)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:573 +0x7e\r\n  created by github.com/gdamore/mangos.(*listener).Listen\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:595 +0x225\r\n  \r\n  goroutine 14 [select]:\r\n  github.com/gdamore/mangos.(*dialer).dialer(0xc208065b00)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:537 +0x2dd\r\n  created by github.com/gdamore/mangos.(*dialer).Dial\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:488 +0x12c\r\n  \r\n  goroutine 18 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc208075260, 0x7f4b2ce4b8d0, 0xc20802e460)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 74 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc208075fc0, 0x7f4b2ce4b8d0, 0xc20802ef00)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 25 [IO wait]:\r\n  net.(*pollDesc).Wait(0xc208010b50, 0x72, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:84 +0x47\r\n  net.(*pollDesc).WaitRead(0xc208010b50, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:89 +0x43\r\n  net.(*netFD).accept(0xc208010af0, 0x0, 0x7f4b2ce48248, 0xc2080dfae8)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_unix.go:419 +0x40b\r\n  net.(*TCPListener).AcceptTCP(0xc20802c0c0, 0xc20802eb40, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/tcpsock_posix.go:234 +0x4e\r\n  github.com/gdamore/mangos/transport/tcp.(*listener).Accept(0xc208101440, 0x0, 0x0, 0x0, 0x0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/transport/tcp/tcp.go:116 +0xa2\r\n  github.com/gdamore/mangos.(*listener).serve(0xc2081013e0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:573 +0x7e\r\n  created by github.com/gdamore/mangos.(*listener).Listen\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:595 +0x225\r\n  \r\n  goroutine 26 [select]:\r\n  github.com/gdamore/mangos.(*dialer).dialer(0xc208113cc0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:537 +0x2dd\r\n  created by github.com/gdamore/mangos.(*dialer).Dial\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:488 +0x12c\r\n  \r\n  goroutine 50 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc20800b600, 0x7f4b2ce4b8d0, 0xc20802e780)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 23 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc2080742a0, 0x7f4b2ce4b8d0, 0xc20802e5f0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 88 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc2080754c0, 0x7f4b2ce4b8d0, 0xc20802f770)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 105 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc208074240, 0x7f4b2ce4b8d0, 0xc208166190)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 101 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc208075fc0, 0x7f4b2ce4b8d0, 0xc2081660f0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 107 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc208075260, 0x7f4b2ce4b8d0, 0xc2081661e0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 31 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc2080754c0, 0x7f4b2ce4b8d0, 0xc20802ea00)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 53 [select]:\r\n  github.com/gdamore/mangos.(*dialer).dialer(0xc2081134c0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:537 +0x2dd\r\n  created by github.com/gdamore/mangos.(*dialer).Dial\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:488 +0x12c\r\n  \r\n  goroutine 36 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc208075820, 0x7f4b2ce4b8d0, 0xc20802eb90)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 92 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc208074240, 0x7f4b2ce4b8d0, 0xc20802f810)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 39 [syscall]:\r\n  syscall.Syscall(0x0, 0x11, 0xc208143248, 0x5b8, 0x19, 0x5d1, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/syscall/asm_linux_amd64.s:21 +0x5\r\n  syscall.read(0x11, 0xc208143248, 0x5b8, 0x5b8, 0x19, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/syscall/zsyscall_linux_amd64.go:867 +0x6e\r\n  syscall.Read(0x11, 0xc208143248, 0x5b8, 0x5b8, 0x19, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/syscall/syscall_unix.go:136 +0x58\r\n  os.(*File).read(0xc20802c230, 0xc208143248, 0x5b8, 0x5b8, 0x19, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/os/file_unix.go:191 +0x5e\r\n  os.(*File).Read(0xc20802c230, 0xc208143248, 0x5b8, 0x5b8, 0x19, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/os/file.go:95 +0x91\r\n  bytes.(*Buffer).ReadFrom(0xc208011880, 0x7f4b2ce481f8, 0xc20802c230, 0x48, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/bytes/buffer.go:169 +0x25a\r\n  io.Copy(0x7f4b2ce488b0, 0xc208011880, 0x7f4b2ce481f8, 0xc20802c230, 0x0, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/io/io.go:358 +0x13d\r\n  os/exec.func·003(0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/os/exec/exec.go:221 +0x7d\r\n  os/exec.func·004(0xc208075c00)\r\n  \t/home/jaten/pkg/go1.4/go/src/os/exec/exec.go:328 +0x2d\r\n  created by os/exec.(*Cmd).Start\r\n  \t/home/jaten/pkg/go1.4/go/src/os/exec/exec.go:329 +0xb6d\r\n  \r\n  goroutine 40 [chan receive]:\r\n  os/exec.(*Cmd).Wait(0xc20804e500, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/os/exec/exec.go:369 +0x30b\r\n  github.com/glycerine/goq.func·012()\r\n  \t/home/jaten/dev/cnet/go/src/github.com/glycerine/goq/shep.go:108 +0x3c\r\n  created by github.com/glycerine/goq.func·013\r\n  \t/home/jaten/dev/cnet/go/src/github.com/glycerine/goq/shep.go:110 +0x1124\r\n  \r\n  goroutine 41 [IO wait]:\r\n  net.(*pollDesc).Wait(0xc208011bf0, 0x72, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:84 +0x47\r\n  net.(*pollDesc).WaitRead(0xc208011bf0, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:89 +0x43\r\n  net.(*netFD).accept(0xc208011b90, 0x0, 0x7f4b2ce48248, 0xc20802b148)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_unix.go:419 +0x40b\r\n  net.(*TCPListener).AcceptTCP(0xc20802c268, 0xc20802e690, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/tcpsock_posix.go:234 +0x4e\r\n  github.com/gdamore/mangos/transport/tcp.(*listener).Accept(0xc2080d6c30, 0x0, 0x0, 0x0, 0x0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/transport/tcp/tcp.go:116 +0xa2\r\n  github.com/gdamore/mangos.(*listener).serve(0xc2080d6bd0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:573 +0x7e\r\n  created by github.com/gdamore/mangos.(*listener).Listen\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:595 +0x225\r\n  \r\n  goroutine 42 [select]:\r\n  github.com/gdamore/mangos.(*dialer).dialer(0xc2080ddb80)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:537 +0x2dd\r\n  created by github.com/gdamore/mangos.(*dialer).Dial\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:488 +0x12c\r\n  \r\n  goroutine 44 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc20800b2a0, 0x7f4b2ce4b8d0, 0xc20802f2c0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 55 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc208074240, 0x7f4b2ce4b8d0, 0xc20802ee10)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 77 [select]:\r\n  github.com/gdamore/mangos.(*dialer).dialer(0xc208113880)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:537 +0x2dd\r\n  created by github.com/gdamore/mangos.(*dialer).Dial\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:488 +0x12c\r\n  \r\n  goroutine 72 [IO wait]:\r\n  net.(*pollDesc).Wait(0xc2080c6680, 0x72, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:84 +0x47\r\n  net.(*pollDesc).WaitRead(0xc2080c6680, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:89 +0x43\r\n  net.(*netFD).accept(0xc2080c6620, 0x0, 0x7f4b2ce48248, 0xc20802b2a8)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_unix.go:419 +0x40b\r\n  net.(*TCPListener).AcceptTCP(0xc20802c168, 0xc20802f310, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/tcpsock_posix.go:234 +0x4e\r\n  github.com/gdamore/mangos/transport/tcp.(*listener).Accept(0xc208101050, 0x0, 0x0, 0x0, 0x0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/transport/tcp/tcp.go:116 +0xa2\r\n  github.com/gdamore/mangos.(*listener).serve(0xc208100ff0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:573 +0x7e\r\n  created by github.com/gdamore/mangos.(*listener).Listen\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:595 +0x225\r\n  \r\n  goroutine 52 [IO wait]:\r\n  net.(*pollDesc).Wait(0xc2080c6fb0, 0x72, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:84 +0x47\r\n  net.(*pollDesc).WaitRead(0xc2080c6fb0, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_poll_runtime.go:89 +0x43\r\n  net.(*netFD).accept(0xc2080c6f50, 0x0, 0x7f4b2ce48248, 0xc2080df648)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/fd_unix.go:419 +0x40b\r\n  net.(*TCPListener).AcceptTCP(0xc20802c208, 0xc20802efa0, 0x0, 0x0)\r\n  \t/home/jaten/pkg/go1.4/go/src/net/tcpsock_posix.go:234 +0x4e\r\n  github.com/gdamore/mangos/transport/tcp.(*listener).Accept(0xc2080d7c20, 0x0, 0x0, 0x0, 0x0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/transport/tcp/tcp.go:116 +0xa2\r\n  github.com/gdamore/mangos.(*listener).serve(0xc2080d7bc0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:573 +0x7e\r\n  created by github.com/gdamore/mangos.(*listener).Listen\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:595 +0x225\r\n  \r\n  goroutine 90 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc208075260, 0x7f4b2ce4b8d0, 0xc20802f7c0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 60 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc2080747e0, 0x7f4b2ce4b8d0, 0xc20802eff0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 100 [select]:\r\n  github.com/gdamore/mangos.(*dialer).dialer(0xc208112340)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:537 +0x2dd\r\n  created by github.com/gdamore/mangos.(*dialer).Dial\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/core.go:488 +0x12c\r\n  \r\n  goroutine 94 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc20800b2a0, 0x7f4b2ce4b8d0, 0xc20802f860)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 103 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc20800b2a0, 0x7f4b2ce4b8d0, 0xc208166140)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 109 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc2080754c0, 0x7f4b2ce4b8d0, 0xc208166230)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 111 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc20800b920, 0x7f4b2ce4b8d0, 0xc208166280)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n  goroutine 113 [select]:\r\n  github.com/gdamore/mangos/protocol/push.(*push).sender(0xc2080d29c0, 0x7f4b2ce4b8d0, 0xc2081662d0)\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:36 +0x1fb\r\n  created by github.com/gdamore/mangos/protocol/push.(*push).AddEndpoint\r\n  \t/home/jaten/dev/cnet/go/src/github.com/gdamore/mangos/protocol/push/push.go:77 +0x46\r\n  \r\n\r\n\r\n31 assertions thus far\r\n\r\n--- FAIL: TestSubmitShutdownToRemoteJobServ (8.21s)\r\n~~~\r\n"},"comment":{"url":"https://api.github.com/repos/gdamore/mangos/issues/comments/73397949","html_url":"https://github.com/gdamore/mangos/issues/77#issuecomment-73397949","issue_url":"https://api.github.com/repos/gdamore/mangos/issues/77","id":73397949,"user":{"login":"gdamore","id":1295229,"avatar_url":"https://avatars.githubusercontent.com/u/1295229?v=3","gravatar_id":"","url":"https://api.github.com/users/gdamore","html_url":"https://github.com/gdamore","followers_url":"https://api.github.com/users/gdamore/followers","following_url":"https://api.github.com/users/gdamore/following{/other_user}","gists_url":"https://api.github.com/users/gdamore/gists{/gist_id}","starred_url":"https://api.github.com/users/gdamore/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gdamore/subscriptions","organizations_url":"https://api.github.com/users/gdamore/orgs","repos_url":"https://api.github.com/users/gdamore/repos","events_url":"https://api.github.com/users/gdamore/events{/privacy}","received_events_url":"https://api.github.com/users/gdamore/received_events","type":"User","site_admin":false},"created_at":"2015-02-08T05:34:54Z","updated_at":"2015-02-08T05:34:54Z","body":"It cannot due to the fact that go doesn't have access to asynch IO which libnanomsg uses heavily.\n\nI have a version that I'm working on that fixes the drain problem more completely.  It involves some tighter coordination but I found that I was able to eliminate at least one select target which should help performance. \n\nSent from my iPhone\n\n> On Feb 7, 2015, at 10:06 AM, Jason E. Aten, Ph.D. <notifications@github.com> wrote:\n> \n> I didn't quite grasp the need for a mirror queue in the go process; can mangos not just do like c nanomsg and make use of the kernel's queues?\n> \n> —\n> Reply to this email directly or view it on GitHub.\n> \n"}}