{"action":"created","issue":{"url":"https://api.github.com/repos/grpc/grpc/issues/2557","labels_url":"https://api.github.com/repos/grpc/grpc/issues/2557/labels{/name}","comments_url":"https://api.github.com/repos/grpc/grpc/issues/2557/comments","events_url":"https://api.github.com/repos/grpc/grpc/issues/2557/events","html_url":"https://github.com/grpc/grpc/issues/2557","id":96228661,"number":2557,"title":"TSAN: tcp_client_posix alarm race","user":{"login":"ctiller","id":10120821,"avatar_url":"https://avatars.githubusercontent.com/u/10120821?v=3","gravatar_id":"","url":"https://api.github.com/users/ctiller","html_url":"https://github.com/ctiller","followers_url":"https://api.github.com/users/ctiller/followers","following_url":"https://api.github.com/users/ctiller/following{/other_user}","gists_url":"https://api.github.com/users/ctiller/gists{/gist_id}","starred_url":"https://api.github.com/users/ctiller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ctiller/subscriptions","organizations_url":"https://api.github.com/users/ctiller/orgs","repos_url":"https://api.github.com/users/ctiller/repos","events_url":"https://api.github.com/users/ctiller/events{/privacy}","received_events_url":"https://api.github.com/users/ctiller/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"ctiller","id":10120821,"avatar_url":"https://avatars.githubusercontent.com/u/10120821?v=3","gravatar_id":"","url":"https://api.github.com/users/ctiller","html_url":"https://github.com/ctiller","followers_url":"https://api.github.com/users/ctiller/followers","following_url":"https://api.github.com/users/ctiller/following{/other_user}","gists_url":"https://api.github.com/users/ctiller/gists{/gist_id}","starred_url":"https://api.github.com/users/ctiller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ctiller/subscriptions","organizations_url":"https://api.github.com/users/ctiller/orgs","repos_url":"https://api.github.com/users/ctiller/repos","events_url":"https://api.github.com/users/ctiller/events{/privacy}","received_events_url":"https://api.github.com/users/ctiller/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/grpc/grpc/milestones/25","html_url":"https://github.com/grpc/grpc/milestones/Beta","labels_url":"https://api.github.com/repos/grpc/grpc/milestones/25/labels","id":1134869,"number":25,"title":"Beta","description":"","creator":{"login":"ctiller","id":10120821,"avatar_url":"https://avatars.githubusercontent.com/u/10120821?v=3","gravatar_id":"","url":"https://api.github.com/users/ctiller","html_url":"https://github.com/ctiller","followers_url":"https://api.github.com/users/ctiller/followers","following_url":"https://api.github.com/users/ctiller/following{/other_user}","gists_url":"https://api.github.com/users/ctiller/gists{/gist_id}","starred_url":"https://api.github.com/users/ctiller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ctiller/subscriptions","organizations_url":"https://api.github.com/users/ctiller/orgs","repos_url":"https://api.github.com/users/ctiller/repos","events_url":"https://api.github.com/users/ctiller/events{/privacy}","received_events_url":"https://api.github.com/users/ctiller/received_events","type":"User","site_admin":false},"open_issues":253,"closed_issues":147,"state":"open","created_at":"2015-05-26T22:29:20Z","updated_at":"2015-07-24T18:29:19Z","due_on":"2015-07-31T07:00:00Z","closed_at":null},"comments":3,"created_at":"2015-07-21T05:10:37Z","updated_at":"2015-07-24T19:51:04Z","closed_at":null,"body":"```\r\nI0721 05:03:08.585198181   11453 qps_test.cc:51]             Running QPS test\r\nD0721 05:03:08.585417314   11453 test_config.c:90]           test slowdown: machine=1.000000 build=10.000000 total=10.000000\r\nD0721 05:03:08.815499142   11487 grpc_profiler.c:48]         You do not have google-perftools installed, profiling is disabled [for qps_server.prof]\r\nD0721 05:03:08.815555924   11487 grpc_profiler.c:52]         To install on ubuntu: sudo apt-get install google-perftools libgoogle-perftools-dev\r\nE0721 05:03:11.792764096   11478 tcp_client_posix.c:163]     on_writable failed during connect\r\nE0721 05:03:11.806934155   11478 tcp_client_posix.c:163]     on_writable failed during connect\r\nE0721 05:03:11.807027709   11478 tcp_client_posix.c:163]     on_writable failed during connect\r\nE0721 05:03:11.807175166   11478 tcp_client_posix.c:163]     on_writable failed during connect\r\nE0721 05:03:11.807177482   11713 tcp_client_posix.c:163]     on_writable failed during connect\r\nE0721 05:03:11.807239567   11478 tcp_client_posix.c:163]     on_writable failed during connect\r\nE0721 05:03:11.809593367   11708 tcp_client_posix.c:163]     on_writable failed during connect\r\nE0721 05:03:11.809719223   11708 tcp_client_posix.c:163]     on_writable failed during connect\r\nI0721 05:03:13.840397041   11453 driver.cc:187]              Warming up\r\nE0721 05:03:14.150807765   11460 tcp_client_posix.c:163]     on_writable failed during connect\r\nE0721 05:03:14.152516540   11708 tcp_client_posix.c:163]     on_writable failed during connect\r\nE0721 05:03:14.152712603   11708 tcp_client_posix.c:163]     on_writable failed during connect\r\n==================\r\nWARNING: ThreadSanitizer: data race (pid=11453)\r\n  Write of size 8 at 0x7d300001dca0 by thread T38:\r\n    #0 free <null>:0 (qps_test+0x00000020ad5b)\r\n    #1 gpr_free /var/local/git/grpc/src/core/support/alloc.c:47 (qps_test+0x000000654520)\r\n    #2 on_alarm /var/local/git/grpc/src/core/iomgr/tcp_client_posix.c:102 (qps_test+0x0000004ac2e4)\r\n    #3 run_some_expired_alarms /var/local/git/grpc/src/core/iomgr/alarm.c:347 (qps_test+0x0000004e195b)\r\n    #4 grpc_alarm_check /var/local/git/grpc/src/core/iomgr/alarm.c:358 (qps_test+0x0000004e2d61)\r\n    #5 grpc_pollset_work /var/local/git/grpc/src/core/iomgr/pollset_posix.c:146 (qps_test+0x0000004543ae)\r\n    #6 grpc_completion_queue_next /var/local/git/grpc/src/core/surface/completion_queue.c:173 (qps_test+0x0000003ff241)\r\n    #7 grpc::CompletionQueue::AsyncNextInternal(void**, bool*, gpr_timespec) /var/local/git/grpc/src/cpp/common/completion_queue.cc:54 (qps_test+0x0000003afc2d)\r\n    #8 grpc::CompletionQueue::NextStatus grpc::CompletionQueue::AsyncNext<std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > > >(void**, bool*, std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > > const&) /var/local/git/grpc/include/grpc++/completion_queue.h:101 (qps_test+0x00000034bcff)\r\n    #9 grpc::testing::AsyncClient::ThreadFunc(grpc::testing::Histogram*, unsigned long) /var/local/git/grpc/test/cpp/qps/client_async.cc:233 (qps_test+0x00000034a4e3)\r\n    #10 operator() /var/local/git/grpc/./test/cpp/qps/client.h:195 (qps_test+0x000000359a25)\r\n    #11 void std::_Bind_simple<grpc::testing::Client::Thread::Thread(grpc::testing::Client*, unsigned long)::{lambda()#1} ()>::_M_invoke<>(std::_Index_tuple<>) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:1699 (qps_test+0x0000003598f0)\r\n    #12 std::_Bind_simple<grpc::testing::Client::Thread::Thread(grpc::testing::Client*, unsigned long)::{lambda()#1} ()>::operator()() /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:1688 (qps_test+0x000000359890)\r\n    #13 std::thread::_Impl<std::_Bind_simple<grpc::testing::Client::Thread::Thread(grpc::testing::Client*, unsigned long)::{lambda()#1} ()> >::_M_run() /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/thread:115 (qps_test+0x000000359839)\r\n    #14 std::this_thread::__sleep_for(std::chrono::duration<long, std::ratio<1l, 1l> >, std::chrono::duration<long, std::ratio<1l, 1000000000l> >) <null>:0 (libstdc++.so.6+0x0000000b696f)\r\n\r\n  Previous read of size 4 at 0x7d300001dca4 by thread T39 (mutexes: write M468):\r\n    #0 grpc_alarm_cancel /var/local/git/grpc/src/core/iomgr/alarm.c:226 (qps_test+0x0000004e29b4)\r\n    #1 on_writable /var/local/git/grpc/src/core/iomgr/tcp_client_posix.c:182 (qps_test+0x0000004ac0a4)\r\n    #2 process_callback /var/local/git/grpc/src/core/iomgr/fd_posix.c:242 (qps_test+0x00000054d800)\r\n    #3 process_callbacks /var/local/git/grpc/src/core/iomgr/fd_posix.c:252 (qps_test+0x00000054bf0f)\r\n    #4 set_ready /var/local/git/grpc/src/core/iomgr/fd_posix.c:333 (qps_test+0x00000054d472)\r\n    #5 grpc_fd_become_writable /var/local/git/grpc/src/core/iomgr/fd_posix.c:445 (qps_test+0x00000054d50b)\r\n    #6 basic_pollset_maybe_work /var/local/git/grpc/src/core/iomgr/pollset_posix.c:438 (qps_test+0x0000004563f8)\r\n    #7 grpc_pollset_work /var/local/git/grpc/src/core/iomgr/pollset_posix.c:153 (qps_test+0x00000045456c)\r\n    #8 grpc_completion_queue_next /var/local/git/grpc/src/core/surface/completion_queue.c:173 (qps_test+0x0000003ff241)\r\n    #9 grpc::CompletionQueue::AsyncNextInternal(void**, bool*, gpr_timespec) /var/local/git/grpc/src/cpp/common/completion_queue.cc:54 (qps_test+0x0000003afc2d)\r\n    #10 grpc::CompletionQueue::NextStatus grpc::CompletionQueue::AsyncNext<std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > > >(void**, bool*, std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > > const&) /var/local/git/grpc/include/grpc++/completion_queue.h:101 (qps_test+0x00000034bcff)\r\n    #11 grpc::testing::AsyncClient::ThreadFunc(grpc::testing::Histogram*, unsigned long) /var/local/git/grpc/test/cpp/qps/client_async.cc:233 (qps_test+0x00000034a4e3)\r\n    #12 operator() /var/local/git/grpc/./test/cpp/qps/client.h:195 (qps_test+0x000000359a25)\r\n    #13 void std::_Bind_simple<grpc::testing::Client::Thread::Thread(grpc::testing::Client*, unsigned long)::{lambda()#1} ()>::_M_invoke<>(std::_Index_tuple<>) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:1699 (qps_test+0x0000003598f0)\r\n    #14 std::_Bind_simple<grpc::testing::Client::Thread::Thread(grpc::testing::Client*, unsigned long)::{lambda()#1} ()>::operator()() /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:1688 (qps_test+0x000000359890)\r\n    #15 std::thread::_Impl<std::_Bind_simple<grpc::testing::Client::Thread::Thread(grpc::testing::Client*, unsigned long)::{lambda()#1} ()> >::_M_run() /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/thread:115 (qps_test+0x000000359839)\r\n    #16 std::this_thread::__sleep_for(std::chrono::duration<long, std::ratio<1l, 1l> >, std::chrono::duration<long, std::ratio<1l, 1000000000l> >) <null>:0 (libstdc++.so.6+0x0000000b696f)\r\n\r\n  Mutex M468 (0x7f558e8c2350) created at:\r\n    #0 pthread_mutex_init <null>:0 (qps_test+0x00000020f880)\r\n    #1 gpr_mu_init /var/local/git/grpc/src/core/support/sync_posix.c:44 (qps_test+0x000000662c9a)\r\n    #2 grpc_alarm_list_init /var/local/git/grpc/src/core/iomgr/alarm.c:91 (qps_test+0x0000004e0d76)\r\n    #3 grpc_iomgr_init /var/local/git/grpc/src/core/iomgr/iomgr.c:93 (qps_test+0x000000425ce9)\r\n    #4 grpc_init /var/local/git/grpc/src/core/surface/init.c:80 (qps_test+0x0000004258d2)\r\n    #5 GrpcLibrary /var/local/git/grpc/include/grpc++/impl/grpc_library.h:43 (qps_test+0x0000003a39c3)\r\n    #6 Server /var/local/git/grpc/src/cpp/server/server.cc:188 (qps_test+0x0000003bed15)\r\n    #7 grpc::ServerBuilder::BuildAndStart() /var/local/git/grpc/src/cpp/server/server_builder.cc:103 (qps_test+0x0000003c83d5)\r\n    #8 QpsWorker /var/local/git/grpc/test/cpp/qps/qps_worker.cc:233 (qps_test+0x0000002e514e)\r\n    #9 grpc::testing::RunScenario(grpc::testing::ClientConfig const&, unsigned long, grpc::testing::ServerConfig const&, unsigned long, int, int, int) /var/local/git/grpc/test/cpp/qps/driver.cc:115 (qps_test+0x0000002c3869)\r\n    #10 grpc::testing::RunQPS() /var/local/git/grpc/test/cpp/qps/qps_test.cc:68 (qps_test+0x00000027210c)\r\n    #11 main /var/local/git/grpc/test/cpp/qps/qps_test.cc:81 (qps_test+0x000000271f82)\r\n\r\n  Thread T38 (tid=11821, running) created by thread T1 at:\r\n    #0 pthread_create <null>:0 (qps_test+0x00000020e471)\r\n    #1 std::thread::_M_start_thread(std::shared_ptr<std::thread::_Impl_base>) <null>:0 (libstdc++.so.6+0x0000000b6a90)\r\n    #2 Thread /var/local/git/grpc/./test/cpp/qps/client.h:212 (qps_test+0x000000356f98)\r\n    #3 grpc::testing::Client::StartThreads(unsigned long) /var/local/git/grpc/./test/cpp/qps/client.h:123 (qps_test+0x000000349ee3)\r\n    #4 AsyncUnaryClient /var/local/git/grpc/test/cpp/qps/client_async.cc:346 (qps_test+0x000000348629)\r\n    #5 grpc::testing::CreateAsyncUnaryClient(grpc::testing::ClientConfig const&) /var/local/git/grpc/test/cpp/qps/client_async.cc:459 (qps_test+0x000000347eee)\r\n    #6 grpc::testing::CreateClient(grpc::testing::ClientConfig const&) /var/local/git/grpc/test/cpp/qps/qps_worker.cc:72 (qps_test+0x0000002e4da7)\r\n    #7 grpc::testing::WorkerImpl::RunTestBody(grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*) /var/local/git/grpc/test/cpp/qps/qps_worker.cc:167 (qps_test+0x0000002ee975)\r\n    #8 grpc::testing::WorkerImpl::RunTest(grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*) /var/local/git/grpc/test/cpp/qps/qps_worker.cc:107 (qps_test+0x0000002ea15f)\r\n    #9 grpc::Status std::_Mem_fn<grpc::Status (grpc::testing::Worker::Service::*)(grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*)>::operator()<grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*, void>(grpc::testing::Worker::Service*, grpc::ServerContext*&&, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*&&) const /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:569 (qps_test+0x00000032cb76)\r\n    #10 std::_Function_handler<grpc::Status (grpc::testing::Worker::Service*, grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*), std::_Mem_fn<grpc::Status (grpc::testing::Worker::Service::*)(grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*)> >::_M_invoke(std::_Any_data const&, grpc::testing::Worker::Service*, grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:2024 (qps_test+0x00000032c4d8)\r\n    #11 std::function<grpc::Status (grpc::testing::Worker::Service*, grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*)>::operator()(grpc::testing::Worker::Service*, grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*) const /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:2439 (qps_test+0x00000032b87c)\r\n    #12 grpc::BidiStreamingHandler<grpc::testing::Worker::Service, grpc::testing::ClientArgs, grpc::testing::ClientStatus>::RunHandler(grpc::MethodHandler::HandlerParameter const&) /var/local/git/grpc/include/grpc++/impl/rpc_service_method.h:194 (qps_test+0x00000032b2ab)\r\n    #13 grpc::Server::SyncRequest::CallData::Run() /var/local/git/grpc/src/cpp/server/server.cc:136 (qps_test+0x0000003c402f)\r\n    #14 grpc::Server::RunRpc() /var/local/git/grpc/src/cpp/server/server.cc:407 (qps_test+0x0000003c23a0)\r\n    #15 void std::_Mem_fn<void (grpc::Server::*)()>::operator()<, void>(grpc::Server*) const /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:569 (qps_test+0x0000003c4b73)\r\n    #16 void std::_Bind<std::_Mem_fn<void (grpc::Server::*)()> (grpc::Server*)>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:1263 (qps_test+0x0000003c4a49)\r\n    #17 void std::_Bind<std::_Mem_fn<void (grpc::Server::*)()> (grpc::Server*)>::operator()<, void>() /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:1321 (qps_test+0x0000003c4991)\r\n    #18 std::_Function_handler<void (), std::_Bind<std::_Mem_fn<void (grpc::Server::*)()> (grpc::Server*)> >::_M_invoke(std::_Any_data const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:2039 (qps_test+0x0000003c4478)\r\n    #19 std::function<void ()>::operator()() const /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:2439 (qps_test+0x0000003b38e0)\r\n    #20 grpc::FixedSizeThreadPool::ThreadFunc() /var/local/git/grpc/src/cpp/server/fixed_size_thread_pool.cc:53 (qps_test+0x0000003b2721)\r\n    #21 void std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()>::operator()<, void>(grpc::FixedSizeThreadPool*) const /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:569 (qps_test+0x0000003bb2b3)\r\n    #22 void std::_Bind_simple<std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()> (grpc::FixedSizeThreadPool*)>::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:1699 (qps_test+0x0000003bb140)\r\n    #23 std::_Bind_simple<std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()> (grpc::FixedSizeThreadPool*)>::operator()() /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:1688 (qps_test+0x0000003bb0b0)\r\n    #24 std::thread::_Impl<std::_Bind_simple<std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()> (grpc::FixedSizeThreadPool*)> >::_M_run() /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/thread:115 (qps_test+0x0000003bb059)\r\n    #25 std::this_thread::__sleep_for(std::chrono::duration<long, std::ratio<1l, 1l> >, std::chrono::duration<long, std::ratio<1l, 1000000000l> >) <null>:0 (libstdc++.so.6+0x0000000b696f)\r\n\r\n  Thread T39 (tid=11822, running) created by thread T1 at:\r\n    #0 pthread_create <null>:0 (qps_test+0x00000020e471)\r\n    #1 std::thread::_M_start_thread(std::shared_ptr<std::thread::_Impl_base>) <null>:0 (libstdc++.so.6+0x0000000b6a90)\r\n    #2 Thread /var/local/git/grpc/./test/cpp/qps/client.h:212 (qps_test+0x000000356f98)\r\n    #3 grpc::testing::Client::StartThreads(unsigned long) /var/local/git/grpc/./test/cpp/qps/client.h:123 (qps_test+0x000000349ee3)\r\n    #4 AsyncUnaryClient /var/local/git/grpc/test/cpp/qps/client_async.cc:346 (qps_test+0x000000348629)\r\n    #5 grpc::testing::CreateAsyncUnaryClient(grpc::testing::ClientConfig const&) /var/local/git/grpc/test/cpp/qps/client_async.cc:459 (qps_test+0x000000347eee)\r\n    #6 grpc::testing::CreateClient(grpc::testing::ClientConfig const&) /var/local/git/grpc/test/cpp/qps/qps_worker.cc:72 (qps_test+0x0000002e4da7)\r\n    #7 grpc::testing::WorkerImpl::RunTestBody(grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*) /var/local/git/grpc/test/cpp/qps/qps_worker.cc:167 (qps_test+0x0000002ee975)\r\n    #8 grpc::testing::WorkerImpl::RunTest(grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*) /var/local/git/grpc/test/cpp/qps/qps_worker.cc:107 (qps_test+0x0000002ea15f)\r\n    #9 grpc::Status std::_Mem_fn<grpc::Status (grpc::testing::Worker::Service::*)(grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*)>::operator()<grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*, void>(grpc::testing::Worker::Service*, grpc::ServerContext*&&, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*&&) const /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:569 (qps_test+0x00000032cb76)\r\n    #10 std::_Function_handler<grpc::Status (grpc::testing::Worker::Service*, grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*), std::_Mem_fn<grpc::Status (grpc::testing::Worker::Service::*)(grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*)> >::_M_invoke(std::_Any_data const&, grpc::testing::Worker::Service*, grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:2024 (qps_test+0x00000032c4d8)\r\n    #11 std::function<grpc::Status (grpc::testing::Worker::Service*, grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*)>::operator()(grpc::testing::Worker::Service*, grpc::ServerContext*, grpc::ServerReaderWriter<grpc::testing::ClientStatus, grpc::testing::ClientArgs>*) const /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:2439 (qps_test+0x00000032b87c)\r\n    #12 grpc::BidiStreamingHandler<grpc::testing::Worker::Service, grpc::testing::ClientArgs, grpc::testing::ClientStatus>::RunHandler(grpc::MethodHandler::HandlerParameter const&) /var/local/git/grpc/include/grpc++/impl/rpc_service_method.h:194 (qps_test+0x00000032b2ab)\r\n    #13 grpc::Server::SyncRequest::CallData::Run() /var/local/git/grpc/src/cpp/server/server.cc:136 (qps_test+0x0000003c402f)\r\n    #14 grpc::Server::RunRpc() /var/local/git/grpc/src/cpp/server/server.cc:407 (qps_test+0x0000003c23a0)\r\n    #15 void std::_Mem_fn<void (grpc::Server::*)()>::operator()<, void>(grpc::Server*) const /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:569 (qps_test+0x0000003c4b73)\r\n    #16 void std::_Bind<std::_Mem_fn<void (grpc::Server::*)()> (grpc::Server*)>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:1263 (qps_test+0x0000003c4a49)\r\n    #17 void std::_Bind<std::_Mem_fn<void (grpc::Server::*)()> (grpc::Server*)>::operator()<, void>() /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:1321 (qps_test+0x0000003c4991)\r\n    #18 std::_Function_handler<void (), std::_Bind<std::_Mem_fn<void (grpc::Server::*)()> (grpc::Server*)> >::_M_invoke(std::_Any_data const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:2039 (qps_test+0x0000003c4478)\r\n    #19 std::function<void ()>::operator()() const /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:2439 (qps_test+0x0000003b38e0)\r\n    #20 grpc::FixedSizeThreadPool::ThreadFunc() /var/local/git/grpc/src/cpp/server/fixed_size_thread_pool.cc:53 (qps_test+0x0000003b2721)\r\n    #21 void std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()>::operator()<, void>(grpc::FixedSizeThreadPool*) const /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:569 (qps_test+0x0000003bb2b3)\r\n    #22 void std::_Bind_simple<std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()> (grpc::FixedSizeThreadPool*)>::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:1699 (qps_test+0x0000003bb140)\r\n    #23 std::_Bind_simple<std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()> (grpc::FixedSizeThreadPool*)>::operator()() /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/functional:1688 (qps_test+0x0000003bb0b0)\r\n    #24 std::thread::_Impl<std::_Bind_simple<std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()> (grpc::FixedSizeThreadPool*)> >::_M_run() /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/thread:115 (qps_test+0x0000003bb059)\r\n    #25 std::this_thread::__sleep_for(std::chrono::duration<long, std::ratio<1l, 1l> >, std::chrono::duration<long, std::ratio<1l, 1000000000l> >) <null>:0 (libstdc++.so.6+0x0000000b696f)\r\n\r\nSUMMARY: ThreadSanitizer: data race ??:0 free\r\n==================\r\n```"},"comment":{"url":"https://api.github.com/repos/grpc/grpc/issues/comments/124682760","html_url":"https://github.com/grpc/grpc/issues/2557#issuecomment-124682760","issue_url":"https://api.github.com/repos/grpc/grpc/issues/2557","id":124682760,"user":{"login":"ctiller","id":10120821,"avatar_url":"https://avatars.githubusercontent.com/u/10120821?v=3","gravatar_id":"","url":"https://api.github.com/users/ctiller","html_url":"https://github.com/ctiller","followers_url":"https://api.github.com/users/ctiller/followers","following_url":"https://api.github.com/users/ctiller/following{/other_user}","gists_url":"https://api.github.com/users/ctiller/gists{/gist_id}","starred_url":"https://api.github.com/users/ctiller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ctiller/subscriptions","organizations_url":"https://api.github.com/users/ctiller/orgs","repos_url":"https://api.github.com/users/ctiller/repos","events_url":"https://api.github.com/users/ctiller/events{/privacy}","received_events_url":"https://api.github.com/users/ctiller/received_events","type":"User","site_admin":false},"created_at":"2015-07-24T19:51:03Z","updated_at":"2015-07-24T19:51:03Z","body":"Likely another double push.\n\nThe connection spam could be eliminated by tuning down the number of\nchannels in qps_test - we are unable to maintain the connection rate\nrequired under tsan.\n\nOn Fri, Jul 24, 2015, 12:46 PM Vijay Pai <notifications@github.com> wrote:\n\n> Here's a better log that also relates it to a tsan issue:\n>\n> $ tools/run_tests/run_tests.py -l c++ -c tsan -n 10\n>\n> I0724 12:20:07.579378393 29609 qps_test.cc:51] Running QPS test\n> D0724 12:20:07.579618186 29609 test_config.c:90] test slowdown:\n> machine=1.000000 build=10.000000 total=10.000000\n> D0724 12:20:08.830722143 29688 grpc_profiler.c:48] You do not have\n> google-perftools installed, profiling is disabled [for qps_server.prof]\n> D0724 12:20:08.830777835 29688 grpc_profiler.c:52] To install on ubuntu:\n> sudo apt-get install google-perftools libgoogle-perftools-dev\n> E0724 12:20:37.275053679 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.275226843 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.286438925 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.286594137 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.286686656 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.286786878 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.323301055 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.323460042 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.323550651 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.325881228 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.326200038 30629 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.326300633 30629 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.330543822 30636 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.330673433 30636 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.330763116 30636 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.332226468 30636 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.332334049 30636 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.332435773 30636 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.332526837 30636 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.344620227 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.344771358 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.344862330 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.344950018 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.345034622 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.345117672 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.345198943 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.345280469 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.345364912 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.345445565 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.345536781 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.345624835 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.347458724 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.347572459 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.348632896 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.348739566 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.348829103 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.348922226 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.349024018 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.349128659 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.349222652 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.371696259 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.371915110 30631 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:20:37.372001478 30631 tcp_client_posix.c:168] on_writable failed\n> during connect\n> [lots more]\n> E0724 12:21:07.945877058 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:21:08.210259015 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> E0724 12:21:08.506516904 29655 tcp_client_posix.c:168] on_writable failed\n> during connect\n> I0724 12:21:09.334132998 29609 driver.cc:193] Starting\n> I0724 12:21:09.403070002 29609 driver.cc:214] Running\n>\n> WARNING: ThreadSanitizer: data race (pid=29609)\n> Write of size 4 at 0x7f26707d6134 by thread T32:\n> #0 gpr_stack_lockfree_push\n> /usr/local/google/home/vpai/Git/grpc/src/core/support/stack_lockfree.c:107\n> (exe+0x00000062bb7e)\n> #1 <https://github.com/grpc/grpc/pull/1> done_request_event\n> /usr/local/google/home/vpai/Git/grpc/src/core/surface/server.c:1269\n> (exe+0x00000044611d)\n> #2 <https://github.com/grpc/grpc/pull/2> grpc_completion_queue_next\n> /usr/local/google/home/vpai/Git/grpc/src/core/surface/completion_queue.c:166\n> (exe+0x0000003cfe1e)\n> #3 <https://github.com/grpc/grpc/pull/3>\n> grpc::CompletionQueue::AsyncNextInternal(void\n> *, bool*, gpr_timespec)\n> /usr/local/google/home/vpai/Git/grpc/src/cpp/common/completion_queue.cc:54\n> (exe+0x000000380b7d) #4 <https://github.com/grpc/grpc/pull/4>\n> grpc::CompletionQueue::Next(void*, bool*)\n> /usr/local/google/home/vpai/Git/grpc/include/grpc++/completion_queue.h:108\n> (exe+0x0000002c89c8)\n> #5 <https://github.com/grpc/grpc/pull/5> operator()\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/server_async.cc:107\n> (exe+0x0000002ce18f)\n> #6 <https://github.com/grpc/grpc/pull/6> void\n> std::_Bind_simple::_M_invoke<>(std::_Index_tuple<>)\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1731\n> (exe+0x0000002ce080)\n> #7 <https://github.com/grpc/grpc/pull/7> std::_Bind_simple::operator()()\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1720\n> (exe+0x0000002ce020)\n> #8 <https://github.com/grpc/grpc/pull/8> std::thread::_Impl >::_M_run()\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/thread:115\n> (exe+0x0000002cdfc9)\n> #9 <https://github.com/grpc/grpc/pull/9> execute_native_thread_routine\n> /build/buildd/gcc-4.8-4.8.4/build/x86_64-linux-gnu/libstdc++-v3/src/c++11/../../../../../src/libstdc++-v3/src/c++11/thread.cc:84\n> (libstdc++.so.6+0x0000000b1a3f)\n>\n> Previous atomic read of size 8 at 0x7f26707d6130 by thread T33:\n> [failed to restore the stack]\n>\n> Location is heap block of size 262159 at 0x7f26707b8000 allocated by\n> thread T14:\n> #0 malloc ??:0 (exe+0x0000001e2f99)\n> #1 <https://github.com/grpc/grpc/pull/1> gpr_malloc\n> /usr/local/google/home/vpai/Git/grpc/src/core/support/alloc.c:40\n> (exe+0x000000621940)\n> #2 <https://github.com/grpc/grpc/pull/2> gpr_malloc_aligned\n> /usr/local/google/home/vpai/Git/grpc/src/core/support/alloc.c:60\n> (exe+0x000000621ba1)\n> #3 <https://github.com/grpc/grpc/pull/3> gpr_stack_lockfree_create\n> /usr/local/google/home/vpai/Git/grpc/src/core/support/stack_lockfree.c:84\n> (exe+0x00000062b90b)\n> #4 <https://github.com/grpc/grpc/pull/4> grpc_server_create_from_filters\n> /usr/local/google/home/vpai/Git/grpc/src/core/surface/server.c:779\n> (exe+0x00000043ec8d)\n> #5 <https://github.com/grpc/grpc/pull/5> grpc_server_create\n> /usr/local/google/home/vpai/Git/grpc/src/core/surface/server_create.c:41\n> (exe+0x00000043e6b7)\n> #6 <https://github.com/grpc/grpc/pull/6> grpc::CreateServer(int)\n> /usr/local/google/home/vpai/Git/grpc/src/cpp/server/server.cc:175\n> (exe+0x00000039003b)\n> #7 <https://github.com/grpc/grpc/pull/7> Server\n> /usr/local/google/home/vpai/Git/grpc/src/cpp/server/server.cc:186\n> (exe+0x00000038fd03)\n> #8 <https://github.com/grpc/grpc/pull/8>\n> grpc::ServerBuilder::BuildAndStart()\n> /usr/local/google/home/vpai/Git/grpc/src/cpp/server/server_builder.cc:103\n> (exe+0x000000399525)\n> #9 <https://github.com/grpc/grpc/pull/9> AsyncQpsServerTest\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/server_async.cc:80\n> (exe+0x0000002c4343)\n> #10 <https://github.com/grpc/grpc/issues/10>\n> grpc::testing::CreateAsyncServer(grpc::testing::ServerConfig const&, int)\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/server_async.cc:342\n> (exe+0x0000002c36c0)\n> #11 <https://github.com/grpc/grpc/pull/11>\n> grpc::testing::CreateServer(grpc::testing::ServerConfig const&, int)\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/qps_worker.cc:86\n> (exe+0x0000002b51f9)\n> #12 <https://github.com/grpc/grpc/pull/12>\n> grpc::testing::WorkerImpl::RunServerBody(grpc::ServerContext*,\n> grpc::ServerReaderWriter*)\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/qps_worker.cc:195\n> (exe+0x0000002baabd)\n> #13 <https://github.com/grpc/grpc/pull/13>\n> grpc::testing::WorkerImpl::RunServer(grpc::ServerContext*,\n> grpc::ServerReaderWriter*)\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/qps_worker.cc:121\n> (exe+0x0000002ba4ff)\n> #14 <https://github.com/grpc/grpc/pull/14> grpc::Status std::_Mem_fn\n> *)>::operator()<grpc::ServerContext*, grpc::ServerReaderWriter*,\n> void>(grpc::testing::Worker::Service*, grpc::ServerContext*&&,\n> grpc::ServerReaderWriter*&&) const\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:601\n> (exe+0x0000002fa676)\n> #15 <https://github.com/grpc/grpc/pull/15> std::_Function_handler*),\n> std::_Mem_fn<grpc::Status (grpc::testing::Worker::Service::*\n> )(grpc::ServerContext*, grpc::ServerReaderWriter*)>\n> >::_M_invoke(std::_Any_data const&, grpc::testing::Worker::Service*,\n> grpc::ServerContext*, grpc::ServerReaderWriter\n> *)\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:2056\n> (exe+0x0000002f9fd8) #16 <https://github.com/grpc/grpc/pull/16>\n> std::function<grpc::Status (grpc::testing::Worker::Service*,\n> grpc::ServerContext*, grpc::ServerReaderWriter*\n> )>::operator()(grpc::testing::Worker::Service*, grpc::ServerContext*,\n> grpc::ServerReaderWriter\n>\n>\n>\n> *) const\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:2471\n> (exe+0x0000002f927c) #17 <https://github.com/grpc/grpc/pull/17>\n> grpc::BidiStreamingHandler::RunHandler(grpc::MethodHandler::HandlerParameter\n> const&)\n> /usr/local/google/home/vpai/Git/grpc/include/grpc++/impl/rpc_service_method.h:194\n> (exe+0x0000002f8cab) #18 <https://github.com/grpc/grpc/pull/18>\n> grpc::Server::SyncRequest::CallData::Run()\n> /usr/local/google/home/vpai/Git/grpc/src/cpp/server/server.cc:136\n> (exe+0x000000394edf) #19 <https://github.com/grpc/grpc/pull/19>\n> grpc::Server::RunRpc()\n> /usr/local/google/home/vpai/Git/grpc/src/cpp/server/server.cc:407\n> (exe+0x000000393240) #20 <https://github.com/grpc/grpc/issues/20> void\n> std::_Mem_fn<void (grpc::Server::*)()>::operator()<, void>(grpc::Server\n> *) const\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:601\n> (exe+0x000000395a23) #21 <https://github.com/grpc/grpc/pull/21> void\n> std::_Bind<std::_Mem_fn<void (grpc::Server::*)()> (grpc::Server\n> *)>::__call(std::tuple<>&&, std::_Index_tuple<0ul>)\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1295\n> (exe+0x0000003958f9) #22 <https://github.com/grpc/grpc/pull/22> void\n> std::_Bind<std::_Mem_fn<void (grpc::Server::*)()> (grpc::Server\n> *)>::operator()<, void>()\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1353\n> (exe+0x000000395841) #23 <https://github.com/grpc/grpc/pull/23>\n> std::_Function_handler<void (), std::_Bind<std::_Mem_fn<void\n> (grpc::Server::*)()> (grpc::Server\n>\n>\n> *)> >::_M_invoke(std::_Any_data const&)\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:2071\n> (exe+0x000000395328) #24 <https://github.com/grpc/grpc/pull/24>\n> std::function::operator()() const\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:2471\n> (exe+0x0000003848c0) #25 <https://github.com/grpc/grpc/pull/25>\n> grpc::FixedSizeThreadPool::ThreadFunc()\n> /usr/local/google/home/vpai/Git/grpc/src/cpp/server/fixed_size_thread_pool.cc:53\n> (exe+0x000000383681) #26 <https://github.com/grpc/grpc/pull/26> void\n> std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()>::operator()<,\n> void>(grpc::FixedSizeThreadPool\n> *) const\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:601\n> (exe+0x00000038bf23) #27 <https://github.com/grpc/grpc/pull/27> void\n> std::_Bind_simple<std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()>\n> (grpc::FixedSizeThreadPool\n> *)>::_M_invoke<0ul>(std::_Index_tuple<0ul>)\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1731\n> (exe+0x00000038bdb0) #28 <https://github.com/grpc/grpc/pull/28>\n> std::_Bind_simple<std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()>\n> (grpc::FixedSizeThreadPool\n> *)>::operator()()\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1720\n> (exe+0x00000038bd20) #29 <https://github.com/grpc/grpc/pull/29>\n> std::thread::_Impl<std::_Bind_simple<std::_Mem_fn<void\n> (grpc::FixedSizeThreadPool::*)()> (grpc::FixedSizeThreadPool*)>\n> >::_M_run()\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/thread:115\n> (exe+0x00000038bcc9)\n> #30 <https://github.com/grpc/grpc/pull/30> execute_native_thread_routine\n> /build/buildd/gcc-4.8-4.8.4/build/x86_64-linux-gnu/libstdc++-v3/src/c++11/../../../../../src/libstdc++-v3/src/c++11/thread.cc:84\n> (libstdc++.so.6+0x0000000b1a3f)\n>\n> Thread T32 (tid=30631, running) created by thread T14 at:\n> #0 pthread_create ??:0 (exe+0x0000001e781b)\n> #1 <https://github.com/grpc/grpc/pull/1> __gthread_create\n> /build/buildd/gcc-4.8-4.8.4/build/x86_64-linux-gnu/libstdc++-v3/include/x86_64-linux-gnu/bits/gthr-default.h:662\n> (libstdc++.so.6+0x0000000b1c8e)\n> #2 <https://github.com/grpc/grpc/pull/2> AsyncQpsServerTest\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/server_async.cc:103\n> (exe+0x0000002c4e89)\n> #3 <https://github.com/grpc/grpc/pull/3>\n> grpc::testing::CreateAsyncServer(grpc::testing::ServerConfig const&, int)\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/server_async.cc:342\n> (exe+0x0000002c36c0)\n> #4 <https://github.com/grpc/grpc/pull/4>\n> grpc::testing::CreateServer(grpc::testing::ServerConfig const&, int)\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/qps_worker.cc:86\n> (exe+0x0000002b51f9)\n> #5 <https://github.com/grpc/grpc/pull/5>\n> grpc::testing::WorkerImpl::RunServerBody(grpc::ServerContext*,\n> grpc::ServerReaderWriter*)\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/qps_worker.cc:195\n> (exe+0x0000002baabd)\n> #6 <https://github.com/grpc/grpc/pull/6>\n> grpc::testing::WorkerImpl::RunServer(grpc::ServerContext*,\n> grpc::ServerReaderWriter*)\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/qps_worker.cc:121\n> (exe+0x0000002ba4ff)\n> #7 <https://github.com/grpc/grpc/pull/7> grpc::Status std::_Mem_fn\n> *)>::operator()<grpc::ServerContext*, grpc::ServerReaderWriter*,\n> void>(grpc::testing::Worker::Service*, grpc::ServerContext*&&,\n> grpc::ServerReaderWriter*&&) const\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:601\n> (exe+0x0000002fa676)\n> #8 <https://github.com/grpc/grpc/pull/8> std::_Function_handler*),\n> std::_Mem_fn<grpc::Status (grpc::testing::Worker::Service::*\n> )(grpc::ServerContext*, grpc::ServerReaderWriter*)>\n> >::_M_invoke(std::_Any_data const&, grpc::testing::Worker::Service*,\n> grpc::ServerContext*, grpc::ServerReaderWriter\n> *)\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:2056\n> (exe+0x0000002f9fd8) #9 <https://github.com/grpc/grpc/pull/9>\n> std::function<grpc::Status (grpc::testing::Worker::Service*,\n> grpc::ServerContext*, grpc::ServerReaderWriter*\n> )>::operator()(grpc::testing::Worker::Service*, grpc::ServerContext*,\n> grpc::ServerReaderWriter\n>\n>\n>\n> *) const\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:2471\n> (exe+0x0000002f927c) #10 <https://github.com/grpc/grpc/issues/10>\n> grpc::BidiStreamingHandler::RunHandler(grpc::MethodHandler::HandlerParameter\n> const&)\n> /usr/local/google/home/vpai/Git/grpc/include/grpc++/impl/rpc_service_method.h:194\n> (exe+0x0000002f8cab) #11 <https://github.com/grpc/grpc/pull/11>\n> grpc::Server::SyncRequest::CallData::Run()\n> /usr/local/google/home/vpai/Git/grpc/src/cpp/server/server.cc:136\n> (exe+0x000000394edf) #12 <https://github.com/grpc/grpc/pull/12>\n> grpc::Server::RunRpc()\n> /usr/local/google/home/vpai/Git/grpc/src/cpp/server/server.cc:407\n> (exe+0x000000393240) #13 <https://github.com/grpc/grpc/pull/13> void\n> std::_Mem_fn<void (grpc::Server::*)()>::operator()<, void>(grpc::Server\n> *) const\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:601\n> (exe+0x000000395a23) #14 <https://github.com/grpc/grpc/pull/14> void\n> std::_Bind<std::_Mem_fn<void (grpc::Server::*)()> (grpc::Server\n> *)>::__call(std::tuple<>&&, std::_Index_tuple<0ul>)\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1295\n> (exe+0x0000003958f9) #15 <https://github.com/grpc/grpc/pull/15> void\n> std::_Bind<std::_Mem_fn<void (grpc::Server::*)()> (grpc::Server\n> *)>::operator()<, void>()\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1353\n> (exe+0x000000395841) #16 <https://github.com/grpc/grpc/pull/16>\n> std::_Function_handler<void (), std::_Bind<std::_Mem_fn<void\n> (grpc::Server::*)()> (grpc::Server\n>\n>\n> *)> >::_M_invoke(std::_Any_data const&)\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:2071\n> (exe+0x000000395328) #17 <https://github.com/grpc/grpc/pull/17>\n> std::function::operator()() const\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:2471\n> (exe+0x0000003848c0) #18 <https://github.com/grpc/grpc/pull/18>\n> grpc::FixedSizeThreadPool::ThreadFunc()\n> /usr/local/google/home/vpai/Git/grpc/src/cpp/server/fixed_size_thread_pool.cc:53\n> (exe+0x000000383681) #19 <https://github.com/grpc/grpc/pull/19> void\n> std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()>::operator()<,\n> void>(grpc::FixedSizeThreadPool\n> *) const\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:601\n> (exe+0x00000038bf23) #20 <https://github.com/grpc/grpc/issues/20> void\n> std::_Bind_simple<std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()>\n> (grpc::FixedSizeThreadPool\n> *)>::_M_invoke<0ul>(std::_Index_tuple<0ul>)\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1731\n> (exe+0x00000038bdb0) #21 <https://github.com/grpc/grpc/pull/21>\n> std::_Bind_simple<std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()>\n> (grpc::FixedSizeThreadPool\n> *)>::operator()()\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1720\n> (exe+0x00000038bd20) #22 <https://github.com/grpc/grpc/pull/22>\n> std::thread::_Impl<std::_Bind_simple<std::_Mem_fn<void\n> (grpc::FixedSizeThreadPool::*)()> (grpc::FixedSizeThreadPool*)>\n> >::_M_run()\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/thread:115\n> (exe+0x00000038bcc9)\n> #23 <https://github.com/grpc/grpc/pull/23> execute_native_thread_routine\n> /build/buildd/gcc-4.8-4.8.4/build/x86_64-linux-gnu/libstdc++-v3/src/c++11/../../../../../src/libstdc++-v3/src/c++11/thread.cc:84\n> (libstdc++.so.6+0x0000000b1a3f)\n>\n> Thread T33 (tid=30634, running) created by thread T14 at:\n> #0 pthread_create ??:0 (exe+0x0000001e781b)\n> #1 <https://github.com/grpc/grpc/pull/1> __gthread_create\n> /build/buildd/gcc-4.8-4.8.4/build/x86_64-linux-gnu/libstdc++-v3/include/x86_64-linux-gnu/bits/gthr-default.h:662\n> (libstdc++.so.6+0x0000000b1c8e)\n> #2 <https://github.com/grpc/grpc/pull/2> AsyncQpsServerTest\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/server_async.cc:103\n> (exe+0x0000002c4e89)\n> #3 <https://github.com/grpc/grpc/pull/3>\n> grpc::testing::CreateAsyncServer(grpc::testing::ServerConfig const&, int)\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/server_async.cc:342\n> (exe+0x0000002c36c0)\n> #4 <https://github.com/grpc/grpc/pull/4>\n> grpc::testing::CreateServer(grpc::testing::ServerConfig const&, int)\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/qps_worker.cc:86\n> (exe+0x0000002b51f9)\n> #5 <https://github.com/grpc/grpc/pull/5>\n> grpc::testing::WorkerImpl::RunServerBody(grpc::ServerContext*,\n> grpc::ServerReaderWriter*)\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/qps_worker.cc:195\n> (exe+0x0000002baabd)\n> #6 <https://github.com/grpc/grpc/pull/6>\n> grpc::testing::WorkerImpl::RunServer(grpc::ServerContext*,\n> grpc::ServerReaderWriter*)\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/qps_worker.cc:121\n> (exe+0x0000002ba4ff)\n> #7 <https://github.com/grpc/grpc/pull/7> grpc::Status std::_Mem_fn\n> *)>::operator()<grpc::ServerContext*, grpc::ServerReaderWriter*,\n> void>(grpc::testing::Worker::Service*, grpc::ServerContext*&&,\n> grpc::ServerReaderWriter*&&) const\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:601\n> (exe+0x0000002fa676)\n> #8 <https://github.com/grpc/grpc/pull/8> std::_Function_handler*),\n> std::_Mem_fn<grpc::Status (grpc::testing::Worker::Service::*\n> )(grpc::ServerContext*, grpc::ServerReaderWriter*)>\n> >::_M_invoke(std::_Any_data const&, grpc::testing::Worker::Service*,\n> grpc::ServerContext*, grpc::ServerReaderWriter\n> *)\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:2056\n> (exe+0x0000002f9fd8) #9 <https://github.com/grpc/grpc/pull/9>\n> std::function<grpc::Status (grpc::testing::Worker::Service*,\n> grpc::ServerContext*, grpc::ServerReaderWriter*\n> )>::operator()(grpc::testing::Worker::Service*, grpc::ServerContext*,\n> grpc::ServerReaderWriter\n>\n>\n>\n> *) const\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:2471\n> (exe+0x0000002f927c) #10 <https://github.com/grpc/grpc/issues/10>\n> grpc::BidiStreamingHandler::RunHandler(grpc::MethodHandler::HandlerParameter\n> const&)\n> /usr/local/google/home/vpai/Git/grpc/include/grpc++/impl/rpc_service_method.h:194\n> (exe+0x0000002f8cab) #11 <https://github.com/grpc/grpc/pull/11>\n> grpc::Server::SyncRequest::CallData::Run()\n> /usr/local/google/home/vpai/Git/grpc/src/cpp/server/server.cc:136\n> (exe+0x000000394edf) #12 <https://github.com/grpc/grpc/pull/12>\n> grpc::Server::RunRpc()\n> /usr/local/google/home/vpai/Git/grpc/src/cpp/server/server.cc:407\n> (exe+0x000000393240) #13 <https://github.com/grpc/grpc/pull/13> void\n> std::_Mem_fn<void (grpc::Server::*)()>::operator()<, void>(grpc::Server\n> *) const\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:601\n> (exe+0x000000395a23) #14 <https://github.com/grpc/grpc/pull/14> void\n> std::_Bind<std::_Mem_fn<void (grpc::Server::*)()> (grpc::Server\n> *)>::__call(std::tuple<>&&, std::_Index_tuple<0ul>)\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1295\n> (exe+0x0000003958f9) #15 <https://github.com/grpc/grpc/pull/15> void\n> std::_Bind<std::_Mem_fn<void (grpc::Server::*)()> (grpc::Server\n> *)>::operator()<, void>()\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1353\n> (exe+0x000000395841) #16 <https://github.com/grpc/grpc/pull/16>\n> std::_Function_handler<void (), std::_Bind<std::_Mem_fn<void\n> (grpc::Server::*)()> (grpc::Server\n>\n>\n> *)> >::_M_invoke(std::_Any_data const&)\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:2071\n> (exe+0x000000395328) #17 <https://github.com/grpc/grpc/pull/17>\n> std::function::operator()() const\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:2471\n> (exe+0x0000003848c0) #18 <https://github.com/grpc/grpc/pull/18>\n> grpc::FixedSizeThreadPool::ThreadFunc()\n> /usr/local/google/home/vpai/Git/grpc/src/cpp/server/fixed_size_thread_pool.cc:53\n> (exe+0x000000383681) #19 <https://github.com/grpc/grpc/pull/19> void\n> std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()>::operator()<,\n> void>(grpc::FixedSizeThreadPool\n> *) const\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:601\n> (exe+0x00000038bf23) #20 <https://github.com/grpc/grpc/issues/20> void\n> std::_Bind_simple<std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()>\n> (grpc::FixedSizeThreadPool\n> *)>::_M_invoke<0ul>(std::_Index_tuple<0ul>)\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1731\n> (exe+0x00000038bdb0) #21 <https://github.com/grpc/grpc/pull/21>\n> std::_Bind_simple<std::_Mem_fn<void (grpc::FixedSizeThreadPool::*)()>\n> (grpc::FixedSizeThreadPool\n> *)>::operator()()\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1720\n> (exe+0x00000038bd20) #22 <https://github.com/grpc/grpc/pull/22>\n> std::thread::_Impl<std::_Bind_simple<std::_Mem_fn<void\n> (grpc::FixedSizeThreadPool::*)()> (grpc::FixedSizeThreadPool*)>\n> >::_M_run()\n> /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/thread:115\n> (exe+0x00000038bcc9)\n> #23 <https://github.com/grpc/grpc/pull/23> execute_native_thread_routine\n> /build/buildd/gcc-4.8-4.8.4/build/x86_64-linux-gnu/libstdc++-v3/src/c++11/../../../../../src/libstdc++-v3/src/c++11/thread.cc:84\n> (libstdc++.so.6+0x0000000b1a3f)\n>\n> Thread T14 (tid=29688, running) created by main thread at:\n> #0 pthread_create ??:0 (exe+0x0000001e781b)\n> #1 <https://github.com/grpc/grpc/pull/1> __gthread_create\n> /build/buildd/gcc-4.8-4.8.4/build/x86_64-linux-gnu/libstdc++-v3/include/x86_64-linux-gnu/bits/gthr-default.h:662\n> (libstdc++.so.6+0x0000000b1c8e)\n> #2 <https://github.com/grpc/grpc/pull/2> FixedSizeThreadPool\n> /usr/local/google/home/vpai/Git/grpc/src/cpp/server/fixed_size_thread_pool.cc:62\n> (exe+0x0000003839f5)\n> #3 <https://github.com/grpc/grpc/pull/3> grpc::CreateDefaultThreadPool()\n> /usr/local/google/home/vpai/Git/grpc/src/cpp/server/create_default_thread_pool.cc:44\n> (exe+0x0000003a9d76)\n> #4 <https://github.com/grpc/grpc/pull/4>\n> grpc::ServerBuilder::BuildAndStart()\n> /usr/local/google/home/vpai/Git/grpc/src/cpp/server/server_builder.cc:100\n> (exe+0x00000039944c)\n> #5 <https://github.com/grpc/grpc/pull/5> QpsWorker\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/qps_worker.cc:233\n> (exe+0x0000002b547e)\n> #6 <https://github.com/grpc/grpc/pull/6>\n> grpc::testing::RunScenario(grpc::testing::ClientConfig const&, unsigned\n> long, grpc::testing::ServerConfig const&, unsigned long, int, int, int)\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/driver.cc:115\n> (exe+0x000000293abd)\n> #7 <https://github.com/grpc/grpc/pull/7> grpc::testing::RunQPS()\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/qps_test.cc:68\n> (exe+0x0000002423ec)\n> #8 <https://github.com/grpc/grpc/pull/8> main\n> /usr/local/google/home/vpai/Git/grpc/test/cpp/qps/qps_test.cc:81\n> (exe+0x000000242262)\n>\n> SUMMARY: ThreadSanitizer: data race\n> /usr/local/google/home/vpai/Git/grpc/src/core/support/stack_lockfree.c:107\n> gpr_stack_lockfree_push\n>\n> —\n> Reply to this email directly or view it on GitHub\n> <https://github.com/grpc/grpc/issues/2557#issuecomment-124677011>.\n>\n"}}