{"action":"created","issue":{"url":"https://api.github.com/repos/edenhill/librdkafka/issues/464","labels_url":"https://api.github.com/repos/edenhill/librdkafka/issues/464/labels{/name}","comments_url":"https://api.github.com/repos/edenhill/librdkafka/issues/464/comments","events_url":"https://api.github.com/repos/edenhill/librdkafka/issues/464/events","html_url":"https://github.com/edenhill/librdkafka/issues/464","id":123134018,"number":464,"title":"Suspicious misuse of thread lock","user":{"login":"microwish","id":4253329,"avatar_url":"https://avatars.githubusercontent.com/u/4253329?v=3","gravatar_id":"","url":"https://api.github.com/users/microwish","html_url":"https://github.com/microwish","followers_url":"https://api.github.com/users/microwish/followers","following_url":"https://api.github.com/users/microwish/following{/other_user}","gists_url":"https://api.github.com/users/microwish/gists{/gist_id}","starred_url":"https://api.github.com/users/microwish/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/microwish/subscriptions","organizations_url":"https://api.github.com/users/microwish/orgs","repos_url":"https://api.github.com/users/microwish/repos","events_url":"https://api.github.com/users/microwish/events{/privacy}","received_events_url":"https://api.github.com/users/microwish/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":11,"created_at":"2015-12-20T04:50:18Z","updated_at":"2015-12-26T12:45:33Z","closed_at":null,"body":"I used valgrind --tool=helgrind to debug program, and there were many errors like \"lock order \"0x636BCC8 before 0x636BC50\" violated\", e.g.:\r\n\r\n==13891== Thread #11: lock order \"0x636BCC8 before 0x636BC50\" violated\r\n==13891== \r\n==13891== Observed (incorrect) order is: acquisition of lock at 0x636BC50\r\n==13891==    at 0x4A0B069: pthread_mutex_lock (hg_intercepts.c:495)\r\n==13891==    by 0x5EF0E58: mtx_lock (tinycthread.c:135)\r\n==13891==    by 0x5EEA1DC: rd_kafka_toppar_fetch_decide (rdkafka_partition.c:1208)\r\n==13891==    by 0x5ECDE1E: rd_kafka_broker_thread_main (rdkafka_broker.c:2098)\r\n==13891==    by 0x5EF0AEE: _thrd_wrapper_function (tinycthread.c:596)\r\n==13891==    by 0x4A0C0D4: mythread_wrapper (hg_intercepts.c:219)\r\n==13891==    by 0x3EDEE07A50: start_thread (in /lib64/libpthread-2.12.so)\r\n==13891==    by 0x3EDE6E893C: clone (in /lib64/libc-2.12.so)\r\n==13891== \r\n==13891==  followed by a later acquisition of lock at 0x636BCC8\r\n==13891==    at 0x4A0B069: pthread_mutex_lock (hg_intercepts.c:495)\r\n==13891==    by 0x5EF0E58: mtx_lock (tinycthread.c:135)\r\n==13891==    by 0x5EE9D59: rd_kafka_q_len (rdkafka_queue.h:193)\r\n==13891==    by 0x5EEA45C: rd_kafka_toppar_fetch_decide (rdkafka_partition.c:1245)\r\n==13891==    by 0x5ECDE1E: rd_kafka_broker_thread_main (rdkafka_broker.c:2098)\r\n==13891==    by 0x5EF0AEE: _thrd_wrapper_function (tinycthread.c:596)\r\n==13891==    by 0x4A0C0D4: mythread_wrapper (hg_intercepts.c:219)\r\n==13891==    by 0x3EDEE07A50: start_thread (in /lib64/libpthread-2.12.so)\r\n==13891==    by 0x3EDE6E893C: clone (in /lib64/libc-2.12.so)\r\n==13891==\r\n==13891== Required order was established by acquisition of lock at 0x636BCC8\r\n==13891==    at 0x4A0B069: pthread_mutex_lock (hg_intercepts.c:495)\r\n==13891==    by 0x5EF0E58: mtx_lock (tinycthread.c:135)\r\n==13891==    by 0x5EDB6EF: rd_kafka_q_serve_rkmessages (rdkafka_queue.c:437)\r\n==13891==    by 0x5EC592B: rd_kafka_consume_batch (rdkafka.c:1387)\r\n==13891==    by 0x4C1598C: consume_messages(rd_kafka_s*, rd_kafka_topic_s*, int, long, void (*)(rd_kafka_message_s*, void*)) (PyKafkaClient.cpp:606)\r\n==13891==    by 0x406C33: consume_to_local(void*) (kafka2hdfs.cpp:583)\r\n==13891==    by 0x4A0C0D4: mythread_wrapper (hg_intercepts.c:219)\r\n==13891==    by 0x3EDEE07A50: start_thread (in /lib64/libpthread-2.12.so)\r\n==13891==    by 0x3EDE6E893C: clone (in /lib64/libc-2.12.so)\r\n==13891==\r\n==13891==  followed by a later acquisition of lock at 0x636BC50\r\n==13891==    at 0x4A0B069: pthread_mutex_lock (hg_intercepts.c:495)\r\n==13891==    by 0x5EF0E58: mtx_lock (tinycthread.c:135)\r\n==13891==    by 0x5EDB8F4: rd_kafka_q_serve_rkmessages (rdkafka_offset.h:48)\r\n==13891==    by 0x5EC592B: rd_kafka_consume_batch (rdkafka.c:1387)\r\n==13891==    by 0x4C1598C: consume_messages(rd_kafka_s*, rd_kafka_topic_s*, int, long, void (*)(rd_kafka_message_s*, void*)) (PyKafkaClient.cpp:606)\r\n==13891==    by 0x406C33: consume_to_local(void*) (kafka2hdfs.cpp:583)\r\n==13891==    by 0x4A0C0D4: mythread_wrapper (hg_intercepts.c:219)\r\n==13891==    by 0x3EDEE07A50: start_thread (in /lib64/libpthread-2.12.so)\r\n==13891==    by 0x3EDE6E893C: clone (in /lib64/libc-2.12.so)\r\n\r\nIs that normal?"},"comment":{"url":"https://api.github.com/repos/edenhill/librdkafka/issues/comments/167321936","html_url":"https://github.com/edenhill/librdkafka/issues/464#issuecomment-167321936","issue_url":"https://api.github.com/repos/edenhill/librdkafka/issues/464","id":167321936,"user":{"login":"AppliedSoul","id":14981459,"avatar_url":"https://avatars.githubusercontent.com/u/14981459?v=3","gravatar_id":"","url":"https://api.github.com/users/AppliedSoul","html_url":"https://github.com/AppliedSoul","followers_url":"https://api.github.com/users/AppliedSoul/followers","following_url":"https://api.github.com/users/AppliedSoul/following{/other_user}","gists_url":"https://api.github.com/users/AppliedSoul/gists{/gist_id}","starred_url":"https://api.github.com/users/AppliedSoul/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AppliedSoul/subscriptions","organizations_url":"https://api.github.com/users/AppliedSoul/orgs","repos_url":"https://api.github.com/users/AppliedSoul/repos","events_url":"https://api.github.com/users/AppliedSoul/events{/privacy}","received_events_url":"https://api.github.com/users/AppliedSoul/received_events","type":"User","site_admin":false},"created_at":"2015-12-26T12:45:32Z","updated_at":"2015-12-26T12:45:32Z","body":"Now I am spawning separate threads for each of the partition queried from metadata and creating consumer batch for them individually. This works fine.  Its a pity though that consumer batch queue doesn't work well. "}}