{"action":"closed","issue":{"url":"https://api.github.com/repos/ntop/PF_RING/issues/47","labels_url":"https://api.github.com/repos/ntop/PF_RING/issues/47/labels{/name}","comments_url":"https://api.github.com/repos/ntop/PF_RING/issues/47/comments","events_url":"https://api.github.com/repos/ntop/PF_RING/issues/47/events","html_url":"https://github.com/ntop/PF_RING/issues/47","id":112601226,"number":47,"title":"Kernel Panic w/ 6.0.3 on CentOS 7.1","user":{"login":"dcode","id":171574,"avatar_url":"https://avatars.githubusercontent.com/u/171574?v=3","gravatar_id":"","url":"https://api.github.com/users/dcode","html_url":"https://github.com/dcode","followers_url":"https://api.github.com/users/dcode/followers","following_url":"https://api.github.com/users/dcode/following{/other_user}","gists_url":"https://api.github.com/users/dcode/gists{/gist_id}","starred_url":"https://api.github.com/users/dcode/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dcode/subscriptions","organizations_url":"https://api.github.com/users/dcode/orgs","repos_url":"https://api.github.com/users/dcode/repos","events_url":"https://api.github.com/users/dcode/events{/privacy}","received_events_url":"https://api.github.com/users/dcode/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-10-21T14:25:11Z","updated_at":"2015-10-21T19:31:26Z","closed_at":"2015-10-21T19:31:25Z","body":"I thought this might be a duplicate of #28 but this is on 6.0.3 and happens even if I remove the pf_ring kernel module. Wether I use bro (which I compiled against the RPM-provided libpcap) or tcpdump (from the CentOS repos), the dynamic linker pushes them through `/usr/local/lib/libpcap.so.1.6.2`. When I rename that shared lib, tcpdump uses the system libpcap and it works just fine, whether the module is loaded or not.\r\n\r\nI've tried running 6.1.1 on this kernel, which _doesn't_ crash, but I get terrible packet loss when going through libpcap, but that is a different issue (that I'll file when I get a chance to run it to ground).\r\n\r\nSo the culprit seems to be libpcap 1.6.2. I'd like to stick to PF_RING 6.0.3 if I can for stability and performance. I'm happy to provide more info or dig into crash dumps more if needed.\r\n\r\nSystem Info (from crash tool)\r\n~~~~~~~~~~~~~~~~~~\r\nThis GDB was configured as \"x86_64-unknown-linux-gnu\"...\r\n\r\n      KERNEL: /usr/lib/debug/lib/modules/3.10.0-229.el7.x86_64/vmlinux\r\n    DUMPFILE: /var/crash/127.0.0.1-2015.10.20-22:22:58/vmcore  [PARTIAL DUMP]\r\n        CPUS: 24\r\n        DATE: Tue Oct 20 22:22:51 2015\r\n      UPTIME: 00:16:49\r\nLOAD AVERAGE: 0.00, 0.01, 0.04\r\n       TASKS: 355\r\n    NODENAME: reciever2\r\n     RELEASE: 3.10.0-229.el7.x86_64\r\n     VERSION: #1 SMP Fri Mar 6 11:36:42 UTC 2015\r\n     MACHINE: x86_64  (2300 Mhz)\r\n      MEMORY: 128 GB\r\n       PANIC: \"\"\r\n         PID: 35\r\n     COMMAND: \"rcuos/0\"\r\n        TASK: ffff8810297a8b60  [THREAD_INFO: ffff8810297b4000]\r\n         CPU: 1\r\n       STATE: TASK_RUNNING (PANIC)\r\n~~~~~~~~~~~~~~~~~~~\r\n\r\ndmesg\r\n~~~~~~~~~~~~~~~~~~\r\n[  691.218347] [PF_RING] Module unloaded\r\n[  843.251621] [PF_RING] Welcome to PF_RING 6.0.3 ($Revision: 6.0.3-stable:8994076d9761315040ed29a0d5825cb74c20c078$)\r\n               (C) 2004-14 ntop.org\r\n[  843.251654] [PF_RING] registered /proc/net/pf_ring/\r\n[  843.251656] NET: Registered protocol family 27\r\n[  843.251666] [PF_RING] Min # ring slots 4096\r\n[  843.251667] [PF_RING] Slot version     16\r\n[  843.251668] [PF_RING] Capture TX       Yes [RX+TX]\r\n[  843.251669] [PF_RING] IP Defragment    No\r\n[  843.251670] [PF_RING] Initialized correctly\r\n[ 1008.709269] device eno3 entered promiscuous mode\r\n[ 1008.727969] general protection fault: 0000 [#1] SMP\r\n[ 1008.727998] Modules linked in: pf_ring(OF) xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter ip_tables nf_nat nf_conntrack bridge stp llc dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop intel_powerclamp coretemp iTCO_wdt iTCO_vendor_support intel_rapl kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sb_edac ses edac_core serio_raw pcspkr mei_me enclosure mei lpc_ich i2c_i801 mfd_core ipmi_si ipmi_msghandler ioatdma ntb shpchp wmi xfs libcrc32c sd_mod crc_t10dif crct10dif_common mgag200 syscopyarea sysfillrect sysimgblt drm_kms_helper ttm isci igb drm ixgbe(OF) vxlan libsas ahci ip_tunnel ptp libahci scsi_transport_sas\r\n[ 1008.728336]  pps_core dca libata i2c_algo_bit aacraid i2c_core dm_mirror dm_region_hash dm_log dm_mod [last unloaded: pf_ring]\r\n[ 1008.728386] CPU: 1 PID: 35 Comm: rcuos/0 Tainted: GF          O--------------   3.10.0-229.el7.x86_64 #1\r\n[ 1008.728421] Hardware name: Supermicro X9DRW-3LN4F+/X9DRW-3TF+/X9DRW-3LN4F+/X9DRW-3TF+, BIOS 3.0a 02/06/2014\r\n[ 1008.728458] task: ffff8810297a8b60 ti: ffff8810297b4000 task.ti: ffff8810297b4000\r\n[ 1008.728485] RIP: 0010:[<ffffffff8106a802>]  [<ffffffff8106a802>] bpf_jit_free+0x22/0x50\r\n[ 1008.728520] RSP: 0018:ffff8810297b7df0  EFLAGS: 00010203\r\n[ 1008.728541] RAX: 0000000fffffffe0 RBX: ffff882014f90280 RCX: dead000000200200\r\n[ 1008.728567] RDX: 636e753a755f6465 RSI: 0000000000000283 RDI: 0000000000001400\r\n[ 1008.728594] RBP: ffff8810297b7e08 R08: ffff8810297b7e80 R09: 0000000000000008\r\n[ 1008.728620] R10: 0000000000000004 R11: 0000000000000005 R12: 0000000000000000\r\n[ 1008.728646] R13: 0000000000000000 R14: 0000000000000000 R15: ffff882014f90290\r\n[ 1008.728673] FS:  0000000000000000(0000) GS:ffff88103fc20000(0000) knlGS:0000000000000000\r\n[ 1008.728703] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033\r\n[ 1008.728724] CR2: 00007fa4edc8a000 CR3: 000000000190a000 CR4: 00000000000407e0\r\n[ 1008.728751] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000\r\n[ 1008.728777] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400\r\n[ 1008.728803] Stack:\r\n[ 1008.728813]  ffff8810297b7e08 ffffffff81510406 ffff882014f90290 ffff8810297b7ec0\r\n[ 1008.728847]  ffffffff81113239 ffff8810297a8b60 ffff8810297a8b60 ffff88103fc0df50\r\n[ 1008.728880]  ffff8810297b7e80 ffff8810297a8b60 ffff88103fc0df28 ffff88103fc0df38\r\n[ 1008.728913] Call Trace:\r\n[ 1008.728928]  [<ffffffff81510406>] ? sk_filter_release_rcu+0x16/0x30\r\n[ 1008.728955]  [<ffffffff81113239>] rcu_nocb_kthread+0x229/0x370\r\n[ 1008.728979]  [<ffffffff81098350>] ? wake_up_bit+0x30/0x30\r\n[ 1008.729002]  [<ffffffff81113010>] ? rcu_start_gp+0x40/0x40\r\n[ 1008.729023]  [<ffffffff8109739f>] kthread+0xcf/0xe0\r\n[ 1008.729044]  [<ffffffff810972d0>] ? kthread_create_on_node+0x140/0x140\r\n[ 1008.729071]  [<ffffffff8161497c>] ret_from_fork+0x7c/0xb0\r\n[ 1008.729092]  [<ffffffff810972d0>] ? kthread_create_on_node+0x140/0x140\r\n[ 1008.729117] Code: 84 e9 94 fa ff ff 0f 1f 00 66 66 66 66 90 48 8b 57 08 48 81 fa b0 07 51 81 74 37 48 b8 e0 ff ff ff 0f 00 00 00 55 bf 00 14 00 00 <48> 89 02 48 8d 42 08 48 8b 35 58 74 9b 00 48 89 e5 48 c7 42 18\r\n[ 1008.729246] RIP  [<ffffffff8106a802>] bpf_jit_free+0x22/0x50\r\n[ 1008.729270]  RSP <ffff8810297b7df0>\r\n~~~~~~~~~~~~~~~~~~\r\n\r\nBacktrace\r\n~~~~~~~~~~~~~~~~~~\r\nPID: 35     TASK: ffff8810297a8b60  CPU: 1   COMMAND: \"rcuos/0\"\r\n #0 [ffff8810297b7b90] machine_kexec at ffffffff8104c681\r\n #1 [ffff8810297b7be8] crash_kexec at ffffffff810e2222\r\n #2 [ffff8810297b7cb8] oops_end at ffffffff8160d188\r\n #3 [ffff8810297b7ce0] die at ffffffff810173eb\r\n #4 [ffff8810297b7d10] do_general_protection at ffffffff8160ca8e\r\n #5 [ffff8810297b7d40] general_protection at ffffffff8160c3a8\r\n    [exception RIP: bpf_jit_free+34]\r\n    RIP: ffffffff8106a802  RSP: ffff8810297b7df0  RFLAGS: 00010203\r\n    RAX: 0000000fffffffe0  RBX: ffff882014f90280  RCX: dead000000200200\r\n    RDX: 636e753a755f6465  RSI: 0000000000000283  RDI: 0000000000001400\r\n    RBP: ffff8810297b7e08   R8: ffff8810297b7e80   R9: 0000000000000008\r\n    R10: 0000000000000004  R11: 0000000000000005  R12: 0000000000000000\r\n    R13: 0000000000000000  R14: 0000000000000000  R15: ffff882014f90290\r\n    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018\r\n #6 [ffff8810297b7df8] sk_filter_release_rcu at ffffffff81510406\r\n #7 [ffff8810297b7e10] rcu_nocb_kthread at ffffffff81113239\r\n #8 [ffff8810297b7ec8] kthread at ffffffff8109739f\r\n #9 [ffff8810297b7f50] ret_from_fork at ffffffff8161497c\r\n~~~~~~~~~~~~~~~~~~~~"}}