{"action":"created","issue":{"url":"https://api.github.com/repos/zfsonlinux/zfs/issues/4015","labels_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/4015/labels{/name}","comments_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/4015/comments","events_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/4015/events","html_url":"https://github.com/zfsonlinux/zfs/issues/4015","id":117047564,"number":4015,"title":"Bad page state in process ... with v0.6.5-31_gf3e2a7a","user":{"login":"RichardSharpe","id":947462,"avatar_url":"https://avatars.githubusercontent.com/u/947462?v=3","gravatar_id":"","url":"https://api.github.com/users/RichardSharpe","html_url":"https://github.com/RichardSharpe","followers_url":"https://api.github.com/users/RichardSharpe/followers","following_url":"https://api.github.com/users/RichardSharpe/following{/other_user}","gists_url":"https://api.github.com/users/RichardSharpe/gists{/gist_id}","starred_url":"https://api.github.com/users/RichardSharpe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RichardSharpe/subscriptions","organizations_url":"https://api.github.com/users/RichardSharpe/orgs","repos_url":"https://api.github.com/users/RichardSharpe/repos","events_url":"https://api.github.com/users/RichardSharpe/events{/privacy}","received_events_url":"https://api.github.com/users/RichardSharpe/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":15,"created_at":"2015-11-16T03:59:05Z","updated_at":"2015-11-20T02:07:04Z","closed_at":null,"body":"This problem still seems to exist:\r\n\r\nKernel 3.10.0.\r\n\r\n[   63.537440] BUG: Bad page state in process kworker/u8:0  pfn:2d3c14\r\n[   63.537618] page:ffffea000b4f0500 count:0 mapcount:-1 mapping:          (null) index:0x0\r\n[   63.537793] page flags: 0x2fffff00008000(tail)\r\n[   63.538044] page dumped because: nonzero mapcount\r\n[   63.538173] Modules linked in: be2iscsi iscsi_boot_sysfs bnx2i cnic uio cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi zfs(POF) zcommon(POF) znvpair(POF) spl(OF) zavl(POF) zunicode(POF) nf_log_ipv4 nf_log_ipv6 nf_log_common xt_LOG xt_limit binfmt_misc dm_round_robin dm_multipath sg ppdev parport_pc parport serio_raw virtio_balloon virtio_net crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcspkr i2c_piix4 i2c_core zlib wp512 twofish_generic twofish_x86_64 twofish_common tea sha512_ssse3 sha512_generic sha256_ssse3 sha1_mb mcryptd sha1_ssse3 serpent_sse2_x86_64 serpent_avx2 serpent_avx_x86_64 serpent_generic seed salsa20_generic salsa20_x86_64 rmd320 rmd256 rmd160 rmd128 michael_mic md4 khazad ghash_generic gcm fcrypt drbg dm_crypt des_generic deflate zlib_deflate cts crypto_null crct10dif_generic ccm cast6_avx_x86_64 cast6_generic cast_common camellia_generic xts blowfish_generic blowfish_common arc4 ansi_cprng ext4 jbd2 mbcache sd_mod crc_t10dif crct10dif_common virtio_scsi floppy crc32c_intel aesni_intel ablk_helper cryptd lrw gf128mul glue_helper virtio_pci virtio_ring virtio pata_acpi ata_generic ata_piix libata dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip6_tables]\r\n[   63.538235] CPU: 0 PID: 6 Comm: kworker/u8:0 Tainted: PF          O--------------   3.10.0-229.20.1.el6.nutanix.20151109.x86_64 #1\r\n[   63.538236] Hardware name: Red Hat KVM, BIOS seabios-1.7.5-8.el6 04/01/2014\r\n[   63.538241] Workqueue: iscsi_q_3 iscsi_xmitworker [libiscsi]\r\n[   63.538242]  ffffea000b4f0500 0000000075370de6 ffff88032e55b5b0 ffffffff816051ca\r\n[   63.538244]  ffff88032e55b5d8 ffffffff816003e8 ffffea000b4f0500 0000000000000000\r\n[   63.538246]  000fffff00000000 ffff88032e55b620 ffffffff8115eacd fff00000fe000000\r\n[   63.538249] Call Trace:\r\n[   63.538256]  [<ffffffff816051ca>] dump_stack+0x19/0x1b\r\n[   63.538258]  [<ffffffff816003e8>] bad_page.part.59+0xdf/0xfc\r\n[   63.538263]  [<ffffffff8115eacd>] free_pages_prepare+0x16d/0x190\r\n[   63.538265]  [<ffffffff8115eea9>] free_compound_page+0x29/0x40\r\n[   63.538268]  [<ffffffff8116455f>] __put_compound_page+0x1f/0x30\r\n[   63.538270]  [<ffffffff811646f5>] put_compound_page+0x155/0x180\r\n[   63.538271]  [<ffffffff8116474c>] put_page+0x2c/0x40\r\n[   63.538275]  [<ffffffff814ed6f8>] skb_release_data+0x88/0x110\r\n[   63.538277]  [<ffffffff814ed7a4>] skb_release_all+0x24/0x30\r\n[   63.538279]  [<ffffffff814ed9ec>] consume_skb+0x2c/0x80\r\n[   63.538282]  [<ffffffff814fbd3d>] dev_kfree_skb_any+0x3d/0x50\r\n[   63.538285]  [<ffffffffa0342ad6>] free_old_xmit_skbs.isra.27+0x66/0xc0 [virtio_net]\r\n[   63.538288]  [<ffffffffa0342b75>] start_xmit+0x45/0x4a0 [virtio_net]\r\n[   63.538291]  [<ffffffff814fea98>] dev_hard_start_xmit+0x328/0x580\r\n[   63.538295]  [<ffffffff8151e66e>] sch_direct_xmit+0xee/0x1c0\r\n[   63.538297]  [<ffffffff814feee8>] dev_queue_xmit+0x1f8/0x4a0\r\n[   63.538301]  [<ffffffff8153bc1d>] ip_finish_output+0x53d/0x7d0\r\n[   63.538304]  [<ffffffff8153d238>] ip_output+0x58/0x90\r\n[   63.538305]  [<ffffffff8153c9a0>] ip_local_out_sk+0x30/0x40\r\n[   63.538307]  [<ffffffff8153cd01>] ip_queue_xmit+0x141/0x3e0\r\n[   63.538310]  [<ffffffff81554069>] tcp_transmit_skb+0x479/0x900\r\n[   63.538312]  [<ffffffff8155461e>] tcp_write_xmit+0x12e/0xb30\r\n[   63.538314]  [<ffffffff8155528e>] __tcp_push_pending_frames+0x2e/0xc0\r\n[   63.538316]  [<ffffffff81547f64>] tcp_sendmsg+0x124/0xdc0\r\n[   63.538318]  [<ffffffff8153b8d8>] ? ip_finish_output+0x1f8/0x7d0\r\n[   63.538322]  [<ffffffff81570c24>] inet_sendmsg+0x64/0xb0\r\n[   63.538324]  [<ffffffff8126cb63>] ? selinux_socket_sendmsg+0x23/0x30\r\n[   63.538328]  [<ffffffff814e44a0>] sock_sendmsg+0xb0/0xf0\r\n[   63.538330]  [<ffffffff814e7448>] ? sk_reset_timer+0x18/0x30\r\n[   63.538332]  [<ffffffff815539ea>] ? tcp_schedule_loss_probe+0x13a/0x1d0\r\n[   63.538334]  [<ffffffff8155475d>] ? tcp_write_xmit+0x26d/0xb30\r\n[   63.538336]  [<ffffffff814e4517>] kernel_sendmsg+0x37/0x50\r\n[   63.538339]  [<ffffffffa0390559>] iscsi_sw_tcp_pdu_xmit+0xc9/0x2d0 [iscsi_tcp]\r\n[   63.538341]  [<ffffffffa03c951a>] iscsi_tcp_task_xmit+0xaa/0x2d0 [libiscsi_tcp]\r\n[   63.538345]  [<ffffffff810b2afe>] ? dequeue_task_fair+0x41e/0x620\r\n[   63.538346]  [<ffffffff810abfe8>] ? sched_clock_cpu+0xa8/0x100\r\n[   63.538349]  [<ffffffffa0449723>] iscsi_xmit_task+0x53/0xa0 [libiscsi]\r\n[   63.538352]  [<ffffffffa044c070>] iscsi_xmitworker+0x230/0x2c0 [libiscsi]\r\n[   63.538355]  [<ffffffff8108f1fb>] process_one_work+0x17b/0x470\r\n[   63.538357]  [<ffffffff8108ffcb>] worker_thread+0x11b/0x400\r\n[   63.538359]  [<ffffffff8108feb0>] ? rescuer_thread+0x400/0x400\r\n[   63.538361]  [<ffffffff810973af>] kthread+0xcf/0xe0\r\n[   63.538363]  [<ffffffff810972e0>] ? kthread_create_on_node+0x140/0x140\r\n[   63.538367]  [<ffffffff81615018>] ret_from_fork+0x58/0x90\r\n[   63.538368]  [<ffffffff810972e0>] ? kthread_create_on_node+0x140/0x140\r\n\r\nI can get any more info needed."},"comment":{"url":"https://api.github.com/repos/zfsonlinux/zfs/issues/comments/158258558","html_url":"https://github.com/zfsonlinux/zfs/issues/4015#issuecomment-158258558","issue_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/4015","id":158258558,"user":{"login":"RichardSharpe","id":947462,"avatar_url":"https://avatars.githubusercontent.com/u/947462?v=3","gravatar_id":"","url":"https://api.github.com/users/RichardSharpe","html_url":"https://github.com/RichardSharpe","followers_url":"https://api.github.com/users/RichardSharpe/followers","following_url":"https://api.github.com/users/RichardSharpe/following{/other_user}","gists_url":"https://api.github.com/users/RichardSharpe/gists{/gist_id}","starred_url":"https://api.github.com/users/RichardSharpe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RichardSharpe/subscriptions","organizations_url":"https://api.github.com/users/RichardSharpe/orgs","repos_url":"https://api.github.com/users/RichardSharpe/repos","events_url":"https://api.github.com/users/RichardSharpe/events{/privacy}","received_events_url":"https://api.github.com/users/RichardSharpe/received_events","type":"User","site_admin":false},"created_at":"2015-11-20T02:07:04Z","updated_at":"2015-11-20T02:07:04Z","body":"Not all of the Bugs have tcp_sendmsg in the stack. For example:\r\n\r\n[92882.905611] BUG: Bad page state in process java  pfn:1a00ac\r\n[92882.907338] page:ffffea0006802b00 count:0 mapcount:-2 mapping:          (null) index:0x0\r\n[92882.910444] page flags: 0x2fffff00008000(tail)\r\n[92882.911967] page dumped because: nonzero mapcount\r\n[92882.913503] Modules linked in: be2iscsi iscsi_boot_sysfs bnx2i cnic uio cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi zfs(POF) zcommon(POF) znvpair(POF) zavl(POF) spl(OF) zunicode(POF) nf_log_ipv4 nf_log_ipv6 nf_log_common xt_LOG xt_limit binfmt_misc dm_round_robin dm_multipath(F) sg ppdev parport_pc parport serio_raw virtio_balloon virtio_net crct10dif_pclmul crc32_pclmul pcspkr i2c_piix4 i2c_core ghash_clmulni_intel aesni_intel zlib wp512 twofish_generic twofish_x86_64 twofish_common tea sha512_ssse3 sha512_generic sha256_ssse3 mcryptd sha1_ssse3 serpent_sse2_x86_64 serpent_generic seed salsa20_generic salsa20_x86_64 rmd320 rmd256 rmd160 rmd128 michael_mic md4 khazad ghash_generic gcm fcrypt drbg dm_crypt des_generic deflate zlib_deflate cts crypto_null crct10dif_generic ccm cast6_generic ablk_helper cryptd cast_common camellia_generic xts glue_helper lrw gf128mul blowfish_generic blowfish_common arc4 ansi_cprng ext4 jbd2 mbcache sd_mod crc_t10dif crct10dif_common virtio_scsi floppy crc32c_intel virtio_pci virtio_ring virtio pata_acpi ata_generic ata_piix libata dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip6_tables]\r\n[92882.913569] CPU: 1 PID: 3001 Comm: java Tainted: PF   B      O--------------   3.10.0-229.14.1.el6.nutanix.20151021.x86_64 #1\r\n[92882.913570] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011\r\n[92882.913572]  ffffea0006802b00 0000000046a4fb96 ffff880033a2f7d0 ffffffff8160510a\r\n[92882.913575]  ffff880033a2f7f8 ffffffff81600328 ffffea0006802b00 0000000000000000\r\n[92882.913577]  000fffff00000000 ffff880033a2f840 ffffffff8115eacd fff00000fe000000\r\n[92882.913579] Call Trace:\r\n[92882.913587]  [<ffffffff8160510a>] dump_stack+0x19/0x1b\r\n[92882.913590]  [<ffffffff81600328>] bad_page.part.59+0xdf/0xfc\r\n[92882.913594]  [<ffffffff8115eacd>] free_pages_prepare+0x16d/0x190\r\n[92882.913597]  [<ffffffff8115eea9>] free_compound_page+0x29/0x40\r\n[92882.913600]  [<ffffffff8116455f>] __put_compound_page+0x1f/0x30\r\n[92882.913602]  [<ffffffff811646f5>] put_compound_page+0x155/0x180\r\n[92882.913604]  [<ffffffff8116474c>] put_page+0x2c/0x40\r\n[92882.913608]  [<ffffffff814ed638>] skb_release_data+0x88/0x110\r\n[92882.913610]  [<ffffffff814ed6e4>] skb_release_all+0x24/0x30\r\n[92882.913612]  [<ffffffff814ed92c>] consume_skb+0x2c/0x80\r\n[92882.913615]  [<ffffffff814fbc7d>] dev_kfree_skb_any+0x3d/0x50\r\n[92882.913620]  [<ffffffffa030cad6>] free_old_xmit_skbs.isra.27+0x66/0xc0 [virtio_net]\r\n[92882.913623]  [<ffffffffa030cb75>] start_xmit+0x45/0x4a0 [virtio_net]\r\n[92882.913626]  [<ffffffff814fe9d8>] dev_hard_start_xmit+0x328/0x580\r\n[92882.913630]  [<ffffffff8151e5ae>] sch_direct_xmit+0xee/0x1c0\r\n[92882.913633]  [<ffffffff814fee28>] dev_queue_xmit+0x1f8/0x4a0\r\n[92882.913637]  [<ffffffff8153bb5d>] ip_finish_output+0x53d/0x7d0\r\n[92882.913639]  [<ffffffff8153d178>] ip_output+0x58/0x90\r\n[92882.913642]  [<ffffffff8153c8e0>] ip_local_out_sk+0x30/0x40\r\n[92882.913644]  [<ffffffff8153cc41>] ip_queue_xmit+0x141/0x3e0\r\n[92882.913647]  [<ffffffff81553fa9>] tcp_transmit_skb+0x479/0x900\r\n[92882.913649]  [<ffffffff81556d84>] tcp_send_ack+0xa4/0xf0\r\n[92882.913652]  [<ffffffff81544756>] tcp_cleanup_rbuf+0x76/0x120\r\n[92882.913655]  [<ffffffff81545bda>] tcp_recvmsg+0x5fa/0xbb0\r\n[92882.913659]  [<ffffffff81570d0b>] inet_recvmsg+0x7b/0xa0\r\n[92882.913662]  [<ffffffff814e4aaf>] sock_recvmsg+0xbf/0x100\r\n[92882.913667]  [<ffffffff810a696d>] ? ttwu_do_activate.constprop.85+0x5d/0x70\r\n[92882.913669]  [<ffffffff810a9536>] ? try_to_wake_up+0x1b6/0x280\r\n[92882.913673]  [<ffffffff814e4c28>] SYSC_recvfrom+0xe8/0x160\r\n[92882.913676]  [<ffffffff814e540e>] SyS_recvfrom+0xe/0x10\r\n[92882.913680]  [<ffffffff81615009>] system_call_fastpath+0x16/0x1b\r\n"}}