{"action":"opened","issue":{"url":"https://api.github.com/repos/natsys/tempesta/issues/109","labels_url":"https://api.github.com/repos/natsys/tempesta/issues/109/labels{/name}","comments_url":"https://api.github.com/repos/natsys/tempesta/issues/109/comments","events_url":"https://api.github.com/repos/natsys/tempesta/issues/109/events","html_url":"https://github.com/natsys/tempesta/issues/109","id":88214054,"number":109,"title":"Crash in tfw_http_msg_free() when making an HTTP request","user":{"login":"keshonok","id":6352580,"avatar_url":"https://avatars.githubusercontent.com/u/6352580?v=3","gravatar_id":"","url":"https://api.github.com/users/keshonok","html_url":"https://github.com/keshonok","followers_url":"https://api.github.com/users/keshonok/followers","following_url":"https://api.github.com/users/keshonok/following{/other_user}","gists_url":"https://api.github.com/users/keshonok/gists{/gist_id}","starred_url":"https://api.github.com/users/keshonok/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/keshonok/subscriptions","organizations_url":"https://api.github.com/users/keshonok/orgs","repos_url":"https://api.github.com/users/keshonok/repos","events_url":"https://api.github.com/users/keshonok/events{/privacy}","received_events_url":"https://api.github.com/users/keshonok/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-06-14T16:51:12Z","updated_at":"2015-06-14T16:51:12Z","closed_at":null,"body":"This is done on top of ab-startfix and ab-locks branches, as current master branch is not quite functional yet.\r\n\r\nPut a server outside of any group in Tempesta configuration, so that requests are forwarded to that server. Cache is enabled by default now. Start Tempesta, and make an HTTP request with `wget`. Get the crash.\r\n\r\nThe trace shows that the crash happens when the response is received, processed by Tempesta, and sent out to the client.\r\n\r\n```\r\nStarting nginx: nginx.\r\nLoading Tempesta kernel modules...\r\n[   12.552633] [tdb] Start Tempesta DB\r\n[   12.723310] [tempesta] Initializing Tempesta FW kernel module...\r\n[   12.723995] [tempesta] init: cfg_if\r\n[   12.724423] [tempesta] init: tls\r\n[   12.724783] [tempesta] init: http\r\n[   12.725149] [tempesta] init: http_sticky\r\n[   12.727493] [tempesta] init: server\r\n[   12.727973] [tempesta] init: client\r\n[   12.728438] [tempesta] init: sock_srv\r\n[   12.728932] [tempesta] init: sock_clnt\r\n[   12.729430] [tempesta]   register cfg: filter\r\n[   12.729993] [tempesta]   register cfg: cache\r\n[   12.730545] [tempesta]   register cfg: http_sticky\r\n[   12.731168] [tempesta]   register cfg: sock_srv\r\n[   12.731722] [tempesta]   register cfg: sock_clnt\r\n[   12.745588] [tempesta] Registering new scheduler: dummy\r\n[   12.763482] [tempesta]   tfw_sched_hash: init\r\n[   12.763935] [tempesta] Registering new scheduler: hash\r\n[   12.777398] [tempesta]   tfw_sched_http: init\r\n[   12.777875] [tempesta]   register cfg: tfw_sched_http\r\n[   12.778347] [tempesta] Registering new scheduler: http\r\n[   12.794597] [tempesta]   tfw_sched_rr: init\r\n[   12.795150] [tempesta] Registering new scheduler: round-robin\r\nStarting Tempesta...\r\n[   12.800068] [tempesta] got state via sysctl: start\r\n[   12.800750] [tempesta]   reading configuration file...\r\n[   12.801472] [tempesta]   reading file: /home/natsys-lab/tfw-run/tempesta/etc/tempesta_fw.conf\r\n[   12.803368] [tempesta]   file size: 5772 bytes\r\n[   12.803875] [tempesta]   read by offset: 0\r\n[   12.804329] [tempesta]   read by offset: 4096\r\n[   12.804793] [tempesta]   read by offset: 5772\r\n[   12.805233] [tempesta] starting all modules...\r\n[   12.805643] [tempesta]   parsing configuration and pushing it to modules...\r\n[   12.806312] [tempesta]   spec handle: 'server'\r\n[   12.806746] [tempesta]   new server group: 'default'\r\n[   12.807344] [tempesta]   use default entry: 'filter_tbl_size 16777216;'\r\n[   12.807971] [tempesta]   spec handle: 'filter_tbl_size'\r\n[   12.808466] [tempesta]   use default entry: 'filter_db /opt/tempesta/db/filter.tdb;'\r\n[   12.809204] [tempesta]   spec handle: 'filter_db'\r\n[   12.809638] [tempesta]   use default entry: 'cache on;'\r\n[   12.810126] [tempesta]   spec handle: 'cache'\r\n[   12.810535] [tempesta]   use default entry: 'cache_size 268435456;'\r\n[   12.811113] [tempesta]   spec handle: 'cache_size'\r\n[   12.811573] [tempesta]   use default entry: 'cache_db /opt/tempesta/db/cache.tdb;'\r\n[   12.812280] [tempesta]   spec handle: 'cache_db'\r\n[   12.812710] [tempesta]   use default entry: 'listen 80;'\r\n[   12.813205] [tempesta]   spec handle: 'listen'\r\n[   12.813621] [tempesta]   starting modules...\r\n[   12.814020] [tempesta]   mod_start(): filter\r\n[   12.820889] [tdb] Opened table /opt/tempesta/db/filter.tdb: size=16777216 rec_size=20 base=ffff880033000000\r\n[   12.821826] [tempesta]   mod_start(): cache\r\n[   13.057428] [tdb] Opened table /opt/tempesta/db/cache.tdb: size=268435456 rec_size=0 base=ffff880023000000\r\n[   13.058836] [tempesta]   mod_start(): http_sticky\r\n[   13.059446] [tempesta]   mod_start(): sock_srv\r\n[   13.060430] [tempesta]   mod_start(): sock_clnt\r\n[   13.061138] [sync_sockets]   ss_tcp_state_change: sk ffff880038bd4940, sk->sk_socket           (null), state (Established)\r\n[   13.062545] [tempesta]   connected: 192.168.10.1:8080\r\n[   13.063230] [sync_sockets]   ss_tcp_state_change: sk ffff88003867f280, sk->sk_socket           (null), state (Established)\r\n[   13.064639] [tempesta]   connected: 192.168.10.1:8080\r\n[   13.065313] [sync_sockets]   ss_tcp_state_change: sk ffff88003867e980, sk->sk_socket           (null), state (Established)\r\n[   13.066738] [tempesta]   connected: 192.168.10.1:8080\r\n[   13.067388] [sync_sockets]   ss_tcp_state_change: sk ffff88003867e080, sk->sk_socket           (null), state (Established)\r\n[   13.068824] [tempesta]   connected: 192.168.10.1:8080\r\n[   13.069489] [tempesta] Open listen socket on: 0.0.0.0\r\n[   13.070202] [tempesta]   start listening on socket: sk=ffff8800385992c0\r\n[   13.071064] [tempesta]   mod_start(): tfw_sched_http\r\n[   13.071709] [tempesta] modules are started\r\nnet.tempesta.state = start\r\ndone\r\ndone\r\n[   84.757239] [sync_sockets]   ss_tcp_state_change: sk ffff8800385980c0, sk->sk_socket           (null), state (Established)\r\n[   84.760840] [tempesta]   new client socket: sk=ffff8800385980c0, state=1\r\n[   84.760840] [tempesta]   new client: cli=ffff8800389f4a70\r\n[   84.760840] [tempesta]   new client socket is accepted: sk=ffff8800385980c0, conn=ffff8800384aa6e8, cli=ffff8800389f4a70\r\n[   84.760840] [sync_sockets]   ss_drain_accept_queue: sk ffff8800385992c0, sk->sk_socket           (null), state (Listen)\r\n[   84.760840] [sync_sockets]   ss_tcp_data_ready: sk ffff8800385980c0, sk->sk_socket           (null), state (Established)\r\n[   84.760840] [tempesta]   Link new msg ffff880038b97040 with connection ffff8800384aa6e8\r\n[   84.760840] [tempesta]   Add skb ffff8800389e13c0 to message ffff880038b97040\r\n[   84.760840] [tempesta]   Received 62 client data bytes (GET / HTTP/1.1\r\n[   84.760840] User-Agent: Wget/1.15 (linux-gnu)\r\n[   84.760840] Accept: */*) on socket (conn=ffff8800384aa6e8)\r\n[   84.760840] [tempesta]   enter FSM at state 0\r\n[   84.760840] [tempesta]   parser: Req_0(0:0): c=0x47(G), r=-2\r\n[   84.760840] [tempesta]   parser: Req_Method(1:0): c=0x47(G), r=-2\r\n[   84.760840] [tempesta]   parser: Req_MUSpace(2:0): c=0x2f(/), r=-2\r\n[   84.760840] [tempesta]   parser: Req_UriAbsPath(6:0): c=0x20( ), r=-2\r\n[   84.760840] [tempesta]   parser: Req_HttpVer(7:0): c=0x48(H), r=-2\r\n[   84.760840] [tempesta]   parser: Req_EoL(15:0): c=0xd(.), r=-2\r\n[   84.760840] [tempesta]   parser: Req_EoL(15:0): c=0xa(.), r=-2\r\n[   84.760840] [tempesta]   parser: Req_Hdr(16:0): c=0x55(U), r=-2\r\n[   84.760840] [tempesta]   parser: Req_HdrOther(92:0): c=0x55(U), r=-2\r\n[   84.760840] [tempesta]   store header w/ ptr=ffff880038c82c92 len=33 flags=0 id=3\r\n[   84.760840] [tempesta]   parser: Req_Hdr(16:0): c=0x41(A), r=-2\r\n[   84.760840] [tempesta]   parser: Req_HdrOther(92:0): c=0x41(A), r=-2\r\n[   84.760840] [tempesta]   store header w/ ptr=ffff880038c82cb5 len=11 flags=20 id=4\r\n[   84.760840] [tempesta]   request parsed: len=62 parsed=62 msg_len=62 res=-1\r\n[   84.760840] [tempesta]   GFSM return code 0\r\n[   84.760840] [tempesta]   postpone skb ffff8800389e13c0\r\n[   84.760840] [tempesta]   Received 50 client data bytes (\r\n[   84.760840] Host: 192.168.10.230\r\n[   84.760840] Connection: Keep-Alive\r\n[   84.760840] \r\n[   84.760840] ) on socket (conn=ffff8800384aa6e8)\r\n[   84.760840] [tempesta]   enter FSM at state 92\r\n[   84.760840] [tempesta]   parser: Req_HdrOther(92:0): c=0xd(.), r=-2\r\n[   84.760840] [tempesta]   store header w/ ptr=          (null) len=0 flags=0 id=4\r\n[   84.760840] [tempesta]   parser: Req_Hdr(16:0): c=0x48(H), r=-2\r\n[   84.760840] [tempesta]   parser: RGen_LWS(10000:21): c=0x20( ), r=-2\r\n[   84.760840] [tempesta]   parser: RGen_LWS(10000:21): c=0x31(1), r=-2\r\n[   84.760840] [tempesta]   enter FSM at state 21\r\n[   84.760840] [tempesta]   parser: Req_HdrHostV(21:0): c=0x31(1), r=-2\r\n[   84.760840] [tempesta]   enter FSM at state 1\r\n[   84.760840] [tempesta]   parser: Req_I_H(1:1): c=0x31(1), r=-2\r\n[   84.760840] [tempesta]   parser: Req_I_H(1:1): c=0x39(9), r=-2\r\n[   84.760840] [tempesta]   parser: Req_I_H(1:1): c=0x32(2), r=-2\r\n[   84.760840] [tempesta]   parser: Req_I_H(1:1): c=0x2e(.), r=-2\r\n[   84.760840] [tempesta]   parser: Req_I_H(1:1): c=0x31(1), r=-2\r\n[   84.760840] [tempesta]   parser: Req_I_H(1:1): c=0x36(6), r=-2\r\n[   84.760840] [tempesta]   parser: Req_I_H(1:1): c=0x38(8), r=-2\r\n[   84.760840] [tempesta]   parser: Req_I_H(1:1): c=0x2e(.), r=-2\r\n[   84.760840] [tempesta]   parser: Req_I_H(1:1): c=0x31(1), r=-2\r\n[   84.760840] [tempesta]   parser: Req_I_H(1:1): c=0x30(0), r=-2\r\n[   84.760840] [tempesta]   parser: Req_I_H(1:1): c=0x2e(.), r=-2\r\n[   84.760840] [tempesta]   parser: Req_I_H(1:1): c=0x32(2), r=-2\r\n[   84.760840] [tempesta]   parser: Req_I_H(1:1): c=0x33(3), r=-2\r\n[   84.760840] [tempesta]   parser: Req_I_H(1:1): c=0x30(0), r=-2\r\n[   84.760840] [tempesta]   parser: Req_I_H(1:1): c=0xd(.), r=-2\r\n[   84.760840] [tempesta]   parser: Req_I_H_EoL(3:1): c=0xd(.), r=-2\r\n[   84.760840] [tempesta]   parser: Req_I_H_EoL(3:1): c=0xa(.), r=-2\r\n[   84.760840] [tempesta]   parse header __req_parse_host: ret=16 plen=14 id=1\r\n[   84.760840] [tempesta]   store header w/ ptr=ffff88003834208e len=20 flags=0 id=1\r\n[   84.760840] [tempesta]   parser: Req_Hdr(16:0): c=0x43(C), r=-2\r\n[   84.760840] [tempesta]   parser: Req_HdrC(22:0): c=0x6f(o), r=-2\r\n[   84.760840] [tempesta]   parser: Req_HdrConnection(44:0): c=0x3a(:), r=-2\r\n[   84.760840] [tempesta]   parser: RGen_LWS(10000:45): c=0x20( ), r=-2\r\n[   84.760840] [tempesta]   parser: RGen_LWS(10000:45): c=0x4b(K), r=-2\r\n[   84.760840] [tempesta]   enter FSM at state 45\r\n[   84.760840] [tempesta]   parser: Req_HdrConnectionV(45:0): c=0x4b(K), r=-2\r\n[   84.760840] [tempesta]   enter FSM at state 1\r\n[   84.760840] [tempesta]   parser: I_Conn(1:1): c=0x4b(K), r=-2\r\n[   84.760840] [tempesta]   parser: I_EoT(6:1): c=0xd(.), r=0\r\n[   84.760840] [tempesta]   parser: I_EoL(7:1): c=0xd(.), r=0\r\n[   84.760840] [tempesta]   parser: I_EoL(7:1): c=0xa(.), r=0\r\n[   84.760840] [tempesta]   parser: Connection parsed: flags 0x2\r\n[   84.760840] [tempesta]   parse header __parse_connection: ret=12 plen=10 id=0\r\n[   84.760840] [tempesta]   store header w/ ptr=ffff8800383420a4 len=22 flags=0 id=0\r\n[   84.760840] [tempesta]   parser: Req_Hdr(16:0): c=0xd(.), r=-2\r\n[   84.760840] [tempesta]   parser: Req_HdrDone(93:0): c=0xa(.), r=-2\r\n[   84.760840] [tempesta]   parse msg body: flags=0x2 content_length=0\r\n[   84.760840] [tempesta]   request parsed: len=50 parsed=50 msg_len=50 res=0\r\n[   84.760840] [tempesta]   GFSM return code 0\r\n[   84.760840] [tempesta]   Matching request: ffff880038b97040, list: ffff880038697040\r\n[   84.760840] [tempesta]   tfw_sched_http: use server group: 'default'\r\n[   84.760840] [sync_sockets]   ss_send: sk ffff88003867f280, sk->sk_socket           (null), state (Established)\r\n[   84.760840] \r\n[   84.760840] =============================================\r\n[   84.760840] [ INFO: possible recursive locking detected ]\r\n[   84.760840] 3.10.10-tempesta #2 Tainted: G           O\r\n[   84.760840] ---------------------------------------------\r\n[   84.760840] swapper/0/0 is trying to acquire lock:\r\n[   84.760840]  (slock-AF_INET/1){+.-...}, at: [<ffffffffa004e0fd>] ss_send+0xb4/0x2e6 [tempesta_fw]\r\n[   84.760840] \r\n[   84.760840] but task is already holding lock:\r\n[   84.760840]  (slock-AF_INET/1){+.-...}, at: [<ffffffff8134b04c>] tcp_v4_rcv+0x40c/0x8b0\r\n[   84.760840] \r\n[   84.760840] other info that might help us debug this:\r\n[   84.760840]  Possible unsafe locking scenario:\r\n[   84.760840] \r\n[   84.760840]        CPU0\r\n[   84.760840]        ----\r\n[   84.760840]   lock(slock-AF_INET/1);\r\n[   84.760840]   lock(slock-AF_INET/1);\r\n[   84.760840] \r\n[   84.760840]  *** DEADLOCK ***\r\n[   84.760840] \r\n[   84.760840]  May be due to missing lock nesting notation\r\n[   84.760840] \r\n[   84.760840] 4 locks held by swapper/0/0:\r\n[   84.760840]  #0:  (rcu_read_lock){.+.+..}, at: [<ffffffff812ee0f3>] __netif_receive_skb_core+0x63/0x590\r\n[   84.760840]  #1:  (rcu_read_lock){.+.+..}, at: [<ffffffff813253e0>] ip_local_deliver_finish+0x40/0x150\r\n[   84.760840]  #2:  (slock-AF_INET/1){+.-...}, at: [<ffffffff8134b04c>] tcp_v4_rcv+0x40c/0x8b0\r\n[   84.760840]  #3:  (clock-AF_INET){++.-..}, at: [<ffffffffa004e359>] ss_tcp_process_proto_skb+0x2a/0xf2 [tempesta_fw]\r\n[   84.760840] \r\n[   84.760840] stack backtrace:\r\n[   84.760840] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G           O 3.10.10-tempesta #2\r\n[   84.760840]  ffffffff81565440 ffff88003fc03788 ffffffff813c5eed ffff88003fc037f8\r\n[   84.760840]  ffffffff81075866 ffffffffa0059969 ffff88003fc03850 ffffffff811fc527\r\n[   84.760840]  ffffffff00000000 01600b0958442256 ffffffff00000000 ffffffff817022a0\r\n[   84.760840] Call Trace:\r\n[   84.760840]  <IRQ>  [<ffffffff813c5eed>] dump_stack+0x19/0x1b\r\n[   84.760840]  [<ffffffff81075866>] __lock_acquire+0xfa6/0x1240\r\n[   84.760840]  [<ffffffff811fc527>] ? __dynamic_pr_debug+0x97/0xc0\r\n[   84.760840]  [<ffffffff81076328>] lock_acquire+0x98/0xd0\r\n[   84.760840]  [<ffffffffa004e0fd>] ? ss_send+0xb4/0x2e6 [tempesta_fw]\r\n[   84.760840]  [<ffffffff813c9c5a>] _raw_spin_lock_nested+0x3a/0x50\r\n[   84.760840]  [<ffffffffa004e0fd>] ? ss_send+0xb4/0x2e6 [tempesta_fw]\r\n[   84.760840]  [<ffffffffa004e0fd>] ss_send+0xb4/0x2e6 [tempesta_fw]\r\n[   84.760840]  [<ffffffff811fc527>] ? __dynamic_pr_debug+0x97/0xc0\r\n[   84.760840]  [<ffffffffa0011e6c>] tfw_connection_send+0x2e/0x30 [tempesta_fw]\r\n[   84.760840]  [<ffffffffa00169f3>] tfw_http_req_cache_cb+0x5d/0x5f [tempesta_fw]\r\n[   84.760840]  [<ffffffffa000dd01>] __cache_req_process_node+0x83/0xa4 [tempesta_fw]\r\n[   84.760840]  [<ffffffffa0016996>] ? tfw_http_adjust_resp+0x13/0x13 [tempesta_fw]\r\n[   84.760840]  [<ffffffffa000ded4>] tfw_cache_req_process+0x15c/0x163 [tempesta_fw]\r\n[   84.760840]  [<ffffffffa0016996>] ? tfw_http_adjust_resp+0x13/0x13 [tempesta_fw]\r\n[   84.760840]  [<ffffffffa0016d36>] tfw_http_req_process+0x341/0x3b0 [tempesta_fw]\r\n[   84.760840]  [<ffffffff811fc527>] ? __dynamic_pr_debug+0x97/0xc0\r\n[   84.760840]  [<ffffffffa0017234>] tfw_http_msg_process+0x43/0x5e [tempesta_fw]\r\n[   84.760840]  [<ffffffffa001311f>] tfw_gfsm_dispatch+0x41/0x43 [tempesta_fw]\r\n[   84.760840]  [<ffffffffa0011ea8>] tfw_connection_recv+0x3a/0x3c [tempesta_fw]\r\n[   84.760840]  [<ffffffffa004e3a3>] ss_tcp_process_proto_skb+0x74/0xf2 [tempesta_fw]\r\n[   84.760840]  [<ffffffffa004e5c9>] ss_tcp_process_skb+0x1a8/0x270 [tempesta_fw]\r\n[   84.760840]  [<ffffffffa004f093>] ss_tcp_process_data+0x12c/0x2bb [tempesta_fw]\r\n[   84.760840]  [<ffffffffa004f432>] ss_tcp_data_ready+0xd1/0x149 [tempesta_fw]\r\n[   84.760840]  [<ffffffff8133beed>] tcp_data_queue+0x4dd/0xd10\r\n[   84.760840]  [<ffffffff8133f2f0>] tcp_rcv_established+0x560/0x620\r\n[   84.760840]  [<ffffffff813494cc>] tcp_v4_do_rcv+0x1dc/0x4e0\r\n[   84.760840]  [<ffffffff811f66b2>] ? do_raw_spin_lock+0x82/0x140\r\n[   84.760840]  [<ffffffff8134b077>] tcp_v4_rcv+0x437/0x8b0\r\n[   84.760840]  [<ffffffff81325455>] ip_local_deliver_finish+0xb5/0x150\r\n[   84.760840]  [<ffffffff813253e0>] ? ip_local_deliver_finish+0x40/0x150\r\n[   84.760840]  [<ffffffff81325c00>] ip_local_deliver+0x50/0x90\r\n[   84.760840]  [<ffffffff8132596e>] ip_rcv_finish+0x47e/0x580\r\n[   84.760840]  [<ffffffff81325efc>] ip_rcv+0x2bc/0x340\r\n[   84.760840]  [<ffffffff812ee4f9>] __netif_receive_skb_core+0x469/0x590\r\n[   84.760840]  [<ffffffff812ee0f3>] ? __netif_receive_skb_core+0x63/0x590\r\n[   84.760840]  [<ffffffff812ee66f>] __netif_receive_skb+0x4f/0x70\r\n[   84.760840]  [<ffffffff812ef1d3>] netif_receive_skb+0xb3/0x140\r\n[   84.760840]  [<ffffffff812e1f09>] ? __netdev_alloc_skb+0x49/0xb0\r\n[   84.760840]  [<ffffffff81278c44>] virtnet_poll+0x624/0x770\r\n[   84.760840]  [<ffffffff812f0413>] net_rx_action+0xa3/0x1f0\r\n[   84.760840]  [<ffffffff81038220>] __do_softirq+0xe0/0x1d0\r\n[   84.760840]  [<ffffffff81038499>] irq_exit+0x69/0xc0\r\n[   84.760840]  [<ffffffff81003c5d>] do_IRQ+0x9d/0xc0\r\n[   84.760840]  [<ffffffff813ca972>] common_interrupt+0x72/0x72\r\n[   84.760840]  <EOI>  [<ffffffff81023626>] ? native_safe_halt+0x6/0x10\r\n[   84.760840]  [<ffffffff8100a8dd>] default_idle+0x1d/0x30\r\n[   84.760840]  [<ffffffff8100b26a>] arch_cpu_idle+0xa/0x10\r\n[   84.760840]  [<ffffffff8106817c>] cpu_startup_entry+0x15c/0x200\r\n[   84.760840]  [<ffffffff813be53d>] rest_init+0xcd/0xe0\r\n[   84.760840]  [<ffffffff813be470>] ? csum_partial_copy_generic+0x170/0x170\r\n[   84.760840]  [<ffffffff815dada5>] start_kernel+0x352/0x35d\r\n[   84.760840]  [<ffffffff815da85c>] ? repair_env_string+0x5e/0x5e\r\n[   84.760840]  [<ffffffff815da120>] ? early_idt_handlers+0x120/0x120\r\n[   84.760840]  [<ffffffff815da55b>] x86_64_start_reservations+0x2a/0x2c\r\n[   84.760840]  [<ffffffff815da64c>] x86_64_start_kernel+0xef/0xfc\r\n[   84.760840] [sync_sockets]   ss_send:167 entail skb=ffff8800389e13c0 data_len=50 len=112\r\n[   84.760840] [sync_sockets]   ss_send:185 sk=ffff88003867f280 is_queue_empty=0 tcp_send_head(sk)=ffff8800389e13c0 sk->sk_state=1\r\n[   84.760840] [tempesta]   Free msg: ffff880038b97040\r\n[   84.760840] [tempesta]   free skb ffff8800389e13c0: truesize=4864 sk=ffff8800385980c0, destructor=ffffffff812dab30 users=1 type=Conn_Clnt\r\n[   84.760840] [sync_sockets]   ss_tcp_data_ready: sk ffff88003867f280, sk->sk_socket           (null), state (Established)\r\n[   84.760840] [tempesta]   Link new msg ffff880038342040 with connection ffff880038759a48\r\n[   84.760840] [tempesta]   Add skb ffff8800389e11c0 to message ffff880038342040\r\n[   84.760840] [tempesta]   received 62 server data bytes (HTTP/1.1 200 OK\r\n[   84.760840] Server: nginx/1.4.6 (Ubuntu)\r\n[   84.760840] Date: Sun, 14 J) on socket (conn=ffff880038759a48)\r\n[   84.760840] [tempesta]   enter FSM at state 0\r\n[   84.760840] [tempesta]   parser: Resp_0(0:0): c=0x48(H), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HttpVer(2:0): c=0x48(H), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_StatusCode(11:0): c=0x32(2), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_ReasonPhrase(12:1): c=0x20( ), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_Hdr(13:1): c=0x53(S), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrOther(87:1): c=0x65(e), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_Hdr(13:1): c=0x44(D), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrOther(87:1): c=0x61(a), r=-2\r\n[   84.760840] [tempesta]   response parsed: len=62 parsed=62 res=-1\r\n[   84.760840] [tempesta]   postpone skb ffff8800389e11c0\r\n[   84.760840] [tempesta]   received 796 server data bytes (un 2015 16:08:17 GMT\r\n[   84.760840] Content-Type: text/html\r\n[   84.760840] Content-Length: 612\r\n[   84.760840] Last-Modified: Tue, 04 Mar 2014 11:46:45 GMT\r\n[   84.760840] Connection: keep-alive\r\n[   84.760840] ETag: \"5315bd25-264\"\r\n[   84.760840] Accept-Ranges: bytes\r\n[   84.760840] \r\n[   84.760840] <!DOCTYPE html>\r\n[   84.760840] <html>\r\n[   84.760840] <head>\r\n[   84.760840] <title>Welcome to nginx!</title>\r\n[   84.760840] <style>\r\n[   84.760840]     body {\r\n[   84.760840]         width: 35em;\r\n[   84.760840]         margin: 0 auto;\r\n[   84.760840]         font-family: Tahoma, Verdana, Arial, sans-serif;\r\n[   84.760840]     }\r\n[   84.760840] </style>\r\n[   84.760840] </head>\r\n[   84.760840] <body>\r\n[   84.760840] <h1>Welcome to nginx!</h1>\r\n[   84.760840] <p>If you see this page, the nginx web server is successfully installed and\r\n[   84.760840] working. Further configuration is required.</p>\r\n[   84.760840] \r\n[   84.760840] [tempesta]   enter FSM at state 87\r\n[   84.760840] [tempesta]   parser: Resp_HdrOther(87:1): c=0x75(u), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_Hdr(13:1): c=0x43(C), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrC(14:1): c=0x6f(o), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrCo(28:1): c=0x6e(n), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrCon(29:1): c=0x74(t), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrCont(38:1): c=0x65(e), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrConte(39:1): c=0x6e(n), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrConten(40:1): c=0x74(t), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrContent(41:1): c=0x2d(-), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrContent_(42:1): c=0x54(T), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrOther(87:1): c=0x79(y), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_Hdr(13:1): c=0x43(C), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrC(14:1): c=0x6f(o), r=-2\r\n[   84.760840] [tempesta]   parser: RGen_LWS(10000:49): c=0x20( ), r=-2\r\n[   84.760840] [tempesta]   parser: RGen_LWS(10000:49): c=0x36(6), r=-2\r\n[   84.760840] [tempesta]   enter FSM at state 49\r\n[   84.760840] [tempesta]   parser: Resp_HdrContent_LengthV(49:0): c=0x36(6), r=-2\r\n[   84.760840] [tempesta]   enter FSM at state 3\r\n[   84.760840] [tempesta]   parser: I_ContLen(3:3): c=0x36(6), r=-2\r\n[   84.760840] [tempesta]   parser: I_EoL(7:3): c=0xd(.), r=-2\r\n[   84.760840] [tempesta]   parser: I_EoL(7:3): c=0xa(.), r=-2\r\n[   84.760840] [tempesta]   parse header __parse_content_length: ret=5 plen=3 id=3\r\n[   84.760840] [tempesta]   store header w/ ptr=ffff880038342548 len=19 flags=1 id=3\r\n[   84.760840] [tempesta]   parser: Resp_Hdr(13:0): c=0x4c(L), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrOther(87:0): c=0x61(a), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_Hdr(13:0): c=0x43(C), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrC(14:0): c=0x6f(o), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrConnection(36:0): c=0x3a(:), r=-2\r\n[   84.760840] [tempesta]   parser: RGen_LWS(10000:37): c=0x20( ), r=-2\r\n[   84.760840] [tempesta]   parser: RGen_LWS(10000:37): c=0x6b(k), r=-2\r\n[   84.760840] [tempesta]   enter FSM at state 37\r\n[   84.760840] [tempesta]   parser: Resp_HdrConnectionV(37:0): c=0x6b(k), r=-2\r\n[   84.760840] [tempesta]   enter FSM at state 1\r\n[   84.760840] [tempesta]   parser: I_Conn(1:1): c=0x6b(k), r=-2\r\n[   84.760840] [tempesta]   parser: I_EoT(6:1): c=0xd(.), r=0\r\n[   84.760840] [tempesta]   parser: I_EoL(7:1): c=0xd(.), r=0\r\n[   84.760840] [tempesta]   parser: I_EoL(7:1): c=0xa(.), r=0\r\n[   84.760840] [tempesta]   parser: Connection parsed: flags 0x2\r\n[   84.760840] [tempesta]   parse header __parse_connection: ret=12 plen=10 id=0\r\n[   84.760840] [tempesta]   store header w/ ptr=ffff880038342588 len=22 flags=1 id=0\r\n[   84.760840] [tempesta]   parser: Resp_Hdr(13:0): c=0x45(E), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrE(50:0): c=0x54(T), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrOther(87:0): c=0x61(a), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_Hdr(13:0): c=0x41(A), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrOther(87:0): c=0x63(c), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_Hdr(13:0): c=0xd(.), r=-2\r\n[   84.760840] [tempesta]   parser: Resp_HdrDone(88:0): c=0xa(.), r=-2\r\n[   84.760840] [tempesta]   parse msg body: flags=0x2 content_length=612\r\n[   84.760840] [tempesta]   parser: Resp_Body(89:0): c=0x3c(<), r=-2\r\n[   84.760840] [tempesta]   read body: to_read=612\r\n[   84.760840] [tempesta]   parser: Resp_BodyReadChunk(92:0): c=0x3c(<), r=-2\r\n[   84.760840] [tempesta]   response parsed: len=796 parsed=796 res=0\r\n[   84.760840] [sync_sockets]   ss_send: sk ffff8800385980c0, sk->sk_socket           (null), state (Established)\r\n[   84.760840] [sync_sockets]   ss_send:167 entail skb=ffff8800389e11c0 data_len=796 len=858\r\n[   84.760840] [sync_sockets]   ss_send:185 sk=ffff8800385980c0 is_queue_empty=0 tcp_send_head(sk)=ffff8800389e11c0 sk->sk_state=1\r\n[   84.760840] [tempesta]   tfw_http_resp_process: req->conn=ffff8800384aa6e8\r\n[   84.760840] [tempesta]   Free msg: ffff880038b97040\r\n[   84.760840] general protection fault: 0000 [#1] SMP \r\n[   84.760840] Modules linked in: tfw_sched_rr(O) tfw_sched_http(O) tfw_sched_hash(O) tfw_sched_dummy(O) tempesta_fw(O) tempesta_db(O)\r\n[   84.760840] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G           O 3.10.10-tempesta #2\r\n[   84.760840] task: ffffffff81565440 ti: ffffffff81554000 task.ti: ffffffff81554000\r\n[   84.760840] RIP: 0010:[<ffffffffa0017b8d>]  [<ffffffffa0017b8d>] tfw_http_msg_free+0x74/0x1bf [tempesta_fw]\r\n[   84.760840] RSP: 0018:ffff88003fc038c8  EFLAGS: 00010202\r\n[   84.760840] RAX: 6b6b6b6b6b6b6b6b RBX: ffff8800389d8a00 RCX: 0000000000000000\r\n[   84.760840] RDX: ffffffff81565440 RSI: 0000000000000001 RDI: 0000000000000282\r\n[   84.760840] RBP: ffff88003fc038f0 R08: 0000000000000000 R09: 0000000000000001\r\n[   84.760840] R10: 0000000000000000 R11: ffff88003fc03546 R12: ffff8800389e11c0\r\n[   84.760840] R13: ffff88003867fa60 R14: 0000000000000000 R15: ffff88003867fa60\r\n[   84.760840] FS:  0000000000000000(0000) GS:ffff88003fc00000(0000) knlGS:0000000000000000\r\n[   84.760840] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b\r\n[   84.760840] CR2: 0000000000609a80 CR3: 00000000389cf000 CR4: 00000000000006b0\r\n[   84.760840] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000\r\n[   84.760840] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400\r\n[   84.760840] Stack:\r\n[   84.760840]  ffff8800389d8a00 ffff8800389e11c0 ffff880038b97040 0000000000000000\r\n[   84.760840]  ffff8800389d8a00 ffff88003fc03988 ffffffffa000da07 ffff880038b97040\r\n[   84.760840]  ffff880038342040 0000000000000048 000000007b54c948 ffff88002300b040\r\n[   84.760840] Call Trace:\r\n[   84.760840]  <IRQ> \r\n[   84.760840]  [<ffffffffa000da07>] tfw_cache_add+0x187/0x19a [tempesta_fw]\r\n[   84.760840]  [<ffffffffa0017111>] tfw_http_resp_process+0x36c/0x3ab [tempesta_fw]\r\n[   84.760840]  [<ffffffffa001724d>] tfw_http_msg_process+0x5c/0x5e [tempesta_fw]\r\n[   84.760840]  [<ffffffffa001311f>] tfw_gfsm_dispatch+0x41/0x43 [tempesta_fw]\r\n[   84.760840]  [<ffffffffa0011ea8>] tfw_connection_recv+0x3a/0x3c [tempesta_fw]\r\n[   84.760840]  [<ffffffffa004e3a3>] ss_tcp_process_proto_skb+0x74/0xf2 [tempesta_fw]\r\n[   84.760840]  [<ffffffffa004e5c9>] ss_tcp_process_skb+0x1a8/0x270 [tempesta_fw]\r\n[   84.760840]  [<ffffffffa004f093>] ss_tcp_process_data+0x12c/0x2bb [tempesta_fw]\r\n[   84.760840]  [<ffffffffa004f432>] ss_tcp_data_ready+0xd1/0x149 [tempesta_fw]\r\n[   84.760840]  [<ffffffff8133beed>] tcp_data_queue+0x4dd/0xd10\r\n[   84.760840]  [<ffffffff8133f2f0>] tcp_rcv_established+0x560/0x620\r\n[   84.760840]  [<ffffffff813494cc>] tcp_v4_do_rcv+0x1dc/0x4e0\r\n[   84.760840]  [<ffffffff811f66b2>] ? do_raw_spin_lock+0x82/0x140\r\n[   84.760840]  [<ffffffff8134b077>] tcp_v4_rcv+0x437/0x8b0\r\n[   84.760840]  [<ffffffff81325455>] ip_local_deliver_finish+0xb5/0x150\r\n[   84.760840]  [<ffffffff813253e0>] ? ip_local_deliver_finish+0x40/0x150\r\n[   84.760840]  [<ffffffff81325c00>] ip_local_deliver+0x50/0x90\r\n[   84.760840]  [<ffffffff8132596e>] ip_rcv_finish+0x47e/0x580\r\n[   84.760840]  [<ffffffff81325efc>] ip_rcv+0x2bc/0x340\r\n[   84.760840]  [<ffffffff812ee4f9>] __netif_receive_skb_core+0x469/0x590\r\n[   84.760840]  [<ffffffff812ee0f3>] ? __netif_receive_skb_core+0x63/0x590\r\n[   84.760840]  [<ffffffff812ee66f>] __netif_receive_skb+0x4f/0x70\r\n[   84.760840]  [<ffffffff812ef1d3>] netif_receive_skb+0xb3/0x140\r\n[   84.760840]  [<ffffffff812e1f09>] ? __netdev_alloc_skb+0x49/0xb0\r\n[   84.760840]  [<ffffffff81278c44>] virtnet_poll+0x624/0x770\r\n[   84.760840]  [<ffffffff812f0413>] net_rx_action+0xa3/0x1f0\r\n[   84.760840]  [<ffffffff81038220>] __do_softirq+0xe0/0x1d0\r\n[   84.760840]  [<ffffffff81038499>] irq_exit+0x69/0xc0\r\n[   84.760840]  [<ffffffff81003c5d>] do_IRQ+0x9d/0xc0\r\n[   84.760840]  [<ffffffff813ca972>] common_interrupt+0x72/0x72\r\n[   84.760840]  <EOI> \r\n[   84.760840]  [<ffffffff81023626>] ? native_safe_halt+0x6/0x10\r\n[   84.760840]  [<ffffffff8100a8dd>] default_idle+0x1d/0x30\r\n[   84.760840]  [<ffffffff8100b26a>] arch_cpu_idle+0xa/0x10\r\n[   84.760840]  [<ffffffff8106817c>] cpu_startup_entry+0x15c/0x200\r\n[   84.760840]  [<ffffffff813be53d>] rest_init+0xcd/0xe0\r\n[   84.760840]  [<ffffffff813be470>] ? csum_partial_copy_generic+0x170/0x170\r\n[   84.760840]  [<ffffffff815dada5>] start_kernel+0x352/0x35d\r\n[   84.760840]  [<ffffffff815da85c>] ? repair_env_string+0x5e/0x5e\r\n[   84.760840]  [<ffffffff815da120>] ? early_idt_handlers+0x120/0x120\r\n[   84.760840]  [<ffffffff815da55b>] x86_64_start_reservations+0x2a/0x2c\r\n[   84.760840]  [<ffffffff815da64c>] x86_64_start_kernel+0xef/0xfc\r\n[   84.760840] Code: 2a 49 1e e1 48 83 7d e8 00 75 05 e9 5f 01 00 00 48 8b 45 e8 48 8b 80 20 03 00 00 48 85 c0 74 28 48 8b 45 e8 48 8b 80 20 03 00 00 <48> 8b 40 38 48 3b 45 e8 75 13 48 8b 45 e8 48 8b 80 20 03 00 00 \r\n[   84.760840] RIP  [<ffffffffa0017b8d>] tfw_http_msg_free+0x74/0x1bf [tempesta_fw]\r\n[   84.760840]  RSP <ffff88003fc038c8>\r\n[   84.760840] ---[ end trace 60044e88aa2f8707 ]---\r\n[   84.760840] Kernel panic - not syncing: Fatal exception in interrupt\r\n```"}}