{"action":"created","issue":{"url":"https://api.github.com/repos/the-tcpdump-group/libpcap/issues/430","labels_url":"https://api.github.com/repos/the-tcpdump-group/libpcap/issues/430/labels{/name}","comments_url":"https://api.github.com/repos/the-tcpdump-group/libpcap/issues/430/comments","events_url":"https://api.github.com/repos/the-tcpdump-group/libpcap/issues/430/events","html_url":"https://github.com/the-tcpdump-group/libpcap/issues/430","id":70111876,"number":430,"title":"Sample ipv4 packet matched by several sample filters iff optimization is enabled.","user":{"login":"kbara","id":1588250,"avatar_url":"https://avatars.githubusercontent.com/u/1588250?v=3","gravatar_id":"","url":"https://api.github.com/users/kbara","html_url":"https://github.com/kbara","followers_url":"https://api.github.com/users/kbara/followers","following_url":"https://api.github.com/users/kbara/following{/other_user}","gists_url":"https://api.github.com/users/kbara/gists{/gist_id}","starred_url":"https://api.github.com/users/kbara/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kbara/subscriptions","organizations_url":"https://api.github.com/users/kbara/orgs","repos_url":"https://api.github.com/users/kbara/repos","events_url":"https://api.github.com/users/kbara/events{/privacy}","received_events_url":"https://api.github.com/users/kbara/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":12,"created_at":"2015-04-22T12:25:30Z","updated_at":"2015-04-24T19:02:58Z","closed_at":null,"body":"Some filters with out-of-bounds packet access behave differently depending on if optimizations are enabled or not. In a two-clause expression with an out of bounds packet access in the first clause, joined by the 'or' operator to a clause that would match a packet, with the optimizer disabled, a filter never matches. With the optimizer enabled, the second clause is sometimes run, allowing the filter to match packets.\r\n\r\ntcpdump version 4.5.1\r\nlibpcap version 1.5.3\r\n\r\nThe examples are run on a 66-byte tcp ack packet, which can be downloaded from https://github.com/Igalia/pflua/raw/master/tests/data/tcp-ack-66-bytes.pcap\r\n\r\n\"ether[1] > tcp[100] or ip\" triggers this inconsistency; it matches the test packet with optimizations enabled, and does not match with optimizations disabled.\r\n```\r\n% tcpdump -ntr tcp-ack-66-bytes.pcap \"ether[1] > tcp[100] or ip\" 2>/dev/null | grep IP || echo \"no match\"\r\nIP 149.174.156.93.54192 > 178.79.150.233.80: Flags [.], ack 3209860838, win 31, options [nop,nop,TS val 2756387939 ecr 4173199779], length 0\r\n% tcpdump -O -ntr tcp-ack-66-bytes.pcap \"ether[1] > tcp[100] or ip\" 2>/dev/null | grep IP || echo \"no match\"\r\nno match\r\n```\r\n\r\nHere are more test cases which show similar behavior. Note that the test packet is 66 bytes, so tcp[100], ether[100], and ip[100] are all out of bounds, and that the protocols match the packet.\r\n\r\n```\r\nThe following expressions show problematic behavior:\r\nether[1] > tcp[100] or ip\r\nether[1] > tcp[100] or tcp\r\nether[1] > tcp[100] or 1 >= 1\r\nether[1] > tcp[100] or not arp\r\nether[100] > tcp[1] or tcp\r\nether[100] > tcp[100] or tcp\r\nip[100] > ether[1] or ip\r\n\r\nDetails:\r\n\r\n% tcpdump -ntr tcp-ack-66-bytes.pcap \"ether[1] > tcp[100] or ip\" 2>/dev/null | grep IP || echo \"no match\"\r\nIP 149.174.156.93.54192 > 178.79.150.233.80: Flags [.], ack 3209860838, win 31, options [nop,nop,TS val 2756387939 ecr 4173199779], length 0\r\n% tcpdump -O -ntr tcp-ack-66-bytes.pcap \"ether[1] > tcp[100] or ip\" 2>/dev/null | grep IP || echo \"no match\"\r\nno match\r\n---\r\n% tcpdump -ntr tcp-ack-66-bytes.pcap \"ether[1] > tcp[100] or tcp\" 2>/dev/null | grep IP || echo \"no match\"\r\nIP 149.174.156.93.54192 > 178.79.150.233.80: Flags [.], ack 3209860838, win 31, options [nop,nop,TS val 2756387939 ecr 4173199779], length 0\r\n% tcpdump -O -ntr tcp-ack-66-bytes.pcap \"ether[1] > tcp[100] or tcp\" 2>/dev/null | grep IP || echo \"no match\"\r\nno match\r\n---\r\n% tcpdump -ntr tcp-ack-66-bytes.pcap \"ether[1] > tcp[100] or 1 >= 1\" 2>/dev/null | grep IP || echo \"no match\"\r\nIP 149.174.156.93.54192 > 178.79.150.233.80: Flags [.], ack 3209860838, win 31, options [nop,nop,TS val 2756387939 ecr 4173199779], length 0\r\n% tcpdump -O -ntr tcp-ack-66-bytes.pcap \"ether[1] > tcp[100] or 1 >= 1\" 2>/dev/null | grep IP || echo \"no match\"\r\nno match\r\n---\r\n% tcpdump -ntr tcp-ack-66-bytes.pcap \"ether[1] > tcp[100] or not arp\" 2>/dev/null | grep IP || echo \"no match\"\r\nIP 149.174.156.93.54192 > 178.79.150.233.80: Flags [.], ack 3209860838, win 31, options [nop,nop,TS val 2756387939 ecr 4173199779], length 0\r\n% tcpdump -O -ntr tcp-ack-66-bytes.pcap \"ether[1] > tcp[100] or not arp\" 2>/dev/null | grep IP || echo \"no match\"\r\nno match\r\n---\r\n% tcpdump -ntr tcp-ack-66-bytes.pcap \"ether[100] > tcp[1] or tcp\" 2>/dev/null | grep IP || echo \"no match\"\r\nIP 149.174.156.93.54192 > 178.79.150.233.80: Flags [.], ack 3209860838, win 31, options [nop,nop,TS val 2756387939 ecr 4173199779], length 0\r\n% tcpdump -O -ntr tcp-ack-66-bytes.pcap \"ether[100] > tcp[1] or tcp\" 2>/dev/null | grep IP || echo \"no match\"\r\nno match\r\n---\r\n% tcpdump -ntr tcp-ack-66-bytes.pcap \"ether[100] > tcp[100] or tcp\" 2>/dev/null | grep IP || echo \"no match\"\r\nIP 149.174.156.93.54192 > 178.79.150.233.80: Flags [.], ack 3209860838, win 31, options [nop,nop,TS val 2756387939 ecr 4173199779], length 0\r\n% tcpdump -O -ntr tcp-ack-66-bytes.pcap \"ether[100] > tcp[100] or tcp\" 2>/dev/null | grep IP || echo \"no match\"\r\nno match\r\n---\r\n% tcpdump -ntr tcp-ack-66-bytes.pcap \"ip[100] > ether[1] or ip\" 2>/dev/null | grep IP || echo \"no match\"\r\nIP 149.174.156.93.54192 > 178.79.150.233.80: Flags [.], ack 3209860838, win 31, options [nop,nop,TS val 2756387939 ecr 4173199779], length 0\r\n% tcpdump -O -ntr tcp-ack-66-bytes.pcap \"ip[100] > ether[1] or ip\" 2>/dev/null | grep IP || echo \"no match\"\r\nno match\r\n```\r\n```\r\nThe following similar expressions show consistent behavior:\r\nether[1] > tcp[100] or ip[1] >= 0\r\nip[100] > ether[1] or tcp\r\n\r\nDetails:\r\n\r\n% tcpdump -ntr tcp-ack-66-bytes.pcap \"ether[1] > tcp[100] or ip[1] >= 0\" 2>/dev/null | grep IP || echo \"no match\"\r\nno match\r\n% tcpdump -O -ntr tcp-ack-66-bytes.pcap \"ether[1] > tcp[100] or ip[1] >= 0\" 2>/dev/null | grep IP || echo \"no match\"\r\nno match\r\n---\r\n% tcpdump -ntr tcp-ack-66-bytes.pcap \"ip[100] > ether[1] or tcp\" 2>/dev/null | grep IP || echo \"no match\"\r\nno match\r\n% tcpdump -O -ntr tcp-ack-66-bytes.pcap \"ip[100] > ether[1] or tcp\" 2>/dev/null | grep IP || echo \"no match\"\r\nno match\r\n```\r\n\r\nSource to run the above tests:\r\n```\r\n#!/usr/bin/env python\r\nimport subprocess\r\n\r\ndef print_and_run(cmd):\r\n    print(\"% \" + cmd)\r\n    subprocess.call(cmd, shell=True)\r\n\r\ndef gen_and_run():\r\n    template_opt = 'tcpdump -ntr tcp-ack-66-bytes.pcap \"%s\" 2>/dev/null | grep IP || echo \"no match\"'\r\n    template_unopt = 'tcpdump -O -ntr tcp-ack-66-bytes.pcap \"%s\" 2>/dev/null | grep IP || echo \"no match\"'\r\n\r\n    prob_pfs = [ \"ether[1] > tcp[100] or ip\",\r\n                 \"ether[1] > tcp[100] or tcp\",\r\n                 \"ether[1] > tcp[100] or 1 >= 1\",\r\n                 \"ether[1] > tcp[100] or not arp\",\r\n                 \"ether[100] > tcp[1] or tcp\",\r\n                 \"ether[100] > tcp[100] or tcp\",\r\n                 \"ip[100] > ether[1] or ip\"\r\n               ]\r\n    print(\"The following expressions show problematic behavior:\")\r\n    print(\"\\n\".join(prob_pfs))\r\n    print(\"\\nDetails:\\n\")\r\n\r\n    for pf in prob_pfs:\r\n        print_and_run(template_opt % pf)\r\n        print_and_run(template_unopt % pf)\r\n\tprint(\"---\")\r\n\r\n    good_pfs = [ \"ether[1] > tcp[100] or ip[1] >= 0\",\r\n                 \"ip[100] > ether[1] or tcp\"\r\n               ]\r\n    print(\"The following similar expressions show consistent behavior:\")\r\n    print(\"\\n\".join(good_pfs))\r\n    print(\"\\nDetails:\\n\")\r\n\r\n    for gpf in good_pfs:\r\n        print_and_run(template_opt % gpf)\r\n        print_and_run(template_unopt % gpf)\r\n\tprint(\"---\")\r\ngen_and_run()\r\n```"},"comment":{"url":"https://api.github.com/repos/the-tcpdump-group/libpcap/issues/comments/96036940","html_url":"https://github.com/the-tcpdump-group/libpcap/issues/430#issuecomment-96036940","issue_url":"https://api.github.com/repos/the-tcpdump-group/libpcap/issues/430","id":96036940,"user":{"login":"guyharris","id":3599441,"avatar_url":"https://avatars.githubusercontent.com/u/3599441?v=3","gravatar_id":"","url":"https://api.github.com/users/guyharris","html_url":"https://github.com/guyharris","followers_url":"https://api.github.com/users/guyharris/followers","following_url":"https://api.github.com/users/guyharris/following{/other_user}","gists_url":"https://api.github.com/users/guyharris/gists{/gist_id}","starred_url":"https://api.github.com/users/guyharris/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/guyharris/subscriptions","organizations_url":"https://api.github.com/users/guyharris/orgs","repos_url":"https://api.github.com/users/guyharris/repos","events_url":"https://api.github.com/users/guyharris/events{/privacy}","received_events_url":"https://api.github.com/users/guyharris/received_events","type":"User","site_admin":false},"created_at":"2015-04-24T19:02:58Z","updated_at":"2015-04-24T19:02:58Z","body":"> The filter matches only with optimization disabled\r\n\r\nThe filter `1/0 == 0` is *successfully compiled* only with optimization disabled.  Tcpdump doesn't even *run* if you optimize that one, as the compiler returns an error, so the filter, not being run, neither matches nor fails to match.\r\n\r\nAnd short-circuiting has nothing to do with that - it happens regardless of the order in which things are tested:\r\n\r\n    $ tcpdump -d '1/0 == 0 or tcp'\r\n    tcpdump: division by zero\r\n    $ tcpdump -d 'tcp or 1/0 == 0'\r\n    tcpdump: division by zero\r\n\r\n(the fact that tcpdump doesn't print out generated filter code means that no such code was generated; \"division by zero\" is an error string returned by `pcap_compile()`, not a string printed by the BPF interpreter when detecting a division by zero at run time).\r\n\r\nThe reason why the behavior is different when optimization is enabled and when it's not enabled is that the check for division and modulus by zero is done in the optimizer at the time it does compile-time evaluation of expressions.  I've checked in a change to detect division by a constant zero at code generation time prior to optimization, so the checks are done regardless of whether optimization is enabled.  That way, `ip or 1/0 == 0` won't make it through the compiler under any circumstances, so developers and users will know that `ip or 1/0 == 0` will not even be given the chance to match anything.\r\n\r\nThe key question here is whether run-time exceptions should be treated, by users and developers, as a mechanism for doing tests that match or don't match packets, so that code that could generate run-time bounds exceptions cannot be optimized away, or whether, if they want to do length checks, they should do so explicitly rather than relying on the bounds exception causing the entire program to fail rather than, for example, causing only tests using the value whose fetch caused the bounds exception to fail.  I vote for the latter."}}