{"action":"created","issue":{"url":"https://api.github.com/repos/jmespath/jmespath.rb/issues/4","labels_url":"https://api.github.com/repos/jmespath/jmespath.rb/issues/4/labels{/name}","comments_url":"https://api.github.com/repos/jmespath/jmespath.rb/issues/4/comments","events_url":"https://api.github.com/repos/jmespath/jmespath.rb/issues/4/events","html_url":"https://github.com/jmespath/jmespath.rb/pull/4","id":55195927,"number":4,"title":"Rewritten interpreter and AST optimizations","user":{"login":"iconara","id":73330,"avatar_url":"https://avatars.githubusercontent.com/u/73330?v=3","gravatar_id":"","url":"https://api.github.com/users/iconara","html_url":"https://github.com/iconara","followers_url":"https://api.github.com/users/iconara/followers","following_url":"https://api.github.com/users/iconara/following{/other_user}","gists_url":"https://api.github.com/users/iconara/gists{/gist_id}","starred_url":"https://api.github.com/users/iconara/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/iconara/subscriptions","organizations_url":"https://api.github.com/users/iconara/orgs","repos_url":"https://api.github.com/users/iconara/repos","events_url":"https://api.github.com/users/iconara/events{/privacy}","received_events_url":"https://api.github.com/users/iconara/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-01-22T19:17:10Z","updated_at":"2015-09-17T06:22:42Z","closed_at":"2015-09-16T21:12:45Z","pull_request":{"url":"https://api.github.com/repos/jmespath/jmespath.rb/pulls/4","html_url":"https://github.com/jmespath/jmespath.rb/pull/4","diff_url":"https://github.com/jmespath/jmespath.rb/pull/4.diff","patch_url":"https://github.com/jmespath/jmespath.rb/pull/4.patch"},"body":"This adds a new JIT-friendly interpreter and AST optimizations that makes evaluating expressions much faster, 2-4x for the benchmarks I'm running.\r\n\r\nThe main thing that has changes is that instead of `TreeInterpreter#dispatch` walking the AST using a big case statement, each AST node contains can interpret itself. This removes almost all of the branching from the interpretation, and it makes it easier for JITing runtimes like JRuby to optimize the interpreter code. In the current implementation quite a lot of time is spent just running `Symbol#===` because of all of the cases that needs to be checked to decide which code to run for the next node in the AST, this has been completely side-stepped since each node calls the next node directly.\r\n\r\nI've tried to move as many decisions as possible to the parser phase to avoid branching in the interpreter. For example, each function is its own AST node, instead of having a case statement that needs to be run at interpretation. The same goes for projections, comparisons, etc.\r\n\r\nI've also started on AST optimizations, such as flattening long chains of field and index lookups, conditions with literals, and a few other things. This part is a work in progress, but it can speed up some expressions by 20-30%.\r\n\r\nI'd like to work on this a bit more, I don't think it's ready to merge yet. The tests all pass, but I think there's a need for some more tests around the optimizer, for example.\r\n\r\nI'm also considering adding options for the optimizer to assume that values will always be hashes, never structs, and always have strings, never symbol keys (I would of course not change the current defaults, this would be an option only), because this would enable another level of optimizations that are not currently possible."},"comment":{"url":"https://api.github.com/repos/jmespath/jmespath.rb/issues/comments/140979317","html_url":"https://github.com/jmespath/jmespath.rb/pull/4#issuecomment-140979317","issue_url":"https://api.github.com/repos/jmespath/jmespath.rb/issues/4","id":140979317,"user":{"login":"iconara","id":73330,"avatar_url":"https://avatars.githubusercontent.com/u/73330?v=3","gravatar_id":"","url":"https://api.github.com/users/iconara","html_url":"https://github.com/iconara","followers_url":"https://api.github.com/users/iconara/followers","following_url":"https://api.github.com/users/iconara/following{/other_user}","gists_url":"https://api.github.com/users/iconara/gists{/gist_id}","starred_url":"https://api.github.com/users/iconara/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/iconara/subscriptions","organizations_url":"https://api.github.com/users/iconara/orgs","repos_url":"https://api.github.com/users/iconara/repos","events_url":"https://api.github.com/users/iconara/events{/privacy}","received_events_url":"https://api.github.com/users/iconara/received_events","type":"User","site_admin":false},"created_at":"2015-09-17T06:22:42Z","updated_at":"2015-09-17T06:22:42Z","body":"Thanks!\r\n\r\nWe forked and released our own gem, [burtpath](https://github.com/burtcorp/burtpath), but we'll probably switch back to jmespath.rb. There are some more contributions coming."}}