{"action":"created","comment":{"url":"https://api.github.com/repos/dropbox/pyston/pulls/comments/32079949","id":32079949,"diff_hunk":"@@ -410,23 +417,83 @@ static unw_word_t getFunctionEnd(unw_word_t ip) {\n     return pip.end_ip;\n }\n \n+static bool inASTInterpreterExecuteInner(unw_word_t ip) {\n+    static unw_word_t interpreter_instr_end = getFunctionEnd((unw_word_t)interpreter_instr_addr);\n+    return ((unw_word_t)interpreter_instr_addr <= ip && ip < interpreter_instr_end);\n+}\n+\n+static bool inGeneratorEntry(unw_word_t ip) {\n+    static unw_word_t generator_entry_end = getFunctionEnd((unw_word_t)generatorEntry);\n+    return ((unw_word_t)generatorEntry <= ip && ip < generator_entry_end);\n+}\n+\n+\n+static inline unw_word_t get_cursor_reg(unw_cursor_t* cursor, int reg) {\n+    unw_word_t v;\n+    unw_get_reg(cursor, reg, &v);\n+    return v;\n+}\n+static inline unw_word_t get_cursor_ip(unw_cursor_t* cursor) {\n+    return get_cursor_reg(cursor, UNW_REG_IP) - 1;\n+}\n+static inline unw_word_t get_cursor_bp(unw_cursor_t* cursor) {\n+    return get_cursor_reg(cursor, UNW_TDEP_BP);\n+}\n+\n+template <typename FrameFunc>\n+bool unwindProcessFrame(unw_word_t ip, unw_word_t bp, unw_cursor_t* cursor, FrameFunc func) {\n+    CompiledFunction* cf = getCFForAddress(ip);\n+    bool jitted = cf != NULL;\n+    if (!cf) {\n+        if (inASTInterpreterExecuteInner(ip)) {\n+            cf = getCFForInterpretedFrame((void*)bp);\n+            assert(cf);\n+        }\n+    }\n+\n+    if (!cf)\n+        return false;\n+\n+    PythonFrameIteratorImpl info(jitted ? PythonFrameId::COMPILED : PythonFrameId::INTERPRETED, ip, bp, cf);\n+    if (jitted) {\n+        // Try getting all the callee-save registers, and save the ones we were able to get.\n+        // Some of them may be inaccessible, I think because they weren't defined by that\n+        // stack frame, which can show up as a -UNW_EBADREG return code.\n+        for (int i = 0; i < 16; i++) {\n+            if (!assembler::Register::fromDwarf(i).isCalleeSave())\n+                continue;\n+            unw_word_t r;\n+            int code = unw_get_reg(cursor, i, &r);\n+            ASSERT(code == 0 || code == -UNW_EBADREG, \"%d %d\", code, i);\n+            if (code == 0) {\n+                info.regs[i] = r;\n+                info.regs_valid |= (1 << i);\n+            }\n+        }\n+    }\n+\n+    if (threading::ThreadStateInternal::getUnwindState() == UNWIND_STATE_NORMAL) {","path":"src/codegen/unwinding.cpp","position":91,"original_position":91,"commit_id":"a4f65a1f2fe69752f357fcf96637b8e2cb8a6531","original_commit_id":"a4f65a1f2fe69752f357fcf96637b8e2cb8a6531","user":{"login":"kmod","id":271916,"avatar_url":"https://avatars.githubusercontent.com/u/271916?v=3","gravatar_id":"","url":"https://api.github.com/users/kmod","html_url":"https://github.com/kmod","followers_url":"https://api.github.com/users/kmod/followers","following_url":"https://api.github.com/users/kmod/following{/other_user}","gists_url":"https://api.github.com/users/kmod/gists{/gist_id}","starred_url":"https://api.github.com/users/kmod/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kmod/subscriptions","organizations_url":"https://api.github.com/users/kmod/orgs","repos_url":"https://api.github.com/users/kmod/repos","events_url":"https://api.github.com/users/kmod/events{/privacy}","received_events_url":"https://api.github.com/users/kmod/received_events","type":"User","site_admin":false},"body":"It feels weird that we combine the logic of how to process the frame with the logic of when to process the frame.  I think it might make more sense to have the caller of unwindProcessFrame know whether to actually process the frame or not, since those rules will be different based on the caller.  For example, `sys._getframe` uses this same machinery to traverse the Python stack, but that traversal will now include the rules to skip the topmost frame if we are handling a reraise.  As far as I can tell, we can never call `sys._getframe` when we are in `UNWIND_STATE_RERAISE` so the extra check *should* be ok, but it seems wrong that we check for reraises when doing `sys._getframe`.","created_at":"2015-06-10T00:43:55Z","updated_at":"2015-06-10T00:43:55Z","html_url":"https://github.com/dropbox/pyston/pull/564#discussion_r32079949","pull_request_url":"https://api.github.com/repos/dropbox/pyston/pulls/564","_links":{"self":{"href":"https://api.github.com/repos/dropbox/pyston/pulls/comments/32079949"},"html":{"href":"https://github.com/dropbox/pyston/pull/564#discussion_r32079949"},"pull_request":{"href":"https://api.github.com/repos/dropbox/pyston/pulls/564"}}},"pull_request":{"url":"https://api.github.com/repos/dropbox/pyston/pulls/564","id":36463949,"html_url":"https://github.com/dropbox/pyston/pull/564","diff_url":"https://github.com/dropbox/pyston/pull/564.diff","patch_url":"https://github.com/dropbox/pyston/pull/564.patch","issue_url":"https://api.github.com/repos/dropbox/pyston/issues/564","number":564,"state":"open","locked":false,"title":"Incremental traceback","user":{"login":"toshok","id":24245,"avatar_url":"https://avatars.githubusercontent.com/u/24245?v=3","gravatar_id":"","url":"https://api.github.com/users/toshok","html_url":"https://github.com/toshok","followers_url":"https://api.github.com/users/toshok/followers","following_url":"https://api.github.com/users/toshok/following{/other_user}","gists_url":"https://api.github.com/users/toshok/gists{/gist_id}","starred_url":"https://api.github.com/users/toshok/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/toshok/subscriptions","organizations_url":"https://api.github.com/users/toshok/orgs","repos_url":"https://api.github.com/users/toshok/repos","events_url":"https://api.github.com/users/toshok/events{/privacy}","received_events_url":"https://api.github.com/users/toshok/received_events","type":"User","site_admin":false},"body":"This works, and results in a pretty decent speedup for django-template.py, from ~10s to ~8s.\r\n\r\nopening a PR to let travis-ci do the slow tests.  need to clean up the changes and figure out a better way to deal with the state I'm carrying around during unwinding.","created_at":"2015-05-29T00:18:39Z","updated_at":"2015-06-10T00:43:55Z","closed_at":null,"merged_at":null,"merge_commit_sha":"bb9949bf7bbbc177c9a7868a79fed870472a793a","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/dropbox/pyston/pulls/564/commits","review_comments_url":"https://api.github.com/repos/dropbox/pyston/pulls/564/comments","review_comment_url":"https://api.github.com/repos/dropbox/pyston/pulls/comments{/number}","comments_url":"https://api.github.com/repos/dropbox/pyston/issues/564/comments","statuses_url":"https://api.github.com/repos/dropbox/pyston/statuses/a4f65a1f2fe69752f357fcf96637b8e2cb8a6531","head":{"label":"toshok:incremental-traceback","ref":"incremental-traceback","sha":"a4f65a1f2fe69752f357fcf96637b8e2cb8a6531","user":{"login":"toshok","id":24245,"avatar_url":"https://avatars.githubusercontent.com/u/24245?v=3","gravatar_id":"","url":"https://api.github.com/users/toshok","html_url":"https://github.com/toshok","followers_url":"https://api.github.com/users/toshok/followers","following_url":"https://api.github.com/users/toshok/following{/other_user}","gists_url":"https://api.github.com/users/toshok/gists{/gist_id}","starred_url":"https://api.github.com/users/toshok/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/toshok/subscriptions","organizations_url":"https://api.github.com/users/toshok/orgs","repos_url":"https://api.github.com/users/toshok/repos","events_url":"https://api.github.com/users/toshok/events{/privacy}","received_events_url":"https://api.github.com/users/toshok/received_events","type":"User","site_admin":false},"repo":{"id":26560265,"name":"pyston","full_name":"toshok/pyston","owner":{"login":"toshok","id":24245,"avatar_url":"https://avatars.githubusercontent.com/u/24245?v=3","gravatar_id":"","url":"https://api.github.com/users/toshok","html_url":"https://github.com/toshok","followers_url":"https://api.github.com/users/toshok/followers","following_url":"https://api.github.com/users/toshok/following{/other_user}","gists_url":"https://api.github.com/users/toshok/gists{/gist_id}","starred_url":"https://api.github.com/users/toshok/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/toshok/subscriptions","organizations_url":"https://api.github.com/users/toshok/orgs","repos_url":"https://api.github.com/users/toshok/repos","events_url":"https://api.github.com/users/toshok/events{/privacy}","received_events_url":"https://api.github.com/users/toshok/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/toshok/pyston","description":"An open-source Python implementation using JIT techniques. ","fork":true,"url":"https://api.github.com/repos/toshok/pyston","forks_url":"https://api.github.com/repos/toshok/pyston/forks","keys_url":"https://api.github.com/repos/toshok/pyston/keys{/key_id}","collaborators_url":"https://api.github.com/repos/toshok/pyston/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/toshok/pyston/teams","hooks_url":"https://api.github.com/repos/toshok/pyston/hooks","issue_events_url":"https://api.github.com/repos/toshok/pyston/issues/events{/number}","events_url":"https://api.github.com/repos/toshok/pyston/events","assignees_url":"https://api.github.com/repos/toshok/pyston/assignees{/user}","branches_url":"https://api.github.com/repos/toshok/pyston/branches{/branch}","tags_url":"https://api.github.com/repos/toshok/pyston/tags","blobs_url":"https://api.github.com/repos/toshok/pyston/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/toshok/pyston/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/toshok/pyston/git/refs{/sha}","trees_url":"https://api.github.com/repos/toshok/pyston/git/trees{/sha}","statuses_url":"https://api.github.com/repos/toshok/pyston/statuses/{sha}","languages_url":"https://api.github.com/repos/toshok/pyston/languages","stargazers_url":"https://api.github.com/repos/toshok/pyston/stargazers","contributors_url":"https://api.github.com/repos/toshok/pyston/contributors","subscribers_url":"https://api.github.com/repos/toshok/pyston/subscribers","subscription_url":"https://api.github.com/repos/toshok/pyston/subscription","commits_url":"https://api.github.com/repos/toshok/pyston/commits{/sha}","git_commits_url":"https://api.github.com/repos/toshok/pyston/git/commits{/sha}","comments_url":"https://api.github.com/repos/toshok/pyston/comments{/number}","issue_comment_url":"https://api.github.com/repos/toshok/pyston/issues/comments{/number}","contents_url":"https://api.github.com/repos/toshok/pyston/contents/{+path}","compare_url":"https://api.github.com/repos/toshok/pyston/compare/{base}...{head}","merges_url":"https://api.github.com/repos/toshok/pyston/merges","archive_url":"https://api.github.com/repos/toshok/pyston/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/toshok/pyston/downloads","issues_url":"https://api.github.com/repos/toshok/pyston/issues{/number}","pulls_url":"https://api.github.com/repos/toshok/pyston/pulls{/number}","milestones_url":"https://api.github.com/repos/toshok/pyston/milestones{/number}","notifications_url":"https://api.github.com/repos/toshok/pyston/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/toshok/pyston/labels{/name}","releases_url":"https://api.github.com/repos/toshok/pyston/releases{/id}","created_at":"2014-11-12T22:56:58Z","updated_at":"2015-03-07T07:42:00Z","pushed_at":"2015-06-09T20:51:55Z","git_url":"git://github.com/toshok/pyston.git","ssh_url":"git@github.com:toshok/pyston.git","clone_url":"https://github.com/toshok/pyston.git","svn_url":"https://github.com/toshok/pyston","homepage":"https://tech.dropbox.com/2014/04/introducing-pyston-an-upcoming-jit-based-python-implementation/","size":21708,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"dropbox:master","ref":"master","sha":"bc36f5df8ee16bdaef8733057e5e1f7e92dabf74","user":{"login":"dropbox","id":559357,"avatar_url":"https://avatars.githubusercontent.com/u/559357?v=3","gravatar_id":"","url":"https://api.github.com/users/dropbox","html_url":"https://github.com/dropbox","followers_url":"https://api.github.com/users/dropbox/followers","following_url":"https://api.github.com/users/dropbox/following{/other_user}","gists_url":"https://api.github.com/users/dropbox/gists{/gist_id}","starred_url":"https://api.github.com/users/dropbox/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dropbox/subscriptions","organizations_url":"https://api.github.com/users/dropbox/orgs","repos_url":"https://api.github.com/users/dropbox/repos","events_url":"https://api.github.com/users/dropbox/events{/privacy}","received_events_url":"https://api.github.com/users/dropbox/received_events","type":"Organization","site_admin":false},"repo":{"id":18320947,"name":"pyston","full_name":"dropbox/pyston","owner":{"login":"dropbox","id":559357,"avatar_url":"https://avatars.githubusercontent.com/u/559357?v=3","gravatar_id":"","url":"https://api.github.com/users/dropbox","html_url":"https://github.com/dropbox","followers_url":"https://api.github.com/users/dropbox/followers","following_url":"https://api.github.com/users/dropbox/following{/other_user}","gists_url":"https://api.github.com/users/dropbox/gists{/gist_id}","starred_url":"https://api.github.com/users/dropbox/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dropbox/subscriptions","organizations_url":"https://api.github.com/users/dropbox/orgs","repos_url":"https://api.github.com/users/dropbox/repos","events_url":"https://api.github.com/users/dropbox/events{/privacy}","received_events_url":"https://api.github.com/users/dropbox/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/dropbox/pyston","description":"An open-source Python implementation using JIT techniques. ","fork":false,"url":"https://api.github.com/repos/dropbox/pyston","forks_url":"https://api.github.com/repos/dropbox/pyston/forks","keys_url":"https://api.github.com/repos/dropbox/pyston/keys{/key_id}","collaborators_url":"https://api.github.com/repos/dropbox/pyston/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/dropbox/pyston/teams","hooks_url":"https://api.github.com/repos/dropbox/pyston/hooks","issue_events_url":"https://api.github.com/repos/dropbox/pyston/issues/events{/number}","events_url":"https://api.github.com/repos/dropbox/pyston/events","assignees_url":"https://api.github.com/repos/dropbox/pyston/assignees{/user}","branches_url":"https://api.github.com/repos/dropbox/pyston/branches{/branch}","tags_url":"https://api.github.com/repos/dropbox/pyston/tags","blobs_url":"https://api.github.com/repos/dropbox/pyston/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/dropbox/pyston/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/dropbox/pyston/git/refs{/sha}","trees_url":"https://api.github.com/repos/dropbox/pyston/git/trees{/sha}","statuses_url":"https://api.github.com/repos/dropbox/pyston/statuses/{sha}","languages_url":"https://api.github.com/repos/dropbox/pyston/languages","stargazers_url":"https://api.github.com/repos/dropbox/pyston/stargazers","contributors_url":"https://api.github.com/repos/dropbox/pyston/contributors","subscribers_url":"https://api.github.com/repos/dropbox/pyston/subscribers","subscription_url":"https://api.github.com/repos/dropbox/pyston/subscription","commits_url":"https://api.github.com/repos/dropbox/pyston/commits{/sha}","git_commits_url":"https://api.github.com/repos/dropbox/pyston/git/commits{/sha}","comments_url":"https://api.github.com/repos/dropbox/pyston/comments{/number}","issue_comment_url":"https://api.github.com/repos/dropbox/pyston/issues/comments{/number}","contents_url":"https://api.github.com/repos/dropbox/pyston/contents/{+path}","compare_url":"https://api.github.com/repos/dropbox/pyston/compare/{base}...{head}","merges_url":"https://api.github.com/repos/dropbox/pyston/merges","archive_url":"https://api.github.com/repos/dropbox/pyston/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/dropbox/pyston/downloads","issues_url":"https://api.github.com/repos/dropbox/pyston/issues{/number}","pulls_url":"https://api.github.com/repos/dropbox/pyston/pulls{/number}","milestones_url":"https://api.github.com/repos/dropbox/pyston/milestones{/number}","notifications_url":"https://api.github.com/repos/dropbox/pyston/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/dropbox/pyston/labels{/name}","releases_url":"https://api.github.com/repos/dropbox/pyston/releases{/id}","created_at":"2014-04-01T07:17:20Z","updated_at":"2015-06-09T12:47:56Z","pushed_at":"2015-06-10T00:09:56Z","git_url":"git://github.com/dropbox/pyston.git","ssh_url":"git@github.com:dropbox/pyston.git","clone_url":"https://github.com/dropbox/pyston.git","svn_url":"https://github.com/dropbox/pyston","homepage":"https://tech.dropbox.com/2014/04/introducing-pyston-an-upcoming-jit-based-python-implementation/","size":29732,"stargazers_count":2988,"watchers_count":2988,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":213,"mirror_url":null,"open_issues_count":63,"forks":213,"open_issues":63,"watchers":2988,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/dropbox/pyston/pulls/564"},"html":{"href":"https://github.com/dropbox/pyston/pull/564"},"issue":{"href":"https://api.github.com/repos/dropbox/pyston/issues/564"},"comments":{"href":"https://api.github.com/repos/dropbox/pyston/issues/564/comments"},"review_comments":{"href":"https://api.github.com/repos/dropbox/pyston/pulls/564/comments"},"review_comment":{"href":"https://api.github.com/repos/dropbox/pyston/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/dropbox/pyston/pulls/564/commits"},"statuses":{"href":"https://api.github.com/repos/dropbox/pyston/statuses/a4f65a1f2fe69752f357fcf96637b8e2cb8a6531"}}}}