{"action":"created","issue":{"url":"https://api.github.com/repos/pyinstaller/pyinstaller/issues/1393","labels_url":"https://api.github.com/repos/pyinstaller/pyinstaller/issues/1393/labels{/name}","comments_url":"https://api.github.com/repos/pyinstaller/pyinstaller/issues/1393/comments","events_url":"https://api.github.com/repos/pyinstaller/pyinstaller/issues/1393/events","html_url":"https://github.com/pyinstaller/pyinstaller/pull/1393","id":99871717,"number":1393,"title":"Module Import Ordering Preserved","user":{"login":"leycec","id":217028,"avatar_url":"https://avatars.githubusercontent.com/u/217028?v=3","gravatar_id":"","url":"https://api.github.com/users/leycec","html_url":"https://github.com/leycec","followers_url":"https://api.github.com/users/leycec/followers","following_url":"https://api.github.com/users/leycec/following{/other_user}","gists_url":"https://api.github.com/users/leycec/gists{/gist_id}","starred_url":"https://api.github.com/users/leycec/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/leycec/subscriptions","organizations_url":"https://api.github.com/users/leycec/orgs","repos_url":"https://api.github.com/users/leycec/repos","events_url":"https://api.github.com/users/leycec/events{/privacy}","received_events_url":"https://api.github.com/users/leycec/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/pyinstaller/pyinstaller/labels/@high","name":"@high","color":"e11d21"},{"url":"https://api.github.com/repos/pyinstaller/pyinstaller/labels/modulegraph","name":"modulegraph","color":"fef2c0"},{"url":"https://api.github.com/repos/pyinstaller/pyinstaller/labels/more%20work","name":"more work","color":"c7def8"},{"url":"https://api.github.com/repos/pyinstaller/pyinstaller/labels/PY3","name":"PY3","color":"fef2c0"}],"state":"open","locked":false,"assignee":{"login":"htgoebel","id":784161,"avatar_url":"https://avatars.githubusercontent.com/u/784161?v=3","gravatar_id":"","url":"https://api.github.com/users/htgoebel","html_url":"https://github.com/htgoebel","followers_url":"https://api.github.com/users/htgoebel/followers","following_url":"https://api.github.com/users/htgoebel/following{/other_user}","gists_url":"https://api.github.com/users/htgoebel/gists{/gist_id}","starred_url":"https://api.github.com/users/htgoebel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/htgoebel/subscriptions","organizations_url":"https://api.github.com/users/htgoebel/orgs","repos_url":"https://api.github.com/users/htgoebel/repos","events_url":"https://api.github.com/users/htgoebel/events{/privacy}","received_events_url":"https://api.github.com/users/htgoebel/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/pyinstaller/pyinstaller/milestones/9","html_url":"https://github.com/pyinstaller/pyinstaller/milestones/PyInstaller%203.0","labels_url":"https://api.github.com/repos/pyinstaller/pyinstaller/milestones/9/labels","id":831132,"number":9,"title":"PyInstaller 3.0","description":" * Add Python 3 support\r\n * Use modulegraph library for dependency analysis.","creator":{"login":"pyinstaller-tickets-migration","id":5377465,"avatar_url":"https://avatars.githubusercontent.com/u/5377465?v=3","gravatar_id":"","url":"https://api.github.com/users/pyinstaller-tickets-migration","html_url":"https://github.com/pyinstaller-tickets-migration","followers_url":"https://api.github.com/users/pyinstaller-tickets-migration/followers","following_url":"https://api.github.com/users/pyinstaller-tickets-migration/following{/other_user}","gists_url":"https://api.github.com/users/pyinstaller-tickets-migration/gists{/gist_id}","starred_url":"https://api.github.com/users/pyinstaller-tickets-migration/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pyinstaller-tickets-migration/subscriptions","organizations_url":"https://api.github.com/users/pyinstaller-tickets-migration/orgs","repos_url":"https://api.github.com/users/pyinstaller-tickets-migration/repos","events_url":"https://api.github.com/users/pyinstaller-tickets-migration/events{/privacy}","received_events_url":"https://api.github.com/users/pyinstaller-tickets-migration/received_events","type":"User","site_admin":false},"open_issues":21,"closed_issues":105,"state":"open","created_at":"2014-10-18T13:32:07Z","updated_at":"2015-09-11T13:27:07Z","due_on":"2015-09-29T22:00:00Z","closed_at":null},"comments":47,"created_at":"2015-08-09T07:04:16Z","updated_at":"2015-09-12T05:24:39Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/pyinstaller/pyinstaller/pulls/1393","html_url":"https://github.com/pyinstaller/pyinstaller/pull/1393","diff_url":"https://github.com/pyinstaller/pyinstaller/pull/1393.diff","patch_url":"https://github.com/pyinstaller/pyinstaller/pull/1393.patch"},"body":"This pull request preserves the order in which modules are imported by the user application. Because that is both needful and good.\r\n\r\nSpecifically, a new public `lib.modulegraph.modulegraph.Node.order` attribute – guaranteed to monotonically increase on _every_ module importation and hence provide a reliable means of comparing importation order – is added. (Tested and working, you may be sure.)\r\n\r\nThis low-level machinery partially fixes #1367. Hooks may now test the order in which two hypothetical modules `foo` and `bar` were initially imported as follows:\r\n\r\n```\r\nfrom PyInstaller.utils.hooks.hookutils import logger\r\n\r\ndef hook(mod):\r\n    if mod.graph.findNode('foo').order < mod.graph.findNode('bar').order:\r\n        logger.debug('Module \"foo\" imported before module \"bar\". Why? I dunno.')\r\n    else:\r\n        logger.debug('Module \"bar\" imported before module \"foo\". Suck it, you.')\r\n\r\n    return mod\r\n```\r\n\r\n## What About #1367? Why You No Fix, Huh?\r\n\r\nWould someone (...hey, no names: @htgoebel) mind posting a minimal-length example (MLE) demonstrating the exact issue of #1367?\r\n\r\nWhile I _do_ have the Python 2 version of wxPython locally installed, I'm unsure how to properly replicate this issue. I know; it's probably trivial. I'm feeble-brained, but hard-working. As soon as I get an MLE, I'll append a commit to this pull request fixing #1367 in entirety.\r\n\r\n## What Else? There's Always Something Else\r\n\r\nwxPython's Python 2 version (referred to as \"Classic\") and Python 3 version (referred to as \"Phoenix\") appear to be [_very_ dissimilar](http://wxpython.org/Phoenix/docs/html/MigrationGuide.html#the-wx-namespace-and-submodules-reorganized) – to the point of actually being different projects that confusingly share the same name. Our import hooks should probably account for these differences in ways I cannot possibly fathom.\r\n\r\nThus [spaketh Walt](http://www.bartleby.com/142/20.html)."},"comment":{"url":"https://api.github.com/repos/pyinstaller/pyinstaller/issues/comments/139722879","html_url":"https://github.com/pyinstaller/pyinstaller/pull/1393#issuecomment-139722879","issue_url":"https://api.github.com/repos/pyinstaller/pyinstaller/issues/1393","id":139722879,"user":{"login":"leycec","id":217028,"avatar_url":"https://avatars.githubusercontent.com/u/217028?v=3","gravatar_id":"","url":"https://api.github.com/users/leycec","html_url":"https://github.com/leycec","followers_url":"https://api.github.com/users/leycec/followers","following_url":"https://api.github.com/users/leycec/following{/other_user}","gists_url":"https://api.github.com/users/leycec/gists{/gist_id}","starred_url":"https://api.github.com/users/leycec/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/leycec/subscriptions","organizations_url":"https://api.github.com/users/leycec/orgs","repos_url":"https://api.github.com/users/leycec/repos","events_url":"https://api.github.com/users/leycec/events{/privacy}","received_events_url":"https://api.github.com/users/leycec/received_events","type":"User","site_admin":false},"created_at":"2015-09-12T05:24:39Z","updated_at":"2015-09-12T05:24:39Z","body":"> Hooks should be given a tree/graph that is in a consistent state. If the graph isn't internally consistent, there's nothing useful the hook can do with it.\r\n\r\nExactly. I'd tell you what you've won, but you already got a tasty fish cake.\r\n\r\n> ...is so strong that it becomes impossible to create any types of hooks except for post-graph hooks.\r\n\r\nNot _quite_, though that's a fair interpretation. While post-graph hooks are indisputably safe, any type of hook run immediately after, before, or instead of the following high-level methods is also safe (...in decreasing order of \"safeness\"):\r\n\r\n1. `ModuleGraph._safe_import_hook()`, the highest-level `ModuleGraph` method with respect to anything PyInstaller hooks would care about. Like the high-level `RedBlackTree.addNode()` method safely hooked above, this method preserves all data integrity constraints and is thus equally safely hookable.\r\n1. `ModuleGraph._import_hook()`, a slightly lower-level method. While this method _also_ preserves all data integrity constraints and is thus safely hookable, it's a bit less intelligent than the parent `_safe_import_hook()` method. For example, since SWIG detection resides in `_safe_import_hook()` rather than `_import_hook()`, hooking the importation of SWIG-generated modules and C extensions  requires `_safe_import_hook()` rather than `_import_hook()` hooks.\r\n1. `ModuleGraph._import_module()`, an even lower-level method. This method technically preserves the \"module graph invariant,\" but fails to add edges from the importing module to the imported module – implying the imported module and all modules imported by that module may or may not be unreachable from the \"root\" graph node immediately after this call, implying that these modules may or may not be iteratable via the commonly called `ModuleGraph.flatten()` method at that point. Fortunately, that _usually_ doesn't matter for low-level hooks. Moreover, there exist justifiable reasons for hooking this method. It's the natural place to detect and handle attempts to import **runtime modules** (i.e., modules that are dynamically constructed at runtime and hence _not_ scannable by `ModuleGraph`), for example, which is why we hook this method to handle the runtime `six.moves` module. Perfectly safe, as long as we don't make overly broad assumptions about connectedness.\r\n1. `ModuleGraph._load_module()` and `ModuleGraph._load_package()`, even lower-level methods with similar safety concerns. Both methods technically preserve the \"module graph invariant,\" but fail to add edges from the package of the imported module to that module. Whereas running a hook immediately after `_import_module()` merely increases the likelihood that the imported module will be unreachable from the root, running a hook immediately after `_load_module()` _guarantees_ unreachability. Again, that _usually_ doesn't matter for low-level hooks. Again, there exist justifiable reasons for hooking `_load_package()` in particular. It's the natural place to detect and handle attempts to modify a package's `__path__` attribute, for example, which is why we hook this method to handle the `distutils` package for #1420. Perfectly safe, as long as we don't make overly broad assumptions about connectedness.\r\n\r\n**That's it.**\r\n\r\nSince `_load_module()` recursively calls `_scan_code()`, module graph recursion effectively \"bottoms out\" at `_load_module()`. This means that even lower-level methods than `_load_module()` (e.g., `addNode()`, `createNode()`) are _not_ safely hookable by external users. _Maybe_ by official PyInstaller hooks if they really think they know what they're doing... but probably not even then.\r\n\r\nImportantly, this implies that **no `ObjectGraph()` methods are safely hookable.**"}}