{"action":"created","issue":{"url":"https://api.github.com/repos/cms-sw/cmssw/issues/9649","labels_url":"https://api.github.com/repos/cms-sw/cmssw/issues/9649/labels{/name}","comments_url":"https://api.github.com/repos/cms-sw/cmssw/issues/9649/comments","events_url":"https://api.github.com/repos/cms-sw/cmssw/issues/9649/events","html_url":"https://github.com/cms-sw/cmssw/pull/9649","id":88867112,"number":9649,"title":"Fix edmPythonSearch to also look in modules from process.load(...) ","user":{"login":"mark-grimes","id":6480160,"avatar_url":"https://avatars.githubusercontent.com/u/6480160?v=3","gravatar_id":"","url":"https://api.github.com/users/mark-grimes","html_url":"https://github.com/mark-grimes","followers_url":"https://api.github.com/users/mark-grimes/followers","following_url":"https://api.github.com/users/mark-grimes/following{/other_user}","gists_url":"https://api.github.com/users/mark-grimes/gists{/gist_id}","starred_url":"https://api.github.com/users/mark-grimes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mark-grimes/subscriptions","organizations_url":"https://api.github.com/users/mark-grimes/orgs","repos_url":"https://api.github.com/users/mark-grimes/repos","events_url":"https://api.github.com/users/mark-grimes/events{/privacy}","received_events_url":"https://api.github.com/users/mark-grimes/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/cms-sw/cmssw/labels/comparison-available","name":"comparison-available","color":"009800"},{"url":"https://api.github.com/repos/cms-sw/cmssw/labels/core-approved","name":"core-approved","color":"009800"},{"url":"https://api.github.com/repos/cms-sw/cmssw/labels/fully-signed","name":"fully-signed","color":"009800"},{"url":"https://api.github.com/repos/cms-sw/cmssw/labels/orp-pending","name":"orp-pending","color":"fbca04"},{"url":"https://api.github.com/repos/cms-sw/cmssw/labels/tests-approved","name":"tests-approved","color":"009800"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/cms-sw/cmssw/milestones/51","html_url":"https://github.com/cms-sw/cmssw/milestones/Next%20CMSSW_7_5_X","labels_url":"https://api.github.com/repos/cms-sw/cmssw/milestones/51/labels","id":982516,"number":51,"title":"Next CMSSW_7_5_X","description":"","creator":{"login":"nclopezo","id":1944922,"avatar_url":"https://avatars.githubusercontent.com/u/1944922?v=3","gravatar_id":"","url":"https://api.github.com/users/nclopezo","html_url":"https://github.com/nclopezo","followers_url":"https://api.github.com/users/nclopezo/followers","following_url":"https://api.github.com/users/nclopezo/following{/other_user}","gists_url":"https://api.github.com/users/nclopezo/gists{/gist_id}","starred_url":"https://api.github.com/users/nclopezo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nclopezo/subscriptions","organizations_url":"https://api.github.com/users/nclopezo/orgs","repos_url":"https://api.github.com/users/nclopezo/repos","events_url":"https://api.github.com/users/nclopezo/events{/privacy}","received_events_url":"https://api.github.com/users/nclopezo/received_events","type":"User","site_admin":false},"open_issues":67,"closed_issues":903,"state":"open","created_at":"2015-02-17T08:21:25Z","updated_at":"2015-06-17T15:33:54Z","due_on":"2015-05-31T22:00:00Z","closed_at":null},"comments":9,"created_at":"2015-06-16T23:36:01Z","updated_at":"2015-06-17T16:24:55Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/cms-sw/cmssw/pulls/9649","html_url":"https://github.com/cms-sw/cmssw/pull/9649","diff_url":"https://github.com/cms-sw/cmssw/pull/9649.diff","patch_url":"https://github.com/cms-sw/cmssw/pull/9649.patch"},"body":"Currently the edmPythonSearch tool doesn't include modules imported with `process.load( <module> )`, which makes it almost useless.  This adds a custom override of the modulefinder opcode scanner which searches for `process.load(...)` statements, as well as the normal `import <module>` type statements.\r\n\r\nIt should work regardless of the name of the Process object, and regardless of how `FWCore.ParameterSet.Config` was imported (e.g. `import FWCore.ParameterSet.Config` or `import FWCore.ParameterSet.Config as foo`)\r\n\r\nAs an example, step3 of workflow 1306.0 (and probably all others) uses \"process.load\" to import `Configuration.StandardSequences.Services_cff`, which (as an arbitrary example) contains the comment:\r\n\r\n    #an \"intermediate layer\" remains, just in case somebody is using it...\r\n\r\nBefore this pull request\r\n\r\n    edmPythonSearch \"just in case somebody\" 1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15+MINIAODMCUP15/step3_RAW2DIGI_L1Reco_RECO_EI_VALIDATION_DQM.py\r\n\r\nyields no results at all.\r\n\r\nAfter this pull request it gives:\r\n\r\n    #an \"intermediate layer\" remains, just in case somebody is using it...\r\n    Configuration.StandardSequences.Services_cff (line: 10)\r\n    From 1306.0.step3_RAW2DIGI_L1Reco_RECO_EI_VALIDATION_DQM\r\n\r\nExamples are easy to find where patterns aren't found as the code stands before this pull request.\r\nPretty sure this only affects the edmPythonSearch tool, I can't see how any reconstruction/simulation code would be affected.\r\nMost of this code was based on the modulefinder source (https://hg.python.org/cpython/file/2.7/Lib/modulefinder.py) and some useful information at http://security.coverity.com/blog/2014/Nov/understanding-python-bytecode.html."},"comment":{"url":"https://api.github.com/repos/cms-sw/cmssw/issues/comments/112868469","html_url":"https://github.com/cms-sw/cmssw/pull/9649#issuecomment-112868469","issue_url":"https://api.github.com/repos/cms-sw/cmssw/issues/9649","id":112868469,"user":{"login":"mark-grimes","id":6480160,"avatar_url":"https://avatars.githubusercontent.com/u/6480160?v=3","gravatar_id":"","url":"https://api.github.com/users/mark-grimes","html_url":"https://github.com/mark-grimes","followers_url":"https://api.github.com/users/mark-grimes/followers","following_url":"https://api.github.com/users/mark-grimes/following{/other_user}","gists_url":"https://api.github.com/users/mark-grimes/gists{/gist_id}","starred_url":"https://api.github.com/users/mark-grimes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mark-grimes/subscriptions","organizations_url":"https://api.github.com/users/mark-grimes/orgs","repos_url":"https://api.github.com/users/mark-grimes/repos","events_url":"https://api.github.com/users/mark-grimes/events{/privacy}","received_events_url":"https://api.github.com/users/mark-grimes/received_events","type":"User","site_admin":false},"created_at":"2015-06-17T16:24:55Z","updated_at":"2015-06-17T16:24:55Z","body":"hold\r\n\r\nI've got an update coming which should fix the problem above.  It comes about because I currently check for cms.Process declarations and then search for \"load\" method calls on any of those objects.  In customisation functions the process object is passed as a function parameter, so isn't actually declared in that file, so there are no checks for the \"load\" calls.\r\n\r\nCurrent plan is to look for any time a \"load\" call with a single argument is made on **any** object.  Then check if the argument is a string, then whether the string refers to a python module.  Then **assume** that the load call was on a Process object.  I don't think this is a problem since it's unlikely to be anything else and the worst thing that can happen is extra search results.\r\nI could also check if the object is named \"process\", but I've been avoiding that because it doesn't necessarily have to be named that.\r\n\r\nAnother thing I've realised it will miss is if a variable is used in th"}}